Sensory cortex is optimised for prediction of future input

Neurons in sensory cortex are tuned to diverse features in natural scenes. But what determines which features neurons become selective to? Here we explore the idea that neuronal selectivity is optimised to represent features in the recent past of sensory input that best predict immediate future inputs. We tested this hypothesis using simple feedforward neural networks, which were trained to predict the next few video or audio frames in clips of natural scenes. The networks developed receptive fields that closely matched those of real cortical neurons, including the oriented spatial tuning of primary visual cortex, the frequency selectivity of primary auditory cortex and, most notably, in their temporal tuning properties. Furthermore, the better a network predicted future inputs the more closely its receptive fields tended to resemble those in the brain. This suggests that sensory processing is optimised to extract those features with the most capacity to predict future input.