Abstract
A long-standing observation about primary visual cortex (V1) is that the stimulus selectivity of neurons can be well explained with a cascade of linear computations followed by a nonlinear rectification stage. This framework remains highly influential in systems neuroscience and has also inspired recent efforts in artificial intelligence. The success of these models include describing the disparity-selectivity of binocular neurons in V1. Some aspects of real neuronal disparity responses are hard to explain with simple linear-nonlinear models, notably the attenuated response of real cells to “anticorrelated” stimuli which violate natural binocular image statistics. General linear-nonlinear models can account for this attenuation, but no one has yet tested whether they quantitatively match the response of real neurons. Here, we exhaustively test this framework using recently developed optimisation techniques. We show that many cells are very poorly characterised by even general linear-nonlinear models. Strikingly, the models can account for neuronal responses to unnatural anticorrelated stimuli as well as to most natural, correlated stimuli. However, the models fail to capture the particularly strong response to binocularly correlated stimuli at the preferred disparity of the cell. Thus, V1 neurons perform an amplification of responses to correlated stimuli which cannot be accounted for by a linear-nonlinear cascade. The implication is that even simple stimulus selectivity in V1 requires more complex computations than previously envisaged.
Significance statement A long-standing question in sensory systems neuroscience is whether the computations performed by neurons in primary visual cortex can be described by repeated elements of linear-nonlinear units (a linear filtering/pooling stage followed by a subsequent output nonlinearity, such as a squaring). This question goes back to the Nobel-prize winning work by Hubel & Wiesel who argued that orientation selectivity in V1 can qualitatively be explained in this way. In this paper, we show that V1 neurons have an amplification of their response to stimuli which are contrast matched in the two eyes, and that the recovered models cannot describe this property. We argue that this likely represents more sophisticated computations than can be compactly described by the linear-nonlinear cascade framework.
