Abstract
The first steps of visual processing are often described as a bank of oriented filters followed by divisive normalization. This approach has been tremendously successful at predicting contrast thresholds in simple visual displays. However, it is unclear to what extent this kind of architecture also supports processing in more complex visual tasks being performed in naturally looking images.
Here, we use a deep generative model for natural images to embed arc segments with different curvatures in naturalistic images. Specifically, these images contain the target as part of the image scene, resulting in considerable appearance variation of target as well as background. Three observers localized arc targets in these images, achieving an accuracy of 74.7% correct responses on average. Data were then fit by a number of biologically inspired models and also by a 5-layer convolutional neural network. Four models were particularly good a predicting observer responses, (i) a bank of oriented filters, similar to complex cells in primate area V1, (ii) a bank of oriented filters followed by tuned gain control, incorporating knowledge about cortical surround interactions, (iii) a bank of oriented filters followed by local normalization, (iv) the 5-layer convolutional neural network. A control experiment with optimized stimuli based on these four models showed that the observers’ data were best explained by model (ii) with tuned gain control.
These data suggest that standard models of early vision provide good descriptions of performance in much more complex tasks than what they were designed for, while general purpose non-linear models such as convolutional neural networks do not.
Author summary Early stages of visual processing are often described as a bank of oriented filters followed by divisive normalization. While this standard model successfully predicts contrast thresholds in simple visual displays, it is unclear to what extent it also supports more complex tasks performed in naturally looking images. One challenge here is that naturalistic targets are not simply superimposed on the image, but they form part of an image. We use a high-capacity image model to generate random naturalistic images constrained to contain a visual target as part of the image. Human target detection performance on these images is indeed well described by variants of the filtering and normalization approach, but a convolutional neural network model based on modern methods from artificial intelligence does so equally well. Yet, the converse is not true; artificial stimuli constructed from the standard model drive human performance in meaningful ways, while artificial stimuli constructed from the convolutional neural network do not. Thus standard models of early vision indeed provide good descriptions of human performance in more natural stimulus conditions, while convolutional neural networks do not.
Footnotes
↵* ifruend{at}yorku.ca
