Abstract
Deep neural networks (DNNs) for object classification have been argued to provide the most promising model of the visual system, accompanied by claims that they have attained or even surpassed human-level performance. Here, we evaluated whether DNNs provide a viable model of human vision when tested with challenging noisy images of objects, sometimes presented at the very limits of visibility. We show that popular state-of-the-art DNNs perform in a qualitatively different manner than humans – they are unusually susceptible to spatially uncorrelated white noise and less impaired by spatially correlated noise. We implemented a noise-training procedure to determine whether noise-trained DNNs exhibit more robust responses that better match human behavioral and neural performance. We found that noise-trained DNNs provide a better qualitative match to human performance; moreover, they reliably predict human recognition thresholds on an image-by-image basis. Functional neuroimaging revealed that noise-trained DNNs provide a better correspondence to the pattern-specific neural representations found in both early visual areas and high-level object areas. A layer-specific analysis of the DNNs indicated that noise training led to broad-ranging modifications throughout the network, with greater benefits of noise robustness accruing in progressively higher layers. Our findings demonstrate that noise-trained DNNs provide a viable model to account for human behavioral and neural responses to objects in challenging noisy viewing conditions. Further, they suggest that robustness to noise may be acquired through a process of visual learning.
Competing Interest Statement
The authors (FT and HJ) have submitted a non-provisional utilty application to the U.S. Patent and Trademark Office with respect to the noise-training methods used in this study to train deep neural networks.
Footnotes
The manuscript has been revised to clarify the novel contributions of this study and to provide more background information on related studies. It also includes some additional analyses and new figures to provide a more comprehensive portrayal of the study's findings.