Skip to main content
bioRxiv
  • Home
  • About
  • Submit
  • ALERTS / RSS
Advanced Search
New Results

Deep convolutional models improve predictions of macaque V1 responses to natural images

Santiago A. Cadena, George H. Denfield, View ORCID ProfileEdgar Y. Walker, Leon A. Gatys, Andreas S. Tolias, Matthias Bethge, Alexander S. Ecker
doi: https://doi.org/10.1101/201764
Santiago A. Cadena
1Centre for Integrative Neuroscience and Institute for Theoretical Physics, University of Tübingen, 72076 Tübingen, Germany
3Bernstein Center for Computational Neuroscience, 72076 Tübingen, Germany
6Center for Neuroscience and Artificial Intelligence, BCM. 77030 Houston, TX, USA
  • Find this author on Google Scholar
  • Find this author on PubMed
  • Search for this author on this site
George H. Denfield
4Department of Neuroscience, Baylor College of Medicine, 77030 Houston, TX, USA
6Center for Neuroscience and Artificial Intelligence, BCM. 77030 Houston, TX, USA
  • Find this author on Google Scholar
  • Find this author on PubMed
  • Search for this author on this site
Edgar Y. Walker
4Department of Neuroscience, Baylor College of Medicine, 77030 Houston, TX, USA
6Center for Neuroscience and Artificial Intelligence, BCM. 77030 Houston, TX, USA
  • Find this author on Google Scholar
  • Find this author on PubMed
  • Search for this author on this site
  • ORCID record for Edgar Y. Walker
Leon A. Gatys
1Centre for Integrative Neuroscience and Institute for Theoretical Physics, University of Tübingen, 72076 Tübingen, Germany
3Bernstein Center for Computational Neuroscience, 72076 Tübingen, Germany
  • Find this author on Google Scholar
  • Find this author on PubMed
  • Search for this author on this site
Andreas S. Tolias
3Bernstein Center for Computational Neuroscience, 72076 Tübingen, Germany
4Department of Neuroscience, Baylor College of Medicine, 77030 Houston, TX, USA
5Department of Electrical and Computer Engineering, Rice University, 77030 Houston, TX, USA
6Center for Neuroscience and Artificial Intelligence, BCM. 77030 Houston, TX, USA
  • Find this author on Google Scholar
  • Find this author on PubMed
  • Search for this author on this site
Matthias Bethge
1Centre for Integrative Neuroscience and Institute for Theoretical Physics, University of Tübingen, 72076 Tübingen, Germany
2Max Planck Institute for Biological Cybernetics, 72076 Tübingen, Germany
3Bernstein Center for Computational Neuroscience, 72076 Tübingen, Germany
6Center for Neuroscience and Artificial Intelligence, BCM. 77030 Houston, TX, USA
  • Find this author on Google Scholar
  • Find this author on PubMed
  • Search for this author on this site
Alexander S. Ecker
1Centre for Integrative Neuroscience and Institute for Theoretical Physics, University of Tübingen, 72076 Tübingen, Germany
3Bernstein Center for Computational Neuroscience, 72076 Tübingen, Germany
6Center for Neuroscience and Artificial Intelligence, BCM. 77030 Houston, TX, USA
  • Find this author on Google Scholar
  • Find this author on PubMed
  • Search for this author on this site
  • Abstract
  • Full Text
  • Info/History
  • Metrics
  • Preview PDF
Loading

Abstract

Despite great efforts over several decades, our best models of primary visual cortex (V1) still predict spiking activity quite poorly when probed with natural stimuli, highlighting our limited understanding of the nonlinear computations in V1. Recently, two approaches based on deep learning have been successfully applied to neural data: On the one hand, transfer learning from networks trained on object recognition worked remarkably well for predicting neural responses in higher areas of the primate ventral stream, but has not yet been used to model spiking activity in early stages such as V1. On the other hand, data-driven models have been used to predict neural responses in the early visual system (retina and V1) of mice, but not primates. Here, we test the ability of both approaches to predict spiking activity in response to natural images in V1 of awake monkeys. Even though V1 is rather at an early to intermediate stage of the visual system, we found that the transfer learning approach performed similarly well to the data-driven approach and both outperformed classical linear-nonlinear and wavelet-based feature representations that build on existing theories of V1. Notably, transfer learning using a pre-trained feature space required substantially less experimental time to achieve the same performance. In conclusion, multi-layer convolutional neural networks (CNNs) set the new state of the art for predicting neural responses to natural images in primate V1 and deep features learned for object recognition are better explanations for V1 computation than all previous filter bank theories. This finding strengthens the necessity of V1 models that are multiple nonlinearities away from the image domain and it supports the idea of explaining early visual cortex based on high-level functional goals.

Author summary Predicting the responses of sensory neurons to arbitrary natural stimuli is of major importance for understanding their function. Arguably the most studied cortical area is primary visual cortex (V1), where many models have been developed to explain its function. However, the most successful models built on neurophysiologists’ intuitions still fail to account for spiking responses to natural images. Here, we model spiking activity in primary visual cortex (V1) of monkeys using deep convolutional neural networks (CNNs), which have been successful in computer vision. We both trained CNNs directly to fit the data, and used CNNs trained to solve a high-level task (object categorization). With these approaches, we are able to outperform previous models and improve the state of the art in predicting the responses of early visual neurons to natural images. Our results have two important implications. First, since V1 is the result of several nonlinear stages, it should be modeled as such. Second, functional models of entire visual pathways, of which V1 is an early stage, do not only account for higher areas of such pathways, but also provide useful representations for V1 predictions.

Footnotes

  • * santiago.cadena{at}uni-tuebingen.de

Copyright 
The copyright holder for this preprint is the author/funder, who has granted bioRxiv a license to display the preprint in perpetuity. It is made available under a CC-BY 4.0 International license.
Back to top
PreviousNext
Posted November 05, 2018.
Download PDF
Email

Thank you for your interest in spreading the word about bioRxiv.

NOTE: Your email address is requested solely to identify you as the sender of this article.

Enter multiple addresses on separate lines or separate them with commas.
Deep convolutional models improve predictions of macaque V1 responses to natural images
(Your Name) has forwarded a page to you from bioRxiv
(Your Name) thought you would like to see this page from the bioRxiv website.
CAPTCHA
This question is for testing whether or not you are a human visitor and to prevent automated spam submissions.
Share
Deep convolutional models improve predictions of macaque V1 responses to natural images
Santiago A. Cadena, George H. Denfield, Edgar Y. Walker, Leon A. Gatys, Andreas S. Tolias, Matthias Bethge, Alexander S. Ecker
bioRxiv 201764; doi: https://doi.org/10.1101/201764
Reddit logo Twitter logo Facebook logo LinkedIn logo Mendeley logo
Citation Tools
Deep convolutional models improve predictions of macaque V1 responses to natural images
Santiago A. Cadena, George H. Denfield, Edgar Y. Walker, Leon A. Gatys, Andreas S. Tolias, Matthias Bethge, Alexander S. Ecker
bioRxiv 201764; doi: https://doi.org/10.1101/201764

Citation Manager Formats

  • BibTeX
  • Bookends
  • EasyBib
  • EndNote (tagged)
  • EndNote 8 (xml)
  • Medlars
  • Mendeley
  • Papers
  • RefWorks Tagged
  • Ref Manager
  • RIS
  • Zotero
  • Tweet Widget
  • Facebook Like
  • Google Plus One

Subject Area

  • Neuroscience
Subject Areas
All Articles
  • Animal Behavior and Cognition (4682)
  • Biochemistry (10357)
  • Bioengineering (7670)
  • Bioinformatics (26332)
  • Biophysics (13523)
  • Cancer Biology (10683)
  • Cell Biology (15438)
  • Clinical Trials (138)
  • Developmental Biology (8497)
  • Ecology (12821)
  • Epidemiology (2067)
  • Evolutionary Biology (16853)
  • Genetics (11399)
  • Genomics (15478)
  • Immunology (10616)
  • Microbiology (25208)
  • Molecular Biology (10220)
  • Neuroscience (54465)
  • Paleontology (401)
  • Pathology (1668)
  • Pharmacology and Toxicology (2897)
  • Physiology (4342)
  • Plant Biology (9245)
  • Scientific Communication and Education (1586)
  • Synthetic Biology (2557)
  • Systems Biology (6780)
  • Zoology (1466)