Skip to main content
bioRxiv
  • Home
  • About
  • Submit
  • ALERTS / RSS
Advanced Search
New Results

Voxel-to-voxel predictive models reveal unexpected structure in unexplained variance

Maggie Mae Mell, Ghislain St-Yves, Thomas Naselaris
doi: https://doi.org/10.1101/692319
Maggie Mae Mell
Medical University of South Carolina Department of Neuroscience, 173 Ashley Avenue Charleston, SC 29425 USA
  • Find this author on Google Scholar
  • Find this author on PubMed
  • Search for this author on this site
  • For correspondence: maggiemaemell@gmail.com
Ghislain St-Yves
Medical University of South Carolina Department of Neuroscience, 173 Ashley Avenue Charleston, SC 29425 USA
  • Find this author on Google Scholar
  • Find this author on PubMed
  • Search for this author on this site
Thomas Naselaris
Medical University of South Carolina Department of Neuroscience, 173 Ashley Avenue Charleston, SC 29425 USA
  • Find this author on Google Scholar
  • Find this author on PubMed
  • Search for this author on this site
  • Abstract
  • Full Text
  • Info/History
  • Metrics
  • Data/Code
  • Preview PDF
Loading

Abstract

Encoding models based on deep convolutional neural networks (DCNN) more accurately predict BOLD responses to natural scenes in the visual system than any other currently available model. However, DCNN-based encoding models fail to predict a significant amount of variance in the activity of most voxels in all visual areas. This failure could reflect limitations in the data (e.g., a noise ceiling), or could reflect limitations of the DCNN as a model of computation in the brain. Understanding the source and structure of the unexplained variance could therefore provide helpful clues for improving models of brain computation. Here, we characterize the structure of the variance that DCNN-based encoding models cannot explain. Using a publicly available dataset of BOLD responses to natural scenes, we determined if the source of unexplained variance was shared across voxels, individual brains, retinotopic locations, and hierarchically distant visual brain areas. We answered these questions using voxel-to-voxel (vox2vox) models that predict activity in a target voxel given activity in a population of source voxels. We found that simple linear vox2vox models increased within-subject prediction accuracy over DCNN-based models for any pair of source/target visual areas, clearly demonstrating that the source of unexplained variance is widely shared within and across visual brain areas. However, vox2vox models were not more accurate than DCNN-based models when source and target voxels came from separate brains, demonstrating that the source of unexplained variance was not shared across brains. Furthermore, the weights of these vox2vox models permitted explicit readout of the receptive field location of target voxels, demonstrating that the source of unexplained variance induces correlations primarily between the activities of voxels with overlapping receptive fields. Finally, we found that vox2vox model prediction accuracy was heavily dependent upon the signed hierarchical distance between the source and target voxels: for feed-forward models (source area lower in the visual hierarchy than target area) prediction accuracy decreased with hierarchical distance between source and target. It did not decrease for feedback models. In contrast, the same analysis applied across layers of a DCNN did not reveal this feed-forward/feedback asymmetry. Given these results, we argue that the structured variance unexplained by DCNN-based encoding models is unlikely to be entirely caused by spatially correlated noise or eye movements; rather, our results point to a need for brain models that include endogenous dynamics and a pattern of connectivity that is not strictly feed-forward.

Footnotes

  • https://crcns.org/data-sets/vc/vim-1

Copyright 
The copyright holder for this preprint is the author/funder, who has granted bioRxiv a license to display the preprint in perpetuity. It is made available under a CC-BY-NC-ND 4.0 International license.
Back to top
PreviousNext
Posted July 04, 2019.
Download PDF
Data/Code
Email

Thank you for your interest in spreading the word about bioRxiv.

NOTE: Your email address is requested solely to identify you as the sender of this article.

Enter multiple addresses on separate lines or separate them with commas.
Voxel-to-voxel predictive models reveal unexpected structure in unexplained variance
(Your Name) has forwarded a page to you from bioRxiv
(Your Name) thought you would like to see this page from the bioRxiv website.
CAPTCHA
This question is for testing whether or not you are a human visitor and to prevent automated spam submissions.
Share
Voxel-to-voxel predictive models reveal unexpected structure in unexplained variance
Maggie Mae Mell, Ghislain St-Yves, Thomas Naselaris
bioRxiv 692319; doi: https://doi.org/10.1101/692319
Digg logo Reddit logo Twitter logo CiteULike logo Facebook logo Google logo Mendeley logo
Citation Tools
Voxel-to-voxel predictive models reveal unexpected structure in unexplained variance
Maggie Mae Mell, Ghislain St-Yves, Thomas Naselaris
bioRxiv 692319; doi: https://doi.org/10.1101/692319

Citation Manager Formats

  • BibTeX
  • Bookends
  • EasyBib
  • EndNote (tagged)
  • EndNote 8 (xml)
  • Medlars
  • Mendeley
  • Papers
  • RefWorks Tagged
  • Ref Manager
  • RIS
  • Zotero
  • Tweet Widget
  • Facebook Like
  • Google Plus One

Subject Area

  • Neuroscience
Subject Areas
All Articles
  • Animal Behavior and Cognition (2633)
  • Biochemistry (5220)
  • Bioengineering (3643)
  • Bioinformatics (15707)
  • Biophysics (7210)
  • Cancer Biology (5590)
  • Cell Biology (8039)
  • Clinical Trials (138)
  • Developmental Biology (4731)
  • Ecology (7458)
  • Epidemiology (2059)
  • Evolutionary Biology (10518)
  • Genetics (7695)
  • Genomics (10079)
  • Immunology (5144)
  • Microbiology (13819)
  • Molecular Biology (5350)
  • Neuroscience (30571)
  • Paleontology (211)
  • Pathology (870)
  • Pharmacology and Toxicology (1519)
  • Physiology (2233)
  • Plant Biology (4980)
  • Scientific Communication and Education (1036)
  • Synthetic Biology (1379)
  • Systems Biology (4129)
  • Zoology (802)