Skip to main content
bioRxiv
  • Home
  • About
  • Submit
  • ALERTS / RSS
Advanced Search
New Results

Deep Residual Network Reveals a Nested Hierarchy of Distributed Cortical Representation for Visual Categorization

Haiguang Wen, Junxing Shi, Wei Chen, Zhongming Liu
doi: https://doi.org/10.1101/151142
Haiguang Wen
2School of Electrical and Computer Engineering, Purdue University, West Lafayette, IN, USA
3Purdue Institute for Integrative Neuroscience, Purdue University, West Lafayette, IN, USA
  • Find this author on Google Scholar
  • Find this author on PubMed
  • Search for this author on this site
Junxing Shi
2School of Electrical and Computer Engineering, Purdue University, West Lafayette, IN, USA
3Purdue Institute for Integrative Neuroscience, Purdue University, West Lafayette, IN, USA
  • Find this author on Google Scholar
  • Find this author on PubMed
  • Search for this author on this site
Wei Chen
4Center for Magnetic Resonance Research, Department of Radiology, University of Minnesota Medical School, Minneapolis, MN, USA
  • Find this author on Google Scholar
  • Find this author on PubMed
  • Search for this author on this site
Zhongming Liu
1Weldon School of Biomedical Engineering, Purdue University, West Lafayette, IN, USA
2School of Electrical and Computer Engineering, Purdue University, West Lafayette, IN, USA
3Purdue Institute for Integrative Neuroscience, Purdue University, West Lafayette, IN, USA
  • Find this author on Google Scholar
  • Find this author on PubMed
  • Search for this author on this site
  • For correspondence: zmliu@purdue.edu
  • Abstract
  • Full Text
  • Info/History
  • Metrics
  • Supplementary material
  • Preview PDF
Loading

Abstract

What enables humans to readily recognize visual objects is attributed to how the brain extracts, represents, and organizes object information from visual input. To understand this process, we used a deep residual network as a hierarchical computational model of visual categorization to predict cortical representation of natural vision. We trained and tested such a predictive encoding model with functional magnetic resonance imaging data from human subjects watching hours of natural video clips, and verified its ability to predict cortical responses to novel images, objects, and categories. We further used the so trained encoding model to synthesize cortical responses to 64,000 visual objects from 80 categories, revealing hierarchical, distributed, and overlapping cortical representations of categories. Such category representations covered both the ventral and dorsal pathways, reflected multiple levels and domains of visual features, and preserved semantic relationships between categories. In the scale of the entire visual cortex, category representations were modularly organized into three categories: biological objects, non-biological objects, and background scenes. In a smaller scale specific to each module, category representation further revealed sub-modules for finer categorization, e.g. biological objects were categorized into terrestrial animals, aquatic animals, humans, and plants. Such nested spatial and representational hierarchies were attributable to different levels of category information to varying degrees. These findings suggest that increasingly more specific category information is represented by cortical patterns in progressively finer spatial scales – an important principle for the brain to categorize visual objects in various levels of abstraction.

Significance Statement This study uses a deep neural network to model how the brain extracts and represents the information from visual objects for efficient and flexible categorization. Results show that the model can predict widespread cortical responses when humans are viewing natural video clips, and generate patterns of cortical responses to tens of thousands of objects from 80 categories. Analysis of these response patterns reveals that the brain organizes distributed, overlapping, and property-based category representations into a nested hierarchy. Increasingly more specific category information is represented by cortical patterns in progressively finer spatial scales. This nested hierarchy may be a fundamental principle for the brain to categorize visual objects in various levels of specificity.

Copyright 
The copyright holder for this preprint is the author/funder, who has granted bioRxiv a license to display the preprint in perpetuity. All rights reserved. No reuse allowed without permission.
Back to top
PreviousNext
Posted June 21, 2017.
Download PDF

Supplementary Material

Email

Thank you for your interest in spreading the word about bioRxiv.

NOTE: Your email address is requested solely to identify you as the sender of this article.

Enter multiple addresses on separate lines or separate them with commas.
Deep Residual Network Reveals a Nested Hierarchy of Distributed Cortical Representation for Visual Categorization
(Your Name) has forwarded a page to you from bioRxiv
(Your Name) thought you would like to see this page from the bioRxiv website.
CAPTCHA
This question is for testing whether or not you are a human visitor and to prevent automated spam submissions.
Share
Deep Residual Network Reveals a Nested Hierarchy of Distributed Cortical Representation for Visual Categorization
Haiguang Wen, Junxing Shi, Wei Chen, Zhongming Liu
bioRxiv 151142; doi: https://doi.org/10.1101/151142
Digg logo Reddit logo Twitter logo CiteULike logo Facebook logo Google logo Mendeley logo
Citation Tools
Deep Residual Network Reveals a Nested Hierarchy of Distributed Cortical Representation for Visual Categorization
Haiguang Wen, Junxing Shi, Wei Chen, Zhongming Liu
bioRxiv 151142; doi: https://doi.org/10.1101/151142

Citation Manager Formats

  • BibTeX
  • Bookends
  • EasyBib
  • EndNote (tagged)
  • EndNote 8 (xml)
  • Medlars
  • Mendeley
  • Papers
  • RefWorks Tagged
  • Ref Manager
  • RIS
  • Zotero
  • Tweet Widget
  • Facebook Like
  • Google Plus One

Subject Area

  • Neuroscience
Subject Areas
All Articles
  • Animal Behavior and Cognition (2408)
  • Biochemistry (4756)
  • Bioengineering (3294)
  • Bioinformatics (14573)
  • Biophysics (6586)
  • Cancer Biology (5125)
  • Cell Biology (7365)
  • Clinical Trials (138)
  • Developmental Biology (4308)
  • Ecology (6817)
  • Epidemiology (2057)
  • Evolutionary Biology (9836)
  • Genetics (7305)
  • Genomics (9463)
  • Immunology (4502)
  • Microbiology (12580)
  • Molecular Biology (4897)
  • Neuroscience (28074)
  • Paleontology (198)
  • Pathology (796)
  • Pharmacology and Toxicology (1372)
  • Physiology (1993)
  • Plant Biology (4447)
  • Scientific Communication and Education (965)
  • Synthetic Biology (1293)
  • Systems Biology (3889)
  • Zoology (716)