Skip to main content
bioRxiv
  • Home
  • About
  • Submit
  • ALERTS / RSS
Advanced Search
New Results

Modality-independent coding of scene categories in prefrontal cortex

Yaelan Jung, Bart Larsen, Dirk B. Walther
doi: https://doi.org/10.1101/142562
Yaelan Jung
11Department of Psychology, University of Toronto.
  • Find this author on Google Scholar
  • Find this author on PubMed
  • Search for this author on this site
Bart Larsen
21Department of Psychology, University of Toronto.
  • Find this author on Google Scholar
  • Find this author on PubMed
  • Search for this author on this site
Dirk B. Walther
11Department of Psychology, University of Toronto.
  • Find this author on Google Scholar
  • Find this author on PubMed
  • Search for this author on this site
  • Abstract
  • Full Text
  • Info/History
  • Metrics
  • Preview PDF
Loading

Abstract

Natural environments convey information through multiple sensory modalities, all of which contribute to people’s percepts. Although it has been shown that visual or auditory content of scene categories can be decoded from brain activity, it remains unclear where and how humans integrate different sensory inputs and represent scene information beyond a specific sensory modality domain. To address this question, we investigated how categories of scene images and sounds are represented in several brain regions. A mixed gender group of healthy human subjects participated the present study, where their brain activity was measured with fMRI while viewing images or listening to sounds of different places. We found that both visual and auditory scene categories can be decoded not only from modality-specific areas, but also from several brain regions in the temporal, parietal, and prefrontal cortex. Intriguingly, only in the prefrontal cortex, but not in any other regions, categories of scene images and sounds appear to be represented in similar activation patterns, suggesting that scene representations in the prefrontal cortex are modality-independent. Furthermore, the error patterns of neural decoders indicate that category-specific neural activity patterns in the middle and superior frontal gyri are tightly linked to categorization behavior. Our findings demonstrate that complex scene information is represented at an abstract level in the prefrontal cortex, regardless of the sensory modality of the stimulus.

Statement of Significance Our experience in daily life requires the integration of multiple sensory inputs such as images, sounds, or scents from the environment. Here, for the first time, we investigated where and how in the brain information about the natural environment from multiple senses is merged to form modality-independent representations of scene categories. We show direct decoding of scene categories across sensory modalities from patterns of neural activity in the prefrontal cortex. We also conclusively tie these neural representations to human categorization behavior based on the errors from the neural decoder and behavior. Our findings suggest that the prefrontal cortex is a central hub for integrating sensory information and computing modality-independent representations of scene categories.

Acknowledgements

We thank Michael Mack and Heeyoung Choo for their helpful comments on the early version of this manuscript. This work is supported by NSERC Discovery Grant (#498390) and Canadian Foundation for Innovation (#32896).

Footnotes

  • Conflict of interest: The authors declare no conflict of interest.

Copyright 
The copyright holder for this preprint is the author/funder, who has granted bioRxiv a license to display the preprint in perpetuity. It is made available under a CC-BY-NC-ND 4.0 International license.
Back to top
PreviousNext
Posted February 20, 2018.
Download PDF
Email

Thank you for your interest in spreading the word about bioRxiv.

NOTE: Your email address is requested solely to identify you as the sender of this article.

Enter multiple addresses on separate lines or separate them with commas.
Modality-independent coding of scene categories in prefrontal cortex
(Your Name) has forwarded a page to you from bioRxiv
(Your Name) thought you would like to see this page from the bioRxiv website.
CAPTCHA
This question is for testing whether or not you are a human visitor and to prevent automated spam submissions.
Share
Modality-independent coding of scene categories in prefrontal cortex
Yaelan Jung, Bart Larsen, Dirk B. Walther
bioRxiv 142562; doi: https://doi.org/10.1101/142562
Digg logo Reddit logo Twitter logo CiteULike logo Facebook logo Google logo Mendeley logo
Citation Tools
Modality-independent coding of scene categories in prefrontal cortex
Yaelan Jung, Bart Larsen, Dirk B. Walther
bioRxiv 142562; doi: https://doi.org/10.1101/142562

Citation Manager Formats

  • BibTeX
  • Bookends
  • EasyBib
  • EndNote (tagged)
  • EndNote 8 (xml)
  • Medlars
  • Mendeley
  • Papers
  • RefWorks Tagged
  • Ref Manager
  • RIS
  • Zotero
  • Tweet Widget
  • Facebook Like
  • Google Plus One

Subject Area

  • Neuroscience
Subject Areas
All Articles
  • Animal Behavior and Cognition (2633)
  • Biochemistry (5221)
  • Bioengineering (3643)
  • Bioinformatics (15709)
  • Biophysics (7213)
  • Cancer Biology (5592)
  • Cell Biology (8042)
  • Clinical Trials (138)
  • Developmental Biology (4732)
  • Ecology (7458)
  • Epidemiology (2059)
  • Evolutionary Biology (10518)
  • Genetics (7697)
  • Genomics (10081)
  • Immunology (5148)
  • Microbiology (13823)
  • Molecular Biology (5354)
  • Neuroscience (30576)
  • Paleontology (211)
  • Pathology (871)
  • Pharmacology and Toxicology (1519)
  • Physiology (2234)
  • Plant Biology (4983)
  • Scientific Communication and Education (1036)
  • Synthetic Biology (1379)
  • Systems Biology (4130)
  • Zoology (803)