Skip to main content
bioRxiv
  • Home
  • About
  • Submit
  • ALERTS / RSS
Advanced Search
New Results

A State Representation for Reinforcement Learning and Decision-Making in the Orbitofrontal Cortex

View ORCID ProfileNicolas W. Schuck, Robert Wilson, View ORCID ProfileYael Niv
doi: https://doi.org/10.1101/210591
Nicolas W. Schuck
1Max Planck Research Group NeuroCode Max Planck Institute for Human Development 14195 Berlin, Germany
2Princeton Neuroscience Institute & Department of Psychology Princeton University Princeton, NJ, 08544, USA
  • Find this author on Google Scholar
  • Find this author on PubMed
  • Search for this author on this site
  • ORCID record for Nicolas W. Schuck
  • For correspondence: [email protected] [email protected]
Robert Wilson
1Max Planck Research Group NeuroCode Max Planck Institute for Human Development 14195 Berlin, Germany
  • Find this author on Google Scholar
  • Find this author on PubMed
  • Search for this author on this site
  • For correspondence: [email protected]
Yael Niv
2Princeton Neuroscience Institute & Department of Psychology Princeton University Princeton, NJ, 08544, USA
  • Find this author on Google Scholar
  • Find this author on PubMed
  • Search for this author on this site
  • ORCID record for Yael Niv
  • For correspondence: [email protected]
  • Abstract
  • Full Text
  • Info/History
  • Metrics
  • Preview PDF
Loading

Abstract

Despite decades of research, the exact ways in which the orbitofrontal cortex (OFC) influences cognitive function have remained mysterious. Anatomically, the OFC is characterized by remarkably broad connectivity to sensory, limbic and subcortical areas, and functional studies have implicated the OFC in a plethora of functions ranging from facial processing to value-guided choice. Notwithstanding such diversity of findings, much research suggests that one important function of the OFC is to support decision making and reinforcement learning. Here, we describe a novel theory that posits that OFC’s specific role in decision-making is to provide an up-to-date representation of task-related information, called a state representation. This representation reflects a mapping between distinct task states and sensory as well as unobservable information. We summarize evidence supporting the existence of such state representations in rodent and human OFC and argue that forming these state representations provides a crucial scaffold that allows animals to efficiently perform decision making and reinforcement learning in high-dimensional and partially observable environments. Finally, we argue that our theory offers an integrating framework for linking the diversity of functions ascribed to OFC and is in line with its wide ranging connectivity.

Copyright 
The copyright holder for this preprint is the author/funder, who has granted bioRxiv a license to display the preprint in perpetuity. It is made available under a CC-BY-NC-ND 4.0 International license.
Back to top
PreviousNext
Posted June 12, 2018.
Download PDF
Email

Thank you for your interest in spreading the word about bioRxiv.

NOTE: Your email address is requested solely to identify you as the sender of this article.

Enter multiple addresses on separate lines or separate them with commas.
A State Representation for Reinforcement Learning and Decision-Making in the Orbitofrontal Cortex
(Your Name) has forwarded a page to you from bioRxiv
(Your Name) thought you would like to see this page from the bioRxiv website.
CAPTCHA
This question is for testing whether or not you are a human visitor and to prevent automated spam submissions.
Share
A State Representation for Reinforcement Learning and Decision-Making in the Orbitofrontal Cortex
Nicolas W. Schuck, Robert Wilson, Yael Niv
bioRxiv 210591; doi: https://doi.org/10.1101/210591
Twitter logo Facebook logo LinkedIn logo Mendeley logo
Citation Tools
A State Representation for Reinforcement Learning and Decision-Making in the Orbitofrontal Cortex
Nicolas W. Schuck, Robert Wilson, Yael Niv
bioRxiv 210591; doi: https://doi.org/10.1101/210591

Citation Manager Formats

  • BibTeX
  • Bookends
  • EasyBib
  • EndNote (tagged)
  • EndNote 8 (xml)
  • Medlars
  • Mendeley
  • Papers
  • RefWorks Tagged
  • Ref Manager
  • RIS
  • Zotero
  • Tweet Widget
  • Facebook Like
  • Google Plus One

Subject Area

  • Neuroscience
Subject Areas
All Articles
  • Animal Behavior and Cognition (6022)
  • Biochemistry (13708)
  • Bioengineering (10436)
  • Bioinformatics (33157)
  • Biophysics (17109)
  • Cancer Biology (14173)
  • Cell Biology (20106)
  • Clinical Trials (138)
  • Developmental Biology (10868)
  • Ecology (16018)
  • Epidemiology (2067)
  • Evolutionary Biology (20346)
  • Genetics (13395)
  • Genomics (18634)
  • Immunology (13750)
  • Microbiology (32164)
  • Molecular Biology (13392)
  • Neuroscience (70069)
  • Paleontology (526)
  • Pathology (2190)
  • Pharmacology and Toxicology (3741)
  • Physiology (5864)
  • Plant Biology (12020)
  • Scientific Communication and Education (1814)
  • Synthetic Biology (3367)
  • Systems Biology (8166)
  • Zoology (1841)