Skip to main content
bioRxiv
  • Home
  • About
  • Submit
  • ALERTS / RSS
Advanced Search
New Results

Learning optimal decisions with confidence

View ORCID ProfileJan Drugowitsch, André G. Mendonça, View ORCID ProfileZachary F. Mainen, Alexandre Pouget
doi: https://doi.org/10.1101/244269
Jan Drugowitsch
1Department of Neurobiology, Harvard Medical School, 220 Longwood Avenue, Boston, MA 02115, USA
  • Find this author on Google Scholar
  • Find this author on PubMed
  • Search for this author on this site
  • ORCID record for Jan Drugowitsch
  • For correspondence: jan_drugowitsch@hms.harvard.edu
André G. Mendonça
2Champalimaud Neuroscience Programme, Champalimaud Centre for the Unknown, Avenida de Brasilia, s/n, 1400-038 Lisbon, Portugal
  • Find this author on Google Scholar
  • Find this author on PubMed
  • Search for this author on this site
Zachary F. Mainen
2Champalimaud Neuroscience Programme, Champalimaud Centre for the Unknown, Avenida de Brasilia, s/n, 1400-038 Lisbon, Portugal
  • Find this author on Google Scholar
  • Find this author on PubMed
  • Search for this author on this site
  • ORCID record for Zachary F. Mainen
Alexandre Pouget
3Department of Basic Neuroscience, University of Geneva, Rue Michel Servet 1, CH-1211 Geneva, Switzerland
  • Find this author on Google Scholar
  • Find this author on PubMed
  • Search for this author on this site
  • Abstract
  • Full Text
  • Info/History
  • Metrics
  • Supplementary material
  • Preview PDF
Loading

Abstract

Diffusion decision models (DDMs) are immensely successful models for decision-making under uncertainty and time pressure. In the context of perceptual decision making, these models typically start with two input units, organized in a neuron-antineuron pair. In contrast, in the brain, sensory inputs are encoded through the activity of large neuronal populations. Moreover, while DDMs are wired by hand, the nervous system must learn the weights of the network through trial and error. There is currently no normative theory of learning in DDMs and therefore no theory of how decision makers could learn to make optimal decisions in this context. Here, we derive the first such rule for learning a near-optimal linear combination of DDM inputs based on trial-by-trial feedback. The rule is Bayesian in the sense that it learns not only the mean of the weights but also the uncertainty around this mean in the form of a covariance matrix. In this rule, the rate of learning is proportional (resp. inversely proportional) to confidence for incorrect (resp. correct) decisions. Furthermore, we show that, in volatile environments, the rule predicts a bias towards repeating the same choice after correct decisions, with a bias strength that is modulated by the previous choice’s difficulty. Finally, we extend our learning rule to cases for which one of the choices is more likely a priori, which provides new insights into how such biases modulate the mechanisms leading to optimal decisions in diffusion models.

Significance Statement Popular models for the tradeoff between speed and accuracy of everyday decisions usually assume fixed, low-dimensional sensory inputs. In contrast, in the brain, these inputs are distributed across larger populations of neurons, and their interpretation needs to be learned from feedback. We ask how such learning could occur and demonstrate that efficient learning is significantly modulated by decision confidence. This modulation predicts a particular dependency pattern between consecutive choices, and provides new insight into how a priori biases for particular choices modulate the mechanisms leading to efficient decisions in these models.

Footnotes

  • Clarifying text, adding relation to learning by optimization.

Copyright 
The copyright holder for this preprint is the author/funder, who has granted bioRxiv a license to display the preprint in perpetuity. It is made available under a CC-BY-NC-ND 4.0 International license.
Back to top
PreviousNext
Posted September 18, 2019.
Download PDF

Supplementary Material

Email

Thank you for your interest in spreading the word about bioRxiv.

NOTE: Your email address is requested solely to identify you as the sender of this article.

Enter multiple addresses on separate lines or separate them with commas.
Learning optimal decisions with confidence
(Your Name) has forwarded a page to you from bioRxiv
(Your Name) thought you would like to see this page from the bioRxiv website.
CAPTCHA
This question is for testing whether or not you are a human visitor and to prevent automated spam submissions.
Share
Learning optimal decisions with confidence
Jan Drugowitsch, André G. Mendonça, Zachary F. Mainen, Alexandre Pouget
bioRxiv 244269; doi: https://doi.org/10.1101/244269
Digg logo Reddit logo Twitter logo Facebook logo Google logo LinkedIn logo Mendeley logo
Citation Tools
Learning optimal decisions with confidence
Jan Drugowitsch, André G. Mendonça, Zachary F. Mainen, Alexandre Pouget
bioRxiv 244269; doi: https://doi.org/10.1101/244269

Citation Manager Formats

  • BibTeX
  • Bookends
  • EasyBib
  • EndNote (tagged)
  • EndNote 8 (xml)
  • Medlars
  • Mendeley
  • Papers
  • RefWorks Tagged
  • Ref Manager
  • RIS
  • Zotero
  • Tweet Widget
  • Facebook Like
  • Google Plus One

Subject Area

  • Neuroscience
Subject Areas
All Articles
  • Animal Behavior and Cognition (3575)
  • Biochemistry (7520)
  • Bioengineering (5479)
  • Bioinformatics (20677)
  • Biophysics (10258)
  • Cancer Biology (7931)
  • Cell Biology (11583)
  • Clinical Trials (138)
  • Developmental Biology (6563)
  • Ecology (10136)
  • Epidemiology (2065)
  • Evolutionary Biology (13540)
  • Genetics (9498)
  • Genomics (12788)
  • Immunology (7872)
  • Microbiology (19451)
  • Molecular Biology (7614)
  • Neuroscience (41875)
  • Paleontology (306)
  • Pathology (1252)
  • Pharmacology and Toxicology (2179)
  • Physiology (3249)
  • Plant Biology (7007)
  • Scientific Communication and Education (1291)
  • Synthetic Biology (1942)
  • Systems Biology (5406)
  • Zoology (1107)