Skip to main content
bioRxiv
  • Home
  • About
  • Submit
  • ALERTS / RSS
Advanced Search
New Results

Predictive coding is a consequence of energy efficiency in recurrent neural networks

View ORCID ProfileAbdullahi Ali, View ORCID ProfileNasir Ahmad, Elgar de Groot, Marcel A. J. van Gerven, View ORCID ProfileTim C. Kietzmann
doi: https://doi.org/10.1101/2021.02.16.430904
Abdullahi Ali
1Donders Institute for Brain, Cognition and Behaviour, Radboud University, Nijmegen, the Netherlands
  • Find this author on Google Scholar
  • Find this author on PubMed
  • Search for this author on this site
  • ORCID record for Abdullahi Ali
  • For correspondence: abdullahi.ali@donders.ru.nl tim.kietzmann@donders.ru.nl
Nasir Ahmad
1Donders Institute for Brain, Cognition and Behaviour, Radboud University, Nijmegen, the Netherlands
  • Find this author on Google Scholar
  • Find this author on PubMed
  • Search for this author on this site
  • ORCID record for Nasir Ahmad
Elgar de Groot
1Donders Institute for Brain, Cognition and Behaviour, Radboud University, Nijmegen, the Netherlands
2Department of Experimental Psychology, Utrecht University, Utrecht, the Netherlands
  • Find this author on Google Scholar
  • Find this author on PubMed
  • Search for this author on this site
Marcel A. J. van Gerven
1Donders Institute for Brain, Cognition and Behaviour, Radboud University, Nijmegen, the Netherlands
  • Find this author on Google Scholar
  • Find this author on PubMed
  • Search for this author on this site
Tim C. Kietzmann
1Donders Institute for Brain, Cognition and Behaviour, Radboud University, Nijmegen, the Netherlands
  • Find this author on Google Scholar
  • Find this author on PubMed
  • Search for this author on this site
  • ORCID record for Tim C. Kietzmann
  • For correspondence: abdullahi.ali@donders.ru.nl tim.kietzmann@donders.ru.nl
  • Abstract
  • Full Text
  • Info/History
  • Metrics
  • Data/Code
  • Preview PDF
Loading

Abstract

Predictive coding represents a promising framework for understanding brain function. It postulates that the brain continuously inhibits predictable sensory input, ensuring a preferential processing of surprising elements. A central aspect of this view is its hierarchical connectivity, involving recurrent message passing between excitatory bottom-up signals and inhibitory top-down feedback. Here we use computational modelling to demonstrate that such architectural hard-wiring is not necessary. Rather, predictive coding is shown to emerge as a consequence of energy efficiency. When training recurrent neural networks to minimise their energy consumption while operating in predictive environments, the networks self-organise into prediction and error units with appropriate inhibitory and excitatory interconnections, and learn to inhibit predictable sensory input. Moving beyond the view of purely top-down driven predictions, we furthermore demonstrate, via virtual lesioning experiments, that networks perform predictions on two timescales: fast lateral predictions among sensory units, and slower prediction cycles that integrate evidence over time.

Competing Interest Statement

The authors have declared no competing interest.

Footnotes

  • https://github.com/KietzmannLab/EmergentPredictiveCoding

Copyright 
The copyright holder for this preprint is the author/funder, who has granted bioRxiv a license to display the preprint in perpetuity. It is made available under a CC-BY 4.0 International license.
Back to top
PreviousNext
Posted November 16, 2021.
Download PDF
Data/Code
Email

Thank you for your interest in spreading the word about bioRxiv.

NOTE: Your email address is requested solely to identify you as the sender of this article.

Enter multiple addresses on separate lines or separate them with commas.
Predictive coding is a consequence of energy efficiency in recurrent neural networks
(Your Name) has forwarded a page to you from bioRxiv
(Your Name) thought you would like to see this page from the bioRxiv website.
CAPTCHA
This question is for testing whether or not you are a human visitor and to prevent automated spam submissions.
Share
Predictive coding is a consequence of energy efficiency in recurrent neural networks
Abdullahi Ali, Nasir Ahmad, Elgar de Groot, Marcel A. J. van Gerven, Tim C. Kietzmann
bioRxiv 2021.02.16.430904; doi: https://doi.org/10.1101/2021.02.16.430904
Digg logo Reddit logo Twitter logo Facebook logo Google logo LinkedIn logo Mendeley logo
Citation Tools
Predictive coding is a consequence of energy efficiency in recurrent neural networks
Abdullahi Ali, Nasir Ahmad, Elgar de Groot, Marcel A. J. van Gerven, Tim C. Kietzmann
bioRxiv 2021.02.16.430904; doi: https://doi.org/10.1101/2021.02.16.430904

Citation Manager Formats

  • BibTeX
  • Bookends
  • EasyBib
  • EndNote (tagged)
  • EndNote 8 (xml)
  • Medlars
  • Mendeley
  • Papers
  • RefWorks Tagged
  • Ref Manager
  • RIS
  • Zotero
  • Tweet Widget
  • Facebook Like
  • Google Plus One

Subject Area

  • Neuroscience
Subject Areas
All Articles
  • Animal Behavior and Cognition (4109)
  • Biochemistry (8812)
  • Bioengineering (6517)
  • Bioinformatics (23456)
  • Biophysics (11788)
  • Cancer Biology (9205)
  • Cell Biology (13318)
  • Clinical Trials (138)
  • Developmental Biology (7433)
  • Ecology (11407)
  • Epidemiology (2066)
  • Evolutionary Biology (15145)
  • Genetics (10433)
  • Genomics (14041)
  • Immunology (9169)
  • Microbiology (22152)
  • Molecular Biology (8808)
  • Neuroscience (47558)
  • Paleontology (350)
  • Pathology (1428)
  • Pharmacology and Toxicology (2491)
  • Physiology (3730)
  • Plant Biology (8079)
  • Scientific Communication and Education (1437)
  • Synthetic Biology (2220)
  • Systems Biology (6037)
  • Zoology (1252)