PT  - JOURNAL ARTICLE
AU  - Zhewei Zhang
AU  - Zhenbo Cheng
AU  - Zhongqiao Lin
AU  - Chechang Nie
AU  - Tianming Yang
TI  - A neural network framework for the orbitofrontal cortex and model-based reinforcement learning
AID  - 10.1101/116608
DP  - 2017 Jan 01
TA  - bioRxiv
PG  - 116608
4099  - http://biorxiv.org/content/early/2017/03/20/116608.short
4100  - http://biorxiv.org/content/early/2017/03/20/116608.full
AB  - Model-based reinforcement learning (mbRL) has been widely used in explaining animal behavior. In mbRL, the model, or the structure of the task, is used to evaluate the associations between actions and outcomes. It has been proposed that the orbitofrontal cortex (OFC) encodes the model during mbRL. However, it is not well understood how the OFC acquires and stores model information. Here, we propose a neural network framework based on reservoir computing. Reservoir networks exhibit heterogeneous and dynamic activity patterns that are suitable to encode task states. The information can be extracted by a linear readout trained with reinforcement learning. We demonstrate how our framework acquires and stores the task state space. The framework exhibits mbRL behavior and its aspects resemble experimental findings of the OFC. Our study provides a theoretical explanation of how the OFC may contribute to mbRL and a new approach to understanding the neural mechanism underlying mbRL.