Summary
The anterior cingulate cortex (ACC) is implicated in learning the value of actions, but it remains poorly understood whether and how it contributes to model-based mechanisms that use action-state predictions and afford behavioural flexibility. To isolate these mechanisms, we developed a multi-step decision task for mice in which both action-state transition probabilities and reward probabilities changed over time. Calcium imaging revealed ramps of choice-selective neuronal activity, followed by an evolving representation of the state reached and trial outcome, with different neuronal populations representing reward in different states. ACC neurons represented the current action-state transition structure, whether state transitions were expected or surprising, and the predicted state given chosen action. Optogenetic inhibition of ACC blocked the influence of action-state transitions on subsequent choice, without affecting the influence of rewards. These data support a role for ACC in model-based reinforcement learning, specifically in using action-state transitions to guide subsequent choice.
Highlights
A novel two-step task disambiguates model-based and model-free RL in mice.
ACC represents all trial events, reward representation is contextualised by state.
ACC represents action-state transition structure, predicted states, and surprise.
Inhibiting ACC impedes action-state transitions from influencing subsequent choice.
Competing Interest Statement
The authors have declared no competing interest.
Footnotes
- Added two-step task calcium imaging data. - Added additional analysis showing correlation between opto effect on transition predictor and subjects use of model-based RL.