RT Journal Article
SR Electronic
T1 A Cortical Microcircuit for Region-Specific Credit Assignment in Reinforcement Learning
JF bioRxiv
FD Cold Spring Harbor Laboratory
SP 2024.10.15.618337
DO 10.1101/2024.10.15.618337
A1 Chevy, Quentin
A1 Szadai, Zoltan
A1 Hertäg, Loreen
A1 Moll, Morgan
A1 Gibson, Elizabeth T
A1 Costa, Rui Ponte
A1 Rozsa, Balazs
A1 Kepecs, Adam
YR 2024
UL http://biorxiv.org/content/early/2024/10/17/2024.10.15.618337.abstract
AB The distributed architecture of the cortex poses a fundamental challenge for reinforcement learning: how to assign credit specifically to regions that contribute to successful behavior? Cortical neurons can be driven by both global reinforcers, like rewards, and local sensory features, making it difficult to disentangle these influences. To address this, we investigated cortical reinforcement learning by manipulating the reward-predictive sensory modality during learning tasks, while monitoring key regulators of cortical activity—local inhibitory neurons, and cholinergic inputs. We found that VIP interneurons are broadly recruited by reward-predictive cues via a modality-independent cholinergic signal. However, when task demands aligned with local computation, SST interneurons suppressed VIP recruitment through an inhibitory feedback loop. A computational model demonstrates that this cholinergic-VIP-SST interneuron circuit motif enables targeted reinforcement learning and region-specific credit assignment in the cortex. These results offer a neurobiologically-grounded framework for how the cortex uses global reinforcement signals to direct plasticity to task-relevant regions, enabling those regions to adapt and fine-tune their responses.Competing Interest StatementBalazs Rozsa is a founder of Femtonics Ltd. and a member of its scientific advisory board