ABSTRACT
We are constantly faced with the trade-off between exploiting past actions with known outcomes and exploring novel actions whose outcomes may be better. The balance between exploitation and exploration has been hypothesized to rely on multiple neuromodulator systems, namely dopaminergic neurons of the substantia nigra pars compacta (SNc) and noradrenergic neurons of the locus coeruleus (LC). However, little is known about the dynamics of these neuromodulator systems during exploitative and exploratory states, or how they interact. We developed a novel behavioral paradigm to capture exploitative and exploratory behavioral states, and imaged calcium dynamics in genetically-identified dopaminergic SNc neurons and noradrenergic LC neurons during the transitions between these states. We found dichotomous changes in sustained activity in SNc and LC during exploitative bouts of action-reward, with SNc showing higher and LC showing lower sustained activity. Exploitative states were also marked by a lengthening of positive SNc response plateaus and negative LC response depressions, as well as hysteretic dynamics in SNc networks. Chemogenetic enhancement of dopaminergic and noradrenergic excitability favored exploitative and exploratory states, respectively. Together, these data suggest that opponent changes in dopaminergic and noradrenergic activity states modulate the transitions between exploitative and exploratory behavioral states, with important implications for downstream circuit dynamics.
