Estrogenic control of reward prediction errors and reinforcement learning

Abstract

Gonadal hormones act throughout the brain¹, and nearly all neuropsychiatric disorders vary in symptom severity with hormonal fluctuations over the reproductive cycle, gestation, and perimenopause^2–4. Yet the mechanisms by which hormones influence mental and cognitive processes are unclear. Exogenous estrogenic hormones modulate dopamine signaling in the nucleus accumbens core (NAcc)^5,6, which instantiates reward prediction errors (RPEs) for reinforcement learning^7–16. Here we show that endogenous estrogenic hormones enhance RPEs and sensitivity to previous rewards by regulating expression of dopamine reuptake proteins in the NAcc. We trained rats to perform a temporal wagering task with different reward states; rats adjusted how quickly they initiated trials across states, balancing effort against expected rewards. Dopamine release in the NAcc reflected RPEs that predicted and causally in-fluenced subsequent initiation times. When fertile, females more quickly adjusted their initiation times to match reward states due to enhanced dopaminergic RPEs in the NAcc. Proteomics revealed reduced expression of dopamine transporters in fertile stages of the reproductive cycle. Finally, genetic suppression of midbrain estrogen receptors eliminated hormonal modulation of behavior. Estrogenic hormones therefore control the rate of reinforcement learning by regulating RPEs via dopamine reuptake, providing a mechanism by which hormones influence neural dynamics for motivation and learning.