Contributions of the striatum to learning, motivation, and performance: an associative account

doi:10.1016/j.tics.2012.07.007

Trends in Cognitive Sciences

Volume 16, Issue 9, September 2012, Pages 467-475

https://doi.org/10.1016/j.tics.2012.07.007 Get rights and content

It has long been recognized that the striatum is composed of distinct functional sub-units that are part of multiple cortico-striatal-thalamic circuits. Contemporary research has focused on the contribution of striatal sub-regions to three main phenomena: learning of associations between stimuli, actions and rewards; selection between competing response alternatives; and motivational modulation of motor behavior. Recent proposals have argued for a functional division of the striatum along these lines, attributing, for example, learning to one region and performance to another. Here, we consider empirical data from human and animal studies, as well as theoretical notions from both the psychological and computational literatures, and conclude that striatal sub-regions instead differ most clearly in terms of the associations being encoded in each region.

Section snippets

Anatomical and functional delineations of the striatum

Early anatomical studies delineated striatal sub-regions in terms of their afferent and efferent cortical projections (Figure 1), demonstrating that the dorsolateral region of the striatum (i.e., putamen) is primarily connected to sensory and motor cortices. In contrast, a dorsomedial region (i.e., caudate) is connected with frontal and parietal association cortices, whereas the ventral striatum is connected with limbic structures, including the amygdala, hippocampus, and medial orbitofrontal

Learning and the striatum

An extensive body of work has focused on the role of the striatum in facilitating two different types of associative learning: Pavlovian learning, in which, through repeated pairings, initially neutral conditioned stimuli (CSs) come to elicit reflexive behaviors in anticipation of the subsequent occurrence of appetitive or aversive events, and instrumental learning in which an organism learns to perform actions that increase the probability of obtaining reward or avoiding punishers [14].

Motor performance

There is considerable evidence to implicate the ventral striatum in generating skeletomotor reflexes elicited by Pavlovian cues 33, 34. Lesions as well as transient inactivation of the VS significantly impair previously acquired conditioned responses (CRs) to food-paired CSs: In particular, a medial part of the nucleus accumbens (Nacc) called the core, distinct from a more lateral part called the shell (Figure 1), has been shown to mediate the retrieval and expression of CS-US associations 33,

Motivation

Another function attributed to the striatum, and to the ventral striatum in particular, is that of motivation. Cues that indicate that a certain amount of reward is available given successful performance of an instrumental action, or even of a complex cognitive task, elicit increases in VS activity proportional to the amount of signaled reward and these signals correlate with the degree of performance enhancement found for larger compared to smaller rewards 12, 13. Paradoxically, whereas

An associative account of striatal function

The evidence reviewed here has implicated the ventral and dorsal striatum (both the DLS and DMS) in the learning as well as the performance of reward-related behaviors. It is unlikely therefore, that these regions differ functionally in terms of their respective contributions to learning vs performance 10, 11. Rather, a more parsimonious interpretation is that striatal regions support dissociable associative learning strategies that may respectively dominate at various stages of training,

Challenges and further directions

RL theories of behavioral control attempt to characterize the instantiation of, and arbitration between, various associative processes and, further, to map such processes – in the form of distinct algorithms – to different striatal subregions. Although there is mounting evidence in favor of this approach, a number of key challenges still remain.

First among these is the question whether Pavlovian signals in the ventral striatum are model-free, model-based, or both. Current computational accounts

Concluding remarks

In this article, we have reviewed evidence implicating the striatum as a whole in a number of distinct processes underlying reinforcement-related motor behavior: in learning of both instrumental actions and Pavlovian conditioned responses, in the expression of such learned behaviors, and in controlling the motivation to respond. We have noted that, rather than being divided along lines of learning versus performance, striatal subregions appear to implement distinct forms of associative

References (102)

E. Lynd-Balta et al.
The organization of midbrain projections to the striatum in the primate: sensorimotor-related striatum versus ventral striatum
Neuroscience
(1994)
B.W. Balleine
The integrative function of the basal ganglia in instrumental conditioning
Behav. Brain Res.
(2009)
J. Jankowski
Distinct striatal regions for planning and executing novel and automated movement sequences
Neuroimage
(2009)
A. Mestres-Misse
An anterior-posterior gradient of cognitive control within the dorsomedial striatum
Neuroimage
(2012)
B.W. Balleine et al.
Goal-directed instrumental action: contingency and incentive learning and their cortical substrates
Neuropharmacology
(1998)
J.P. O’Doherty
Temporal difference models and reward-related learning in the human brain
Neuron
(2003)
N.D. Daw
Model-based influences on humans’ choices and striatal prediction errors
Neuron
(2011)
C.A. Blaiss et al.
The nucleus accumbens core and shell are critical for the expression, but not the consolidation, of Pavlovian conditioned approach
Behav. Brain Res.
(2009)
T.D. Wager
Common and unique components of response inhibition revealed by fMRI
Neuroimage
(2005)
V.S. Chib
Neural mechanisms underlying paradoxical performance for monetary incentives are driven by loss aversion
Neuron
(2012)

K.C. Berridge

Dissecting components of reward: ‘liking’, ‘wanting’, and learning

Curr. Opin. Pharmacol.

(2009)

K.C. Berridge et al.

What is the role of dopamine in reward: hedonic impact, reward learning, or incentive salience?

Brain Res. Brain Res. Rev.

(1998)

B. Lex et al.

The role of nucleus accumbens dopamine in outcome encoding in instrumental and Pavlovian conditioning

Neurobiol. Learn. Mem.

(2010)

A.M. Bornstein et al.

Multiplicity of control in the basal ganglia: computational roles of striatal subregions

Curr. Opin. Neurobiol.

(2011)

M.S. Fanselow

From contextual fear to a dynamic view of memory systems

Trends Cogn. Sci.

(2010)

J. Jensen

Direct activation of the ventral striatum in anticipation of aversive stimuli

Neuron

(2003)

J. Wrase

Different neural systems adjust motor behavior in response to reward and punishment

Neuroimage

(2007)

E.A. Budygin

Aversive stimulus differentially triggers subsecond dopamine release in reward regions

Neuroscience

(2012)

J.C. Horvitz

Mesolimbocortical and nigrostriatal dopamine responses to salient non-reward events

Neuroscience

(2000)

B.C. Wittmann

Striatal activity underlies novelty-based choice in humans

Neuron

(2008)

S. Kakade et al.

Dopamine: generalization and bonuses

Neural Netw.

(2002)

F.G. Ashby

Cortical and basal ganglia contributions to habit learning and automaticity

Trends Cogn. Sci.

(2010)

L.G. Ungerleider

Imaging brain plasticity during motor skill learning

Neurobiol. Learn. Mem.

(2002)

D. Rodriguez-Moreno et al.

The dynamics of deductive reasoning: an fMRI investigation

Neuropsychologia

(2009)

D. Badre

Rostrolateral prefrontal cortex and individual differences in uncertainty-driven exploration

Neuron

(2012)

P. Dayan

The misbehavior of value and the discipline of the will

Neural Netw.

(2006)

D.S. Zahm et al.

On the significance of subterritories in the ‘accumbens’ part of the rat ventral striatum

Neuroscience

(1992)

R.A. Poldrack et al.

Category learning and the memory systems debate

Neurosci. Biobehav. Rev.

(2008)

H. Bokura

Event-related potentials for response inhibition in Parkinson's disease

Neuropsychologia

(2005)

F. Baglio

Functional brain changes in early Parkinson's disease during motor response and motor inhibition

Neurobiol. Aging

(2011)

D.S. Zahm

Ventral striatopallidothalamic projection: IV. Relative involvements of neurochemically distinct subterritories in the ventral pallidum and adjacent parts of the rostroventral forebrain

J. Comp. Neurol.

(1996)

B. Balleine

Multiple forms of value learning and the function of dopamine

S.T. Grafton

Functional mapping of sequence learning in normal humans

Cogn. Neurosci.

(1995)

M. Jueptner

Anatomy of motor learning. I. Frontal cortex and attention to action

J. Neurophysiol.

(1997)

S. Miyachi

Differential activation of monkey striatal neurons in the early and late stages of procedural learning

Exp. Brain Res.

(2002)

H.H. Yin

Dynamic reorganization of striatal circuits during the acquisition and consolidation of a skill

Nat. Neurosci.

(2009)

H.E. Atallah

Separate neural substrates for skill learning and performance in the ventral and dorsal striatum

Nat. Neurosci.

(2007)

M. Pessiglione

How the brain translates money into force: a neuroimaging study of subliminal motivation

Science

(2007)

L. Schmidt

Neural mechanisms underlying motivation of mental versus physical effort

PLoS Biol.

(2012)

G. Hall

Associative structures in Pavlovian and instrumental conditioning

J.J. Day

Associative learning mediates dynamic shifts in dopamine signaling in the nucleus accumbens

Nat. Neurosci.

(2007)

J.J. Day

Nucleus accumbens neurons encode Pavlovian approach behaviors: evidence from an autoshaping paradigm

Eur. J. Neurosci.

(2006)

H.H. Yin

Lesions of dorsolateral striatum preserve outcome expectancy but disrupt habit formation in instrumental learning

Eur. J. Neurosci.

(2004)

H.H. Yin

Blockade of NMDA receptors in the dorsomedial striatum prevents action-outcome learning in instrumental conditioning

Eur. J. Neurosci.

(2005)

H.H. Yin

The role of the dorsomedial striatum in instrumental conditioning

Eur. J. Neurosci.

(2005)

M. Liljeholm

Neural correlates of instrumental contingency learning: differential effects of action-reward conjunction and disjunction

J. Neurosci.

(2011)

S.C. Tanaka

Calculating consequences: brain systems that encode the causal effects of actions

J. Neurosci.

(2008)

E. Tricomi

A specific role for posterior dorsolateral striatum in human habit learning

Eur. J. Neurosci.

(2009)

R. Sutton et al.

Reinforcement Learning: An Introduction

(1998)

N.D. Daw

Uncertainty-based competition between prefrontal and dorsolateral striatal systems for behavioral control

Nat. Neurosci.

(2005)

Cited by (229)

Brain-wide circuit-specific targeting of astrocytes
2023, Cell Reports Methods
Astrocytes are integral components of brain circuitry. They enwrap synapses, react to neuronal activity, and regulate synaptic transmission. Astrocytes are heterogeneous and exhibit distinct features and functions in different circuits. Selectively targeting the astrocytes associated with a given neuronal circuit would enable elucidation of their circuit-specific functions but has been technically challenging to date. Recently, we constructed anterograde transneuronal viral vectors based on yellow fever vaccine YFV-17D. Among them, the replication-incompetent YFV^ΔNS1-Cre can selectively turn on reporter genes in postsynaptic neurons if the viral gene NS1 is expressed in postsynaptic neurons. Here we show that without exogenous expression of NS1 at the postsynaptic sites, locally injected YFV^ΔNS1-Cre selectively turns on reporter genes in astrocytes in downstream brain regions. The targeting of astrocytes can occur across the whole brain but is specific for the neuronal circuits traced. Therefore, YFV^ΔNS1-Cre provides a tool for selective genetic targeting of astrocytes to reveal their circuit-specific roles.
Functional magnetic resonance imaging study on anxiety and depression disorders induced by chronic restraint stress in rats
2023, Behavioural Brain Research
Persistent and negative stress stimulation is one of the most important factors leading to anxiety and depression in individuals, and it can negatively affect the normal function and structure of brain-related regions. However, the maladaptive changes of brain neural networks in anxiety and depression induced by chronic stress have not been explored in detail. In this study, we analyzed the changes in global information transfer efficiency, stress related blood oxygen level dependent (BOLD)- and diffusion tensor imaging (DTI)- signals and functional connectivity (FC) in rat models based on resting-state functional magnetic resonance imaging (rs-fMRI). The results showed that compared to control group, rats treated with chronic restraint stress (CRS) for 5 weeks had reconstructed the small-world network properties. In addition, CRS group had increased coherence and activity in bilateral Striatum (ST_R & L), but decreased coherence and activity in unilateral (left) Frontal Association Cortex (FrA_L) and unilateral (left) Medial Entorhinal Cortex (MEC_L). DTI analysis and correlation analysis confirmed the disrupted integrity of MEC_L and ST_R & L and their correlation to anxiety- and depressive-liked behaviors. Functional connectivity further showed these regions of interest (ROI) had decreased positive correlations with several brain areas, respectively. Our study comprehensively revealed the adaptive changes of brain neural networks induced by chronic stress and emphasized the abnormal activity and functional connectivity of ST_R & L and MEC_L in the pathological condition.
Semantic knowledge of social interactions is mediated by the hedonic evaluation system in the brain
2023, Cortex
Attaching semantic meaning to sensory information received from both inside and outside our bodies is a fundamental function of the human brain. The theory of Controlled Semantic Cognition (CSC) proposes that the formation of semantic knowledge relies on connections between spatially distributed modality-specific spoke-nodes, and a modality-general hub in the anterior temporal lobes (ATLs). This theory can also be applied to social semantic knowledge, though certain domain-specific spoke-nodes may make a disproportionate contribution to the understanding of social concepts. The ATLs have strong connections with spoke-node structures such as the subgenual ACC (sgACC) and the orbitofrontal cortex (OFC) that play an important role in predicting the hedonic value of stimuli. We hypothesized that in addition to the ATL semantic hub, a social semantic task would also require input from hedonic evaluation structures. We used voxel based morphometry (VBM) to examine structural brain–behavior relationships in 152 patients with neurodegeneration (Alzheimer's disease [N = 12], corticobasal syndrome (N = 18], progressive supranuclear palsy [N = 13], behavioral variant frontotemporal dementia [N = 56], and primary progressive aphasia (PPA) [N = 53]) using the Social Interaction Vocabulary Task (SIVT). This task measures the ability to correctly match a social term (e.g. “gossiping”) with a visual depiction of that social interaction. As predicted, VBM showed that worse SIVT scores corresponded with volume loss in bilateral ATL semantic hub regions, but also in the sgACC, OFC, caudate and putamen (pFWE <0.05). These results support the CSC model of a hub-and-spoke organization of social semantic knowledge with the ATL as a domain-general semantic hub, and ventromedial and striatal structures as domain specific spoke-nodes. Importantly, these results suggest that correct comprehension of social semantic concepts requires emotional ‘tagging’ of a concept by the evaluation system, and that the social deficits observed in some neurodegenerative disease syndromes may be caused by the break-down of this mechanism.
A microbial workout
2023, Cell Host and Microbe
Altered functional connectivity during performance feedback processing in multiple sclerosis
2023, NeuroImage: Clinical
Effective learning from performance feedback is vital for adaptive behavior regulation necessary for successful cognitive performance. Yet, how this learning operates in clinical groups that experience cognitive dysfunction is not well understood. Multiple sclerosis (MS) is an autoimmune, degenerative disease of the central nervous system characterized by physical and cognitive dysfunction. A highly prevalent impairment in MS is cognitive fatigue (CF). CF is associated with altered functioning within cortico-striatal regions that also facilitate feedback-based learning in neurotypical (NT) individuals. Despite this cortico-striatal overlap, research about feedback-based learning in MS, its associated neural underpinnings, and its sensitivity to CF, are all lacking. The present study investigated feedback-based learning ability in MS, as well as associated cortico-striatal function and connectivity. MS and NT participants completed a functional magnetic resonance imaging (fMRI) paired-word association task during which they received trial-by-trial monetary, non-monetary, and uninformative performance feedback. Despite reporting greater CF throughout the task, MS participants displayed comparable task performance to NTs, suggesting preserved feedback-based learning ability in the MS group. Both groups recruited the ventral striatum (VS), caudate nucleus, and ventromedial prefrontal cortex in response to the receipt of performance feedback, suggesting that people with MS also recruit cortico-striatal regions during feedback-based learning. However, compared to NT participants, MS participants also displayed stronger functional connectivity between the VS and task-relevant regions, including the left angular gyrus and right superior temporal gyrus, in response to feedback receipt. Results indicate that CF may not interfere with feedback-based learning in MS. Nonetheless, people with MS may recruit alternative connections with the striatum to assist with this form of learning. These findings have implications for cognitive rehabilitation treatments that incorporate performance feedback to remediate cognitive dysfunction in clinical populations.
An fMRI meta-analysis of the role of the striatum in everyday-life vs laboratory-developed habits
2022, Neuroscience and Biobehavioral Reviews
The dorsolateral striatum plays a critical role in the acquisition and expression of stimulus-response habits that are learned in experimental laboratories. Here, we use meta-analytic procedures to contrast the neural circuits activated by laboratory-acquired habits with those activated by stimulus-response behaviours acquired in everyday-life. We confirmed that newly learned habits rely more on the anterior putamen with activation extending into caudate and nucleus accumbens. Motor and associative components of everyday-life habits were identified. We found that motor-dominant stimulus-response associations developed outside the laboratory primarily engaged posterior dorsal putamen, supplementary motor area (SMA) and cerebellum. Importantly, associative components were also represented in the posterior putamen. Thus, common neural representations for both naturalistic and laboratory-based habits were found in the left posterior and right anterior putamen. These findings suggest a partial common striatal substrate for habitual actions that are performed predominantly by stimulus-response associations represented in the posterior striatum. The overlapping neural substrates for laboratory and everyday-life habits supports the use of both methods for the analysis of habitual behaviour.

View all citing articles on Scopus

View full text

Trends in Cognitive Sciences

ReviewContributions of the striatum to learning, motivation, and performance: an associative account

Section snippets

Anatomical and functional delineations of the striatum

Learning and the striatum

Motor performance

Motivation

An associative account of striatal function

Challenges and further directions

Concluding remarks

Neuroscience

Behav. Brain Res.

Neuroimage

Neuroimage

Neuropharmacology

Neuron

Neuron

Behav. Brain Res.

Neuroimage

Neuron

Curr. Opin. Pharmacol.

Brain Res. Brain Res. Rev.

Neurobiol. Learn. Mem.

Curr. Opin. Neurobiol.

Trends Cogn. Sci.

Neuron

Neuroimage

Neuroscience

Neuroscience

Neuron

Neural Netw.

Trends Cogn. Sci.

Neurobiol. Learn. Mem.

Neuropsychologia

Neuron

Neural Netw.

Neuroscience

Neurosci. Biobehav. Rev.

Neuropsychologia

Neurobiol. Aging

Ventral striatopallidothalamic projection: IV. Relative involvements of neurochemically distinct subterritories in the ventral pallidum and adjacent parts of the rostroventral forebrain

J. Comp. Neurol.

Multiple forms of value learning and the function of dopamine

Functional mapping of sequence learning in normal humans

Cogn. Neurosci.

Anatomy of motor learning. I. Frontal cortex and attention to action

J. Neurophysiol.

Differential activation of monkey striatal neurons in the early and late stages of procedural learning

Exp. Brain Res.

Dynamic reorganization of striatal circuits during the acquisition and consolidation of a skill

Nat. Neurosci.

Separate neural substrates for skill learning and performance in the ventral and dorsal striatum

Nat. Neurosci.

How the brain translates money into force: a neuroimaging study of subliminal motivation

Science

Neural mechanisms underlying motivation of mental versus physical effort

PLoS Biol.

Associative structures in Pavlovian and instrumental conditioning

Associative learning mediates dynamic shifts in dopamine signaling in the nucleus accumbens

Nat. Neurosci.

Nucleus accumbens neurons encode Pavlovian approach behaviors: evidence from an autoshaping paradigm

Eur. J. Neurosci.

Lesions of dorsolateral striatum preserve outcome expectancy but disrupt habit formation in instrumental learning

Eur. J. Neurosci.

Blockade of NMDA receptors in the dorsomedial striatum prevents action-outcome learning in instrumental conditioning

Eur. J. Neurosci.

The role of the dorsomedial striatum in instrumental conditioning

Eur. J. Neurosci.

Neural correlates of instrumental contingency learning: differential effects of action-reward conjunction and disjunction

J. Neurosci.

Calculating consequences: brain systems that encode the causal effects of actions

J. Neurosci.

A specific role for posterior dorsolateral striatum in human habit learning

Eur. J. Neurosci.

Reinforcement Learning: An Introduction

Uncertainty-based competition between prefrontal and dorsolateral striatal systems for behavioral control

Nat. Neurosci.

Review
Contributions of the striatum to learning, motivation, and performance: an associative account