ReviewAdaptive decision making and value in the anterior cingulate cortex
Introduction
Much of our everyday lives, as anyone willing to consider the issue will be aware, consists of choosing between alternative courses of action. Determining which response is appropriate in a particular context is a complex issue, influenced by current motivation and past history as well as individual assessments of the desirability of each alternative. Sometimes our actions are selected voluntarily, based on an internal assessment of what is the optimal option to choose, other times they are constrained by instruction or prompted by external stimuli. Nevertheless, in a static environment, where the relationships between what we choose to do and the concomitant consequences were fixed, such decisions would still be relatively straightforward. Unfortunately, however, we exist in a dynamic and uncertain world, cooperating and competing with others, in which the outcome of our choices can be influenced by what we and others have previously decided to do.
Many parts of the frontal lobe are integral to the selection and control of goal-directed actions. However, it is an open question what the exact contribution of discrete prefrontal and anterior cingulate cortical regions is to this process. This is exemplified by the debates over the function of the anterior cingulate cortex (ACC) (Fig. 1). Activations have been found in this region in a diverse range of cognitive tasks as well as in response to autonomic arousal (Botvinick et al., 2004, Bush et al., 2000, Critchley, 2005, Paus et al., 1998, Rushworth et al., 2004), leading to the question of what the common factor driving the ACC response in these situations may be. However, the ACC is also seldom activated in isolation of other interconnected regions such as the orbitofrontal cortex (OFC), lateral prefrontal cortex and parts of the striatum (Duncan and Owen, 2000, Paus et al., 1998). This raises the related issue of how the function of the ACC differs from these other prefrontal and subcortical areas. Furthermore, there has frequently not been perfect correspondence between the data gathered using different techniques and species.
The purpose of this present paper is not to give a detailed critique of the extant theories of ACC function, particularly with regard to the human electrophysiological literature, as this has been accomplished comprehensively by several recent reviews (Botvinick et al., 2001, Holroyd and Coles, 2002, Paus, 2001, Ridderinkhof et al., 2004, Rushworth et al., 2004). Instead, we wish to discuss how recent work investigating decision making, where the relationships between actions and their consequences do not remain fixed, has illuminated one aspect of the function of the ACC. By forcing subjects to use their recent experience to select an appropriate course of action, these studies uncover a crucial role for the ACC, particularly the dorsal sulcal region (see Fig. 1), in forming action–outcome associations to guide response selection.
Section snippets
Interactions between action selection and performance monitoring in ACC
In a changeable environment, it is vital to be able to adapt one’s behaviour rapidly depending on how successfully one’s goals were achieved. Such adaptability has been seen as a cardinal aspect of cognitive control in the human literature (Allport et al., 1994, Bunge, 2004), as well as an indication in animal studies that responses are not merely being selected out of habit (though see Balleine and Dickinson, 1998 for more comprehensive criteria). Neuroimaging studies investigating task
Monitoring instructed actions and the orbitofrontal cortex (OFC)
Neuroimaging studies of choosing and evaluating responses during decision making contexts often show activations in OFC as well as in dorsal ACC (Berns et al., 2001, Cohen et al., 2005, Coricelli et al., 2005, Elliott et al., 2000, Ernst and Paulus, 2005, O’Doherty et al., 2003a, O’Doherty et al., 2003b). Lesions which include OFC in humans, monkeys or rats cause impairments in using reward information to guide and alter choices (Bechara et al., 2000, Izquierdo et al., 2005, Jones and Mishkin,
An anatomy of performance monitoring
Functional differences can often be understood through inspection of the pattern of connections of the respective regions (Passingham et al., 2002). While there is detailed literature documenting connectivity in the monkey brain, it has to date been difficult to obtain comparable information in humans. However, using diffusion weighted magnetic resonance imaging (DWI) and probabilistic tractography, it has recently become possible to examine trajectories of white matter fibre tracts in vivo in
Performance monitoring and reward history in macaque ACC
Activations in neuroimaging experiments demonstrate a correlation between a component of the experimental task and an indirect measure of neuronal activity, but cannot establish whether or not the region is essential for that function. As well as evidence from neuroimaging, a large number of electrophysiological studies in monkeys have reported cells in the ACCs which respond to errors or reductions in reward which facilitate corrective behaviour (Amiez et al., 2005, Ito et al., 2003, Shima and
Adaptive decision making and learning the value of actions in the ACC
The vast majority of the tasks used to study the function of the ACC have relied on situations which have limited possibilities of response and a single well-defined outcome. In the controlled, pre-programmed environment of the laboratory, it is straightforward for researchers to determine what the correct response to make is and to judge a participant’s behaviour accordingly. All the studies described thus far in the present review have involved situations in which uncertainty is
ACC and distributed circuits for the value of actions
One difficulty with interpretations of brain function gained from a number of neuroscientific techniques, including the lesion method, is that they tend to encourage focus on a single area in isolation rather than allowing consideration of how a particular region works in concert with others to guide behaviour. As was discussed earlier, the ACC and OFC have frequently been co-activated in neuroimaging studies of outcome monitoring, though the evidence from Walton and colleagues (2004) is that
Conclusion
Deciding what is the most desirable or advantageous action to choose in a variable world of numerous competing agents is a challenging question which has exercised economists and behavioural ecologists for numerous years. Recently, neuroscientists have also been approaching this topic, moving away from tasks in which only a single option is always categorically correct to ones in which contingencies change, outcomes are uncertain and the likelihood of success depends on the results of choices
Acknowledgments
This work was supported by the Medical Research Council, the Royal Society (MFSR) and the Wellcome Trust (MEW, PLC).
References (108)
- et al.
Goal-directed instrumental action: contingency and incentive learning and their cortical substrates
Neuropharmacology
(1998) - et al.
Midbrain dopamine neurons encode a quantitative reward prediction error signal
Neuron
(2005) - et al.
Conflict monitoring and anterior cingulate cortex: an update
Trends Cogn. Sci.
(2004) - et al.
Cognitive and emotional influences in anterior cingulate cortex
Trends Cogn. Sci.
(2000) Optimal foraging: the marginal value theorem
Theor. Popul. Biol.
(1976)- et al.
Functional connectivity with anterior cingulate and orbitofrontal cortices during decision-making
Brain Res. Cogn. Brain Res.
(2005) - et al.
Prefrontal cortex activation in task switching: an event-related fMRI study
Brain Res. Cogn. Brain Res.
(2000) Metalearning and neuromodulation
Neural Netw.
(2002)- et al.
Common regions of the human frontal lobe recruited by diverse cognitive demands
Trends Neurosci.
(2000) - et al.
Neurobiology of decision making: a selective review from a neurocognitive and clinical perspective
Biol. Psychiatry
(2005)