What is reinforced by phasic dopamine signals?

Abstract

The basal ganglia have been associated with processes of reinforcement learning. A strong line of supporting evidence comes from the recording of dopamine (DA) neurones in behaving monkeys. Unpredicted, biologically salient events, including rewards cause a stereotypic short-latency (70–100 ms), short-duration (100–200 ms) burst of DA activity — the phasic response. This response is widely considered to represent reward prediction errors used as teaching signals in appetitive learning to promote actions that will maximise future reward acquisition. For DA signalling to perform this function, sensory processing afferent to DA neurones should discriminate unpredicted reward-related events. However, the comparative response latencies of DA neurones and orienting gaze-shifts indicate that phasic DA responses are triggered by pre-attentive sensory processing. Consequently, in circumstances where biologically salient events are both spatially and temporally unpredictable, it is unlikely their identity will be known at the time of DA signalling. The limited quality of afferent sensory processing and the precise timing of phasic DA signals, suggests that they may play a less direct role in ‘Law of Effect’ appetitive learning. Rather, the ‘time-stamp’ nature of the phasic response, in conjunction with the other signals likely to be present in the basal ganglia at the time of phasic DA input, suggests it may reinforce the discovery of unpredicted sensory events for which the organism is responsible. Furthermore, DA-promoted repetition of preceding actions/movements should enable the system to converge on those aspects of context and behavioural output that lead to the discovery of novel actions.

Introduction

When trying to discover the role of a specific component in a complicated system it is helpful to have some understanding of the system's overall function. The nigro-striatal and mesolimbic dopamine (DA) projections are important components of the basal ganglia, which are one of the brain's fundamental processing units. Thus, in this article we will start by considering briefly some prevalent ideas on basal ganglia involvement in action selection and reinforcement learning. Within this context we will proceed to evaluate the specific contribution of the phasic response of DA neurones to the processes of reinforcement learning. Although it is clear that DA projections ascending from the ventral midbrain target numerous structures, including intrinsic nuclei of the basal ganglia, frontal cortex, amygdala, hippocampus, septal area, several thalamic nuclei, and the habenula (Lindvall and Bjorklund, 1974), for the purpose of the present article we will restrict ourselves to discussion of how striatal function may be influenced by phasic DA input. Hopefully, a better understanding of this comparatively well characterised system will provide important clues concerning the role of phasic DA input to other target structures. This paper is an extension of ideas expressed in our recent perspectives article (Redgrave and Gurney, 2006).