A disengaging property of medial accumbens shell dopamine

Electrical stimulation of the medial forebrain bundle vigorously reinforces self-stimulation behaviour, yet rodents perform operant responses to terminate this stimulation. The accumbens shell emerged as a substrate subserving the reinforcing properties of electrical medial forebrain bundle stimulation, whereas disengaging properties were attributed to incidentally recruited substrates near the electrode. Here, we examine whether there are dissociable reinforcing and disengaging properties of medial accumbens shell dopamine and probe the substrates underlying these properties. Using a temporally delimited self-stimulation procedure, transgenic DAT-Cre mice expressing channelrhodopsin-II in ventral tegmental area dopamine neurons were trained to hold-down a lever to engage, and then release the lever to disengage, optogenetic stimulation of dopaminergic inputs to the medial accumbens shell through an implanted optic fiber. The cumulative and mean duration of hold-downs show divergent frequency responses identifying dissociable reinforcing and disengaging properties of medial accumbens shell dopamine. At higher stimulation frequencies the cumulative duration of hold-downs grows, whereas the mean duration of hold-downs wanes. Dopamine agonists reduced the cumulative duration of self-stimulation hold-downs, but only a D1 agonist produced this reduction through decreases in the mean duration of hold-downs, which were lengthened with a D2 antagonist. Thus, reinforcing and disengaging properties of electrical medial forebrain stimulation may arise from the downstream activation of dopamine receptors, uncovering a disengaging property of medial accumbens shell dopamine. Significance Statement Dopamine is thought to promote behaviour by acting as a reinforcer or error signal. Here, we show that mice vigorously self-stimulate dopamine inputs to the medial accumbens shell but control the duration of duration of these stimulations and prefer them to be brief. This disengaging property of medial accumbens shell dopamine depends on downstream neurotransmission at dopamine type 1 and 2 receptors. Thus, a single dopaminergic substrate, inputs to the medial accumbens shell, reinforces and disengages self-stimulation behaviour, highlighting the complexity and regional specificity of striatal dopamine function.


Introduction.
The discovery that rodents [1] and people [2,3] performed operant responses to electrically stimulate the medial forebrain bundle ignited speculation about the psychological consequences of this stimulation: Did it substitute for natural goals, positive memories, or utility -generate hunger, reward, or drive [4]?A parallel, search for the neural substrates recruited by electrical stimulation of the medial forebrain bundle spurred [5], ever complicated by the swath of cells and transgressing fibers characterizing this large bundle [5].It was soon realized that the quality of this stimulation was not owed to a single psychological process.Rodents [6,7] and people [2] would perform responses to terminate ongoing stimulation.The reinforcing and disengaging properties of electrical medial forebrain bundle stimulation were presumed to arise from distinct neural substrates indiscriminately recruited by the electrode [8].An imprecise but indelible link, between substrate and psychology was established.
A description of the neural substrates underlying the reinforcing properties of electrical brain stimulation focused on an area of the ventral striatum called the nucleus accumbens.Diverse patterns of electrical medial forebrain bundle stimulation similarly activated ventral tegmental area dopamine neurons [9] and their release of dopamine in the accumbens, but not dorsal striatum [10,11].Within the accumbens, the medial shell subregion aligned with the reinforcing properties of medial forebrain bundle stimulation because dopamine levels here closely tracked maximally reinforcing stimulation parameters [12].Further, pharmacologically elevating dopamine in the medial accumbens shell, but not adjacent striatal subregions, amplified the reinforcing efficacy of medial forebrain bundle stimulation [13].Thus, the reinforcing properties of medial forebrain bundle stimulation are partly attributable to elevations in medial accumbens shell dopamine occurring through transsynaptic activation of ventral tegmental area dopamine neurons [14][15][16].
Optogenetics [17] and transgenic rodents [18,19] confirmed that ventral tegmental area dopamine neurons robustly supported self-stimulation, particularly at high stimulation frequencies, suggesting that these neurons underpinned the reinforcing properties of electrical medial forebrain bundle stimulation [18,20].Once again however, questions were raised about diverse psychological processes engaged by optogenetic stimulation of ventral tegmental area dopamine neurons.When mice could choose between operant responses that delivered brief or prolonged bursts of high frequency optogenetic stimulation of ventral tegmental area dopamine neurons, brief bursts were preferred [21].The preferential activation of ventral tegmental area dopamine neurons for brief durations is akin to behavioural responses that terminate ongoing electrical brain stimulation.Differently, however, the substrate serving to reinforce and disengage optogenetic self-stimulation of ventral tegmental area dopamine neurons is unequivocal.
Self-stimulation studies highlight the capacity for dopamine to reinforce behaviour, but the dominant view of dopamine as an error signal nuances the interpretation of such studies.The firing of ventral tegmental area dopamine neurons, and consequent striatal release of dopamine, is thought to encode a prediction error [22,23].Operant responses for stimulation of dopamine neurons creates an error such that the receipt of stimulation is better than expected, and the response is repeated with increasing vigour.Attempts have been made to disentangle the hypotheses that optogenetic stimulation of ventral tegmental area dopamine neurons acts as a reinforcer or error signal [15,24], but these procedures overlook separable roles of dopamine neurons with distinct striatal targets [25][26][27].Notably, dopamine activity in the medial accumbens shell is inconsistent with error coding [28], modulated by the value of natural taste stimuli [29], and directly supports self-stimulation [27], highlighting a role for this substrate as a reinforcer.
A key feature of how animals engage with natural reinforcers is the capacity to control the duration of behaviour.Mammals [30], including wild mice [31], tightly control the duration of bouts of consummatory behaviour around taste [32].In lab studies mice increase the duration of their bouts of licking a solution as it becomes sweeter and decrease this duration as the solution becomes bitter [33].Changes in bout duration are often divorced from changes in bout frequency which relate more to the postingestive consequences of consumption, like satiety or malaise [34].
That is, the taste of a positively valenced sweet solution can support long bouts of licking but reinforce few bouts in total.It is unclear if, given the requisite control, mice would modulate the duration for which a non-sensory neural substrate was stimulated, much like they do with sensory taste receptors activated by natural reinforcers, like food.
We developed a temporally constrained self-stimulation procedure wherein mice are trained to hold-down a lever to engage, and then release the lever to disengage, optogenetic stimulation of dopaminergic inputs to the medial accumbens shell.As such, two principal measures describe self-stimulation: the cumulative and mean duration of lever hold-downs.The cumulative duration of hold-downs reflects the reinforcing properties of stimulation, arising either from changes in the mean duration or number of hold-downs, or both.Differently, the mean duration of hold-downs reflects a quality of the stimulation that arises as it is being experienced, allowing mice to disengage stimulation by releasing the lever.We use the temporally constrained self-stimulation procedure to dissociate reinforcing and disengaging properties of medial accumbens shell dopamine by examining whether the mean and cumulative durations of hold-downs diverge across stimulation frequences.Then, by modulating neurotransmission at dopamine type-1 (D1) and type-2 (D2) receptors with pharmacology we determined whether distinct downstream substrates impact the cumulative or mean duration of self-stimulation hold-downs.Selective recruitment of a genetically defined and input-specific substrate clarifies speculation about separable or overlapping substrates subserving the reinforcing and disengaging properties of electrical brain stimulation which we argue both arise at the level of dopamine inputs to the medial accumbens shell.
In the same surgery optic fibers (200 µm) were implanted bilaterally in the medial accumbens shell (10° angle, AP 1.5 mm, ML ±1.4 mm, DV -4.32 mm).Mice were left to recover for 2-3 months in their home-cage allowing sufficient viral expression in dopaminergic terminals.

Behavioral apparatus.
Behavioral training occurred in six conditioning chambers (ENV-307W-CT; Med-Associates Inc.), enclosed in sound-attenuating, fan-ventilated (ENV-025F) melamine cubicles (42.5 x 62.5 x 42 cm).Each chamber had a stainless-steel bar floor, paneled aluminum sidewalls, and a clear polycarbonate rear wall, ceiling, and front door.The upper left wall of the chamber featured a central white house-light (ENV-215M).The right wall contained two retractable levers (ENV-312-3) located on each side of a central fluid-port (ENV-303LP2-3).All experimental events were controlled and recorded using Med PC-V software.

Temporally constrained self-stimulation.
Mice received 40 self-stimulation training sessions (36 min) consisting of trials (30s; 48 trials/session) initiated and terminated with the extension and retraction, respectively, of active and inactive levers.In each trial, the active lever was pseudorandomly assigned to deliver pulsed laser stimulation (5 ms; 473 nm; 10 mW) at 2.5, 10, or 40 Hz such that one third of all trials represented each frequency.Trials occurred around a variable time inter-trial interval that was drawn from an exponential distribution from 2-18 s.When mice performed hold-downs by depressing the active lever, pulsed laser-light was delivered through an implanted optic fiber to the medial accumbens shell, which continued so long as the lever remained depressed and terminated when the lever was released.Inactive lever hold-downs had no consequence.
Retraining sessions without injections intervened each test session to mitigate potential treatment carryover effects.

Histology.
Mice were deeply anesthetized with sodium pentobarbital (Euthanyl TM , 270 mg/kg) and perfused with a 4% paraformaldehyde 0.1 M phosphate-buffered saline solution.Brains were extracted and cryoprotected in a 4% paraformaldehyde 30% sucrose solution for 1-2 days before being frozen at -80°C.Brain tissue was sliced into 50 µm coronal sections on a Lecia TM cryostat, thaw-mounted on microscope slides, and cover-slipped with a MOWIOL + DAPI solution, then imaged using a Leica TM epifluorescent microscope to identify viral expression in the ventral tegmental area and optical fiber tracts in the medial accumbens shell.

Data and Statistical Analysis.
MED-PC V software controlled and recorded the timing of all experimental events.The cumulative and mean duration, number and latency of hold-downs were analyzed using repeated measures analysis of variance (RMANOVA) in Prism TM v9.5.1 or SPSS v27.The Kolmogorov-Smirnov test was used to compare distributions.Conditioning sessions were blocked into groups of five (training) or three (test) sessions to ensure that enough hold-downs occurred to calculate mean hold-down duration.All analyses were conducted on raw data, although difference scores are shown to decompose significant interactions.Two mice did not perform a hold-down at one set of frequency-treatment conditions during the test phase and their data was excluded from test analyses.Post-hoc t-tests were Bonferroni-corrected (one family) and multiplicity-adjusted [35].

Temporally constrained self-stimulation of dopamine inputs to the medial accumbens shell.
DAT-Cre mice were trained (Fig

Discussion
The medial accumbens shell emerged as a substrate subserving the reinforcing properties of electrical medial forebrain bundle stimulation [12,13] and early studies attributed diverse psychological consequences of this stimulation to the indiscriminate recruitment of neural substrates near the electrode.Here, by allowing mice to control the duration for which they optogenetically self-stimulate a genetically defined and input-specific substrate we describe reinforcing and disengaging properties of medial accumbens shell dopamine.The principal evidence for this claim is that the cumulative and mean duration for which mice hold-down a lever to optogenetically stimulate dopaminergic inputs to the medial accumbens shell show divergent frequency responses.At higher stimulation frequencies the cumulative duration of hold-downs grows, consistent with more reinforcement, whereas the mean duration of hold-downs wanes, consistent with disengagement.Using pharmacology, we begin identifying substrates underpinning the reinforcing and disengaging properties of medial accumbens shell dopamine.Dopamine agonists reduced the cumulative duration of self-stimulation hold-downs, but only a D1 agonist produced this reduction through decreases in the mean duration of hold-downs, which were lengthened by a D2 antagonist.Thus, reinforcing and disengaging properties of electrical medial forebrain stimulation may arise from the downstream activation of dopamine receptors, uncovering a disengaging property of medial accumbens shell dopamine.
Across training, mice increased the cumulative duration, mean duration, and number of active relative to inactive lever hold-downs, consistent with the acquisition of temporally constrained selfstimulation.One previous study reported that mice could not learn to hold-down a lever to selfstimulate dopaminergic cell bodies in the substantia nigra [46], which may relate to an impoverished role of nigral, relative to ventral tegmental area dopamine neurons, in reinforcing behaviour [27].Additionally, nigral projection targets in the dorsal striatum [26] differ from ventral tegmental area dopamine inputs to the medial accumbens shell and likely engage circuitries that serve different functions.Also, the previous attempt to use hold-down duration as a selfstimulation operant may have been unsuccessful because of the high frequency of stimulation used, 50 Hz, which generally supported sub half-second hold-downs, consistent with the disengaging property of high frequency dopamine activity discovered in the current work.The effects of stimulation frequency detected here are smaller than in procedures where rodents control only the number, not duration, of trains of dopamine stimulation.Importantly, most previous work directly stimulated dopamine cell bodies in the ventral tegmental area [21,47], rather than accumbal inputs and few studies varied stimulation frequency.Nevertheless, mice demonstrate sophisticated and considerable (~25%) control over the duration with which dopamine inputs to the medial accumbens shell are stimulated.
We attribute decreases in mean hold-down duration during high frequency stimulation trials to the emergence of a disengaging property of medial accumbens shell dopamine.This interpretation is incongruent with a paradigm in contemporary neuroscience describing a relationship between striatal cells and motivated behaviour.In the striatum, the activity of neurons expressing D1 receptors is thought to encode positive valence and encourage behaviour, whereas neurons expressing D2 receptors oppose this influence, encoding negative valence when active and discouraging behaviour [48,49].Thus, input from the ventral tegmental area to the striatum, including the accumbens, is thought to reinforce behaviour because dopamine elevates the activity of D1 neurons and mutes the activity of D2 neurons, serving as a strong positive valence signal.Certainly, we observed increases in the cumulative time spent self-stimulating dopamine inputs to the medial accumbens shell, consistent with reinforcing and positively valenced properties of driving this substrate.Differently, the mean duration for which mice hold-down a lever to stimulate dopaminergic inputs to the medial accumbens shell wanes at high stimulation frequencies, consistent with the emergence of a disengaging property of this substrate.It is unclear whether this disengaging property is inherently valenced, although mice also reduce the duration with which they interact with negatively valenced taste stimuli [33,34].Ultimately, it is difficult to reconcile a disengaging property of medial accumbens shell dopamine with conventional views of striatum-wide dopamine as a reinforcer or error signal.Rather, the psychological consequences of striatal dopamine likely reflect subregion specificity and the modulation of downstream neuron activity, which is convergently shaped by corticostriatal inputs expressing dopamine receptors [50].
The disengaging property of medial accumbens shell dopamine may arise from the co-release of glutamate by ventral tegmental area dopamine neurons that innervate the accumbens shell.When mice could nose-poke at five concurrently available apertures armed with 40 Hz optogenetic stimulation of different durations (i.e., 1, 5, 20, 40 s), ventral tegmental area dopamine cell bodies predominantly supported responses for 5 s of stimulation.Differently, a subpopulation of ventral tegmental area dopamine neurons that co-release glutamate predominantly supported responses for 1 s of stimulation [21] and mice would perform an operant response to terminate noncontingent stimulation of these inputs to the medial accumbens shell [51].Thus, glutamate co-release from dopaminergic inputs to the medial accumbens shell may contribute to the current behavior, where high-frequency stimulation supported the briefest mean duration of self-stimulation hold-downs; however, the modulation of self-stimulation hold-downs with dopamine receptor agonists and antagonists, and a recent report that dopamine release absent glutamate is aversive [52], disfavours this possibility.
Dopamine receptor agonists and antagonists produced dissociable effects on the cumulative and mean durations for which mice self-stimulated dopaminergic inputs to the medial accumbens shell.While dopamine receptor agonists reduced the cumulative duration of self-stimulation holddowns, only the D1 agonist SKF38393 did so by reducing the mean duration of hold-downs which was unaffected by the D2 agonist quinpirole.Augmenting neurotransmission at D2 receptors with quinpirole diminished the reinforcing properties of medial accumbens shell dopamine, which likely manifested through subtle coincident reductions in the mean duration and number of hold-downs.Although dopamine antagonists did not affect the cumulative duration of self-stimulation holddowns, blunting neurotransmission at D2 receptors with raclopride lengthened self-stimulation hold-downs, as if to mitigate the disengaging property of medial accumbens shell dopamine.Differently, augmenting neurotransmission at D1 receptors with SKF38393 shortened selfstimulation hold-downs, consistent with amplification of the disengaging property of medial accumbens shell dopamine.
In the temporally constrained self-stimulation procedure, the first active hold-down of a trial is a clear measure of the disengaging properties of medial accumbens dopamine because mice cannot predetermine the available stimulation frequency.The first active hold-down on highfrequency trials was briefer than on low-frequency trials -this result is only explained by mice disengaging the lever to terminate stimulation in response to some quality of the stimulation arising in real time.Despite experiencing a disengaging property of medial accumbens shell dopamine on the first high-frequency hold-down, mice were quicker to initiate a second hold-down than during lower frequency trials.A single experience of high-frequency dopamine activity in the medial accumbens shell is sufficient to reinforce, and disengage, self-stimulation behaviour more strongly than lower frequency stimulation of the same substrate.Thus, the effects of stimulation frequency and pharmacological treatments on mean hold-down duration were observed on the first hold-down.Additionally, the D2 agonist quinpirole reduced the duration of the first active holddown, which explains the companion reduction in cumulative active hold-down duration.
Neurotransmission at D2 receptors thus partially underpins a disengaging property of medial accumbens shell dopamine, however, this effect diminished on subsequent hold-downs.The modulation of mean, or first, active hold-down duration with changes in neurotransmission at D2 receptors suggests that these neurons may play a principal role in determining whether, or when, animals disengage behavior.

Conclusion
Dopamine has varied and complex actions on inputs to and cells within the ventral striatum, but the consequence of these actions are simple: to reinforce behaviour.This conjecture is supported by decades of preclinical research manipulating and observing dopamine activity during behaviour and particularly the indiscriminate capacity for drugs that elevate dopamine levels to reinforce behaviour [53].However, much of this work relied on changes in behavioural frequency to describe the reinforcing properties of dopamine.Even still, some findings hinted a disengaging property of dopamine.For example, psychostimulants paired with food consumption [54,55], even when administered volitionally [56], reduced food consumption while encouraging food approach [57].Further, the self-administration of psychostimulants is reduced, not amplified, by microinfusion of dopamine agonists into the accumbens [58].These findings point to an inherent role of dopamine receptor activation in disengaging behaviour.
Here, using a temporally constrained self-stimulation procedure we report behaviour consistent with dissociable reinforcing and disengaging properties of medial accumbens shell dopamine.Specifically, intense dopamine activity in the medial accumbens shell is powerfully reinforcing but produces short, numerous, self-stimulation hold-downs.The reinforcing and disengaging properties of medial accumbens shell dopamine are differentially affected by neurotransmission at dopamine receptors.Notably, augmenting neurotransmission at D1 receptors serves to enhance the disengaging property, whereas blunting neurotransmission at D2 receptors amplifies the reinforcing property, of medial accumbens shell dopamine -effects which may arise from unintuitive striatal responses to dopamine activity [59,60].The current finding that medial accumbens shell dopamine is both reinforcing and disengaging runs counter to conventional views of striatal dopamine but is well accompanied by recent work diversifying the role of dopamine in psychology [61][62][63].
Fig. 1│Temporally constrained self-stimulation of dopamine inputs to the medial accumbens shell.a, Active and inactive levers were inserted into the conditioning chamber for 30 s trials (48 trials/session) which were separated by intertrial intervals (ITI; ~10 s).Active lever hold-downs triggered the delivery of patterned laser light (473 nm) to the medial accumbens shell at 2.5, 10, or 40 Hz which terminated when the lever was released.b, DAT-Cre mice (n=12, 7F, 5M) received microinfusion of the viral construct AAV2/9-EF1a-fDIO-hChR2(H134R)-eYFP in the ventral tegmental area and c, implantation of an optical fibre in the medial accumbens shell which are shown on modified panels from the atlas of Franklin and Watson (2007).Representative microscopy (5x) images with overlaid anatomical delineations showing viral expression (eYFP, green) and nuclei (DAPI, blue) in the d, ventral tegmental area and e, medial accumbens shell.f, Across blocks of 5 sessions, mice increased the cumulative time spent holding down the active lever during 30 s trials whereas the cumulative time spent holding down the inactive lever remained low.Higher-frequency trials came to support more cumulative active hold-down time than did lower-frequency trials [Lever x Frequency x Session, F(14, 154)=2.323,p=.006].g, The mean duration of active, but not inactive, lever hold-downs increased over sessions [Lever x Session, F(7, 77)=4.509,p<.001].h, Mice increased the number of active, but not inactive, lever hold-downs performed during 30 s trials across sessions particularly so for high-frequency relative to low-frequency trials [Lever x Frequency x Session, F(14, 154)=2.679,p=.002].Averaged data are mean ± s.e.m.