Summary
The ability to extinguish conditioned fear memory is critical for adaptive control of fear response, and its impairment is a hallmark of emotional disorders like post-traumatic stress disorder (PTSD). Fear extinction is thought to take place when animals form a new memory that suppresses the original fear memory. However, little is known about the nature and the site of formation and storage of the new extinction memory. Here, we demonstrate that a fear extinction memory engram is formed and stored in a genetically distinct basolateral amygdala (BLA) neuronal population that drive reward behaviors and antagonize the BLA’s original fear neurons. The activation of the fear extinction engram neurons and natural reward-responsive neurons overlap extensively in the BLA. Furthermore, these two neuron subsets are mutually interchangeable in driving reward behaviors and fear extinction behaviors. Thus, fear extinction memory is a newly formed reward memory.
Introduction
The inability to extinguish fear is a hallmark of many psychiatric disorders, such as PTSD and generalized anxiety disorder (Shalev et al., 2017; Stein and Sareen, 2015). Following Pavlovian fear conditioning (Pavlov and Anrep, 1927), repeated or prolonged presentations of the conditioned stimulus (CS) without an expected aversive unconditioned stimulus (US) diminishes the conditioned fear response, a phenomenon called fear extinction (Herry et al., 2010; Myers and Davis, 2007; Quirk and Mueller, 2008). Fear extinction has been proposed to involve the formation of a new memory in competition with the original fear memory (Bouton, 2004; Quirk and Mueller, 2008; Quirk et al., 2010). The amygdala is a key structure for fear memory (Davis, 1992; Duvarci and Pare, 2014; Ehrlich et al., 2009; Maren and Fanselow, 1996), and is also involved in the fear extinction memory (Amano et al., 2010; Grewe et al., 2017; Herry et al., 2008). However, it is unknown whether the amygdala is the site for the storage of fear extinction memory, and if so, which subset of amygdala neuron stores this memory.
Excitatory neurons in the mouse basolateral amygdala (BLA) respond to both positive and negative valence stimuli (Beyeler et al., 2016; Davis and Whalen, 2001; Kim et al., 2016; Namburi et al., 2015; Redondo et al., 2014), and more than 90% of these neurons are composed of two genetically, functionally and anatomically distinct neuronal populations (Kim et al., 2016; Kim et al., 2017). R-spondin-2-expressing (Rspo2+) neurons located in the anterior BLA respond to negative valence stimuli and control negative behaviors and memories, whereas protein-phosphatase-1-regulatory-inhibitor-subunit-1B-expressing (Ppp1r1b+) neurons located in the posterior BLA (pBLA) respond to positive valence stimuli and control appetitive behaviors and memories (Kim et al., 2016). Furthermore, these two neuronal populations antagonize each other through feed-forward inhibitions mediated by local inhibitory interneurons (Kim et al., 2016).
In this study, we investigated a potential role of BLA Ppp1r1b+ neurons in fear extinction using a contextual fear extinction paradigm. We found that fear extinction memory engram cells are formed and stored within the BLA Ppp1r1b+ neuronal population, and these engram cells are necessary for suppressing the original fear memory. Furthermore, fear extinction engram cells and natural reward-responsive cells in the pBLA Ppp1r1b+ neurons are mutually interchangeable in driving reward functions and fear extinction behaviors.
Results
BLA Ppp1r1b+ neurons are activated during contextual fear extinction
Fear extinction phenomena have been observed and studied in both cue-dependent fear conditioning and context-dependent fear conditioning paradigms (Amano et al., 2010; Baldi and Bucherelli, 2014; Herry et al., 2008; Trouche et al., 2013; Zushida et al., 2007). Since the BLA plays a critical role in contextual fear conditioning (Calandreau et al., 2005; Goosens and Maren, 2001; Huff and Rudy, 2004; Redondo et al., 2014) and a substantial amount of information is available regarding the excitatory neuronal subsets in BLA (see Introduction) (Kim et al., 2016), we employed a contextual fear extinction paradigm in this study. On Day 1, the Extinction group received contextual fear conditioning (CFC) in a box where the context served as the CS and three rounds of footshocks served as the US. On Day 2, mice were returned to the conditioned box for 45 min in the absence of footshocks to receive contextual fear extinction training (FET). On Day 3, the mice were tested for 5 min extinction memory retrieval in the conditioned box and then sacrificed (Figure 1A). Two control groups were set up as follows. The Non-Extinction group went through the same protocol as the Extinction group, except that on Day 2 the mice stayed in their home cage (HC) and didn’t receive any extinction training (Figure 1A). The Non-Shock group received the same protocol as the Extinction group, but received no footshocks on Day 1 (Figure 1A). The Extinction group and Non-Extinction group, but not the Non-Shock group, displayed robust freezing behavior on Day 1 that increased as more footshocks were presented (Figure 1B). On Day 3, the Extinction group froze much less than the Non-Extinction group (Figure 1C).
Double smFISH (single molecular Fluorescence in situ Hybridization) was performed to detect the activity-dependent expression of the immediate early gene, Fos, in Rspo2+ and Ppp1r1b+ cells (see Methods). The proportions of Fos+ cells of Rspo2+ cells and Fos+ cells of Ppp1r1b+ cells were quantified across the anterior-posterior (A/P) axis for the aBLA and pBLA, respectively (Kim et al., 2016). The proportion of Fos+/Rspo2+ was lower in the Extinction group compared to the Non-Extinction group (Figures 1D, 1E and 1H). In contrast, the proportion of Fos+/Ppp1r1b+ was higher in the Extinction group compared to the Non-Extinction group while no difference was observed between the Non-Extinction and Non-Shock groups (Figures 1F, 1G and 1I). When BLA Ppp1r1b+ neurons were optogenetically inhibited during extinction retrieval, the neuronal activity of Rspo2+ cells (Fos+/Rspo2+) as well as the freezing level increased during extinction retrieval (Figures S1). This result suggests that Ppp1r1b+ neurons suppress Rspo2+ neurons during fear extinction retrieval, which is consistent with the previously reported feed-forward inhibition of BLA Rspo2+ cells by BLA Ppp1r1b+ cells (Kim et al., 2016).
Dynamics of individual BLA neurons throughout CFC and fear extinction
Next, we performed in vivo calcium imaging with a microendoscope to directly track individual BLA neuronal activity during CFC followed by contextual fear extinction. The genetically encoded calcium indicator GCaMP6f was expressed in BLA Rspo2+ and Ppp1r1b+ neurons by injecting adeno-associated virus 5 (AAV5)-human synapsin (hsyn):double-floxed inverse open reading frame (DIO)-GCaMP6f into the aBLA of Rspo2-Cre mice and the pBLA of Ppp1r1b-Cre mice, respectively (Figures 2A-2C). The efficiency of Cre-loxP recombination in Rspo2-Cre and Ppp1r1b-Cre mice lines are around 90% and 75%, respectively. (Kim et al., 2016). We used an automated sorting algorithm to identify individual neurons and tracked their longitudinal activity across days (Figures 2D, S2A, and S2B; see Methods) (Kitamura et al., 2017; Mukamel et al., 2009; Okuyama et al., 2016). Neuronal activity was analyzed under five conditions: during habituation to the conditioning chamber before CFC (Hab), after footshocks during CFC (Shock), fear retrieval (FR), extinction training (FET) and extinction retrieval (ER) (Figures 2D; see Methods). In order to quantify how neuronal activity changed across these conditions, the activity in each condition was explicitly compared to activity in another condition that served as a baseline, i.e. shock compared to habituation, fear retrieval compared to habituation, FET compared to fear retrieval, and extinction retrieval compared to fear retrieval (Figures 2E and 2F; see Methods). The cells with increased activity were referred to as Up cells, those with decreased activity were referred to as Down cells, and cells whose activity did not change were referred to as No change cells.
BLA Rspo2+ cells were predominantly activated by footshocks during CFC and fear retrieval, and inhibited during extinction training and extinction retrieval (Figures 2E, 2G, and S2C-S2E). Conversely, BLA Ppp1r1b+ neurons were predominantly activated during extinction training and extinction retrieval, and inhibited by footshocks and fear retrieval (Figures 2F, 2G, and S2F-S2H). A larger percentage of BLA Rspo2+ cells were activated during fear retrieval compared to extinction retrieval, whereas a larger percentage of BLA Ppp1r1b+ cells were activated during extinction retrieval compared to fear retrieval (Figure 2G). Compared to BLA Rspo2+ cells whose activity did not change in response to footshocks (Shock-No Change), shock-activated Rspo2+ cells (Shock-Up) were preferentially inhibited during extinction training and extinction retrieval (Figures 2H, 2J and 2K). Specifically, 30% of the shock-activated Rspo2+ cells were also activated during fear retrieval and inhibited during both extinction training and extinction retrieval (Figures 2J and 2K). Compared to BLA Ppp1r1b+ neurons whose activity did not change by extinction training (E.T.-No Change), Ppp1r1b+ neurons that were activated during extinction training (E.T.-Up) were preferentially reactivated during extinction retrieval (Figures 2I, 2L and 2M). Combined with the c-Fos quantification data (Figure 1), these results show that pBLA Ppp1r1b+ neurons are recruited during fear extinction training and reactivated during extinction memory retrieval whereas shock-activated aBLA Rspo2+ neurons are suppressed during these processes.
BLA Ppp1r1b+ neurons drive fear extinction learning
We next investigated the roles of Ppp1r1b+ and Rspo2+ neurons in fear extinction at the behavioral level using optogenetic manipulations. We injected a Cre-dependent AAV carrying ChR2 (channelrhodopsin-2) or ArchT (archaerhodopsin) into the aBLA of the Rspo2-Cre mice (Figure 3A and S3A) or the pBLA of the Ppp1r1b-Cre mice (Figure 3B and S3B). Mice underwent the three-day contextual fear extinction protocol as before (Figure 1A). Starting 5 min after the onset of extinction training on Day 2, 3 min of blue or green laser light was delivered during FET repeatedly at 2 min intervals (Figure 3C). Inhibition of Rspo2+ neurons and activation of Ppp1r1b+ neurons both facilitated fear extinction learning (Figures 3E and 3F). In contrast, activation of Rspo2+ neurons and inhibition of Ppp1r1b+ neurons both impaired fear extinction and extinction memory retrieval (Figures 3D and 3G). These results show that pBLA Ppp1r1b+ neurons are necessary and sufficient to form fear extinction memory, whereas aBLA Rspo2+ neurons antagonize it.
Fear extinction memory engram cells are formed in BLA Ppp1r1b+ neuronal population
As memories are stored in specific engram cell circuits (Tonegawa et al., 2015), we then investigated whether the engram for fear extinction memory can be identified in the pBLA Ppp1r1b+ neurons. By using our previously established engram technology (Liu et al., 2012; Ramirez et al., 2013; Redondo et al., 2014; Ryan et al., 2015), we injected AAV9-c-Fos-tTA (tetracycline transactivator) together with AAV9-TRE (tetracycline response element)-ChR2-EYFP (Ext-ChR2 group) or AAV9-TRE-EYFP (Ext-EYFP group) into the pBLA of C57BL/6 mice (Figures 4A and 4B; see Methods), and subjected them to a series of engram labeling and behavioral steps (Figure 4D; see Methods). This genetic manipulation permits labeling of engram neurons during memory formation or memory retrieval (Figures S4A-S4C) (Khalaf et al., 2018; Liu et al., 2012; Ramirez et al., 2013). During fear extinction training, formation of the putative fear extinction memory engram cells occurs in parallel with retrieval of the original fear memory. In order to isolate the putative fear extinction memory engram neurons, we used the retrieval stage of the fear extinction memory for their labelling (Day3) as fear memory expression will have subsided to the background level by then (Figure 4D).
A large fraction (79 ± 2%) of pBLA neurons activated by fear extinction retrieval were Ppp1r1b+ (Figure 4C). In order to access the contribution of food and water supplied in the home cage (HC) to the baseline activity of Ppp1r1b+ neurons, we set up another negative control group, HC-ChR2 group (Figure 4D). As expected, ChR2-EYFP+ cells in the HC-ChR2 group was very sparse compared to those in the ChR2-EYFP group (Figure 4B, S4C-S4D). Three groups of mice, Ext-ChR2 group, Ext-EYFP group and HC-ChR2 group, exhibited indistinguishable freezing levels during the first round of contextual fear extinction protocol from Day 1 to Day 3 (Figure 4E). To examine the roles of these labeled neurons in contextual fear extinction, we subjected the mice to another round of CFC in the same box for fear reinstatement (Day 5). When these neurons were optogenetically activated during the second round of fear extinction training on Day 6 (see Methods), the Ext-ChR2 group showed accelerated extinction compared to the Ext-EYFP and HC-ChR2 groups (Figure 4F). This reactivation-induced accelerated extinction was not observed when the laser was off (Figures S4E-S4F). These results show that the extinction memory engram is formed in the BLA Ppp1r1b+ neuron population.
BLA Ppp1r1b+ extinction engram cells are crucial for suppressing Rspo2+ fear cells
We then investigated whether pBLA Ppp1r1b+ extinction engram neurons are not only capable of driving fear extinction, but also necessary for maintaining extinction memory and for suppressing Rspo2+ fear cells. We labeled the pBLA extinction engram cells with ArchT-mCherry during fear extinction memory retrieval (Figures 5A-5C and S5A). As expected, a large proportion (75 ± 2%) of the pBLA neurons activated by fear extinction retrieval were PPP1R1B+ (Figures S5B-S5C). Optogenetic inhibition of these ArchT-labeled neurons specifically impaired the retrieval of fear extinction memory on Day 4 (Figure 5D) and failed to suppress BLA Rspo2+ fear neurons (Figures 5E-5G). The later is consistent with the feed-forward inhibition of Rspo2+ neurons by Ppp1r1b+ extinction neurons within the BLA (Kim et al., 2016). Together, these results demonstrate a causal role of pBLA Ppp1r1b+ engram cells in extinction memory. These fear extinction engram cells: i) are reactivated during extinction memory retrieval (Figure 2M), ii) have the capacity to drive fear extinction (Figure 4F), and iii) are necessary for suppressing the original fear memory (Figures 5D-5G).
Reward neurons and fear extinction engram neurons overlap extensively in the amygdala
Our previous study demonstrated that pBLA Ppp1r1b+ neurons responded to a variety of appetitive stimuli (Kim et al., 2016), whereas our present study shows that fear extinction memory is stored in the pBLA Ppp1r1b+ neuronal population (Figures 1-5). Therefore, we investigated the relationship between these two subsets of Ppp1r1b+ neurons. To label reward-responding pBLA neurons, AAV9-c-Fos-tTA and AVV9-TRE-EYFP virus were injected into pBLA of C57BL/6 mice on Dox-ON diet (Figure 6A). One week post-surgery, all mice were partially deprived of water for 24 hrs on Dox-ON diet (Day 1) and water was given as a reward on Dox-OFF diet (Day 2) (Figure 6B). Mice were then divided into W/W (Water/Water), W/F (Water/Food), W/Ext (Water/Extinction), and W/NonExt (Water/NonExtinction) groups, and kept on Dox-ON diet throughout the remainder of the experiment (Figure 6B; see Methods).
Neurons activated by water reward on Day 2 were labeled with EYFP, and their proportions of all neurons (EYFP+/DAPI+) were similar across all four groups (Figures 6C, 6F and S6A). Neurons activated by the stimulus delivered on Day 5, namely the second delivery of water reward (W/W group), food reward (W/F group), extinction memory retrieval (W/Ext group), and fear memory retrieval (W/NonExt group), were detected by immunohistochemistry for endogenous c-Fos. The proportions of c-Fos+ cells of all neurons (c-Fos+/DAPI+) were similar between W/W group, W/F group and W/Ext group, and were substantially higher compared to the W/NonExt group (Figures 6D, 6F, and S6A). The proportion of the c-Fos and EYFP double-positive cells of EYFP+ cells (c-Fos+ EYFP+/EYFP+) was used to measure the degree of similarity between the cells responding to the pair of stimuli used in each mouse group. This proportion reached as high as 40% in the W/W group, which represented the ceiling effect of a repeated presentation of the same reward. The proportion of W/Ext group was lower compared to W/W group but it did not differ significantly from the proportion of W/F group, and both of these groups and W/W group gave greater proportions than W/NonExt group (Figures 6E, 6F, and S6A). These results show that the overlap of pBLA Ppp1r1b+ neurons activated by fear extinction and water reward is as high as the overlap of pBLA Ppp1r1b+ neurons activated by two bona fide rewards (water and food).
Ppp1r1b+ fear extinction engram cells and natural reward cells are interchangeable
We then examined whether this observed cellular overlap could also be demonstrated at the behavioral level. To test whether optogenetic activation of fear extinction engram cells can drive appetitive behavior, four groups of C57BL/6 mice were set up (Figures 7A and 7B). In the Ext-ChR2 and Ext-EYFP groups, extinction engram cells were labeled with ChR2-EYFP and EYFP during extinction retrieval on Day 3, respectively (Figures 7B, S7A, and S7B). As controls, neurons activated by water reward and fear memory retrieval were labeled with ChR2 in Water-ChR2 group and CFC-ChR2 group, respectively (Figures 7B, S7C, and S7D). On Day 4, all mice were subjected to an optogenetic self-stimulation behavior test or optogenetic place preference test (Figures 7B, S7E and S7F; see Methods). 50% of Ext-ChR2 group mice and 67% of Water-ChR2 group mice displayed self-stimulation behavior, but not the Ext-EYFP group or CFC-ChR2 group (Figure 7C). In the optogenetic real-time place preference test, mice in the Ext-ChR2 group and Water-ChR2 group spent significantly more time on the stimulated side, whereas Ext-EYFP and CFC-ChR2 groups spent equal times on both sides of the chamber (Figures 7D, 7E, and S7G). Reciprocally, optogenetic activation of ChR2-expressing water reward cells accelerated fear extinction learning (Figures 7F and 7G). Thus, activation of fear extinction engram cells generates appetitive behaviors and activation of reward neurons facilitates fear extinction.
Discussion
In this study, we have shown that fear extinction requires new memory engram cells that are formed and stored within BLA Ppp1r1b+ reward neuronal population. The fear extinction engram cells are defined by their reactivation during extinction retrieval, and their sufficiency and necessity to drive fear extinction. The overlap of neuronal activation of extinction memory engram neurons and natural reward-responsive neurons is extensive and these two types of neurons are mutually interchangeable in driving reward functions and fear extinction behaviors. Thus, the shift from fear to safety in the fear extinction process is mediated by newly formed Ppp1r1b+ reward memory neurons and their inhibition of the original Rspo2+ fear memory neurons, both present within the BLA.
The inhibition of Rspo2+ fear neurons by Ppp1r1b+ extinction neurons is likely to be mediated by the local inhibitory interneurons in BLA (Kim et al., 2016), supporting the emerging notion that the switch between fear expression and fear extinction takes place within the BLA (Ehrlich et al., 2009; Grewe et al., 2017; Herry et al., 2008; Pare and Duvarci, 2012). Consistent with previous recording studies (Grewe et al., 2017; Herry et al., 2008), our study shows fear extinction is due to new learning rather than a loss of the original fear memory (Herry et al., 2010; Myers and Davis, 2007; Pavlov, 1927). The Ppp1r1b+ extinction neurons may correspond to the previously reported BLA neurons identified by in vivo recording, which were activated throughout extinction learning after cue fear conditioning (Herry et al., 2008). However, our findings extend beyond that study as we further demonstrated that the newly formed extinction memory is an appetitive memory and inhibits the original fear memory.
Fear extinction recruits the brain’s reward circuit
The finding that the fear extinction memory is a rewarding memory indicates that the emotional valence associated with CS is switched from negative to positive during fear extinction. Notably, the finding that activation of fear extinction engram cells can elicit a reward effect (Figure 7) supports the notion that the absence of expected punishments is in itself a reward. Therefore, fear extinction is positively reinforced by recruiting the brain’s reward circuitry. Previous studies correlated the reward system to fear extinction (Correia et al., 2016; Felsenberg et al., 2018), but their causal relationship was never demonstrated. In contrast, our results show a causal role of the reward system in fear extinction: the pBLA reward neuron population is a necessary component of the fear extinction memory. Furthermore, these findings provide strong evidence of the antagonistic interactions between aversive and appetitive systems that have been observed in Pavlovian fear conditioning (Nasser and McNally, 2012).
Multiple parallel circuits in fear extinction
In addition to direct feed-forward inhibition within BLA, BLA Ppp1r1b+ extinction neurons could also drive fear extinction through their interactions with other brain regions. As downstream targets of BLA, both intercalated cells (ITC) and central amygdala (CeA) have been shown to play important roles in fear extinction (Likhtik et al., 2008). Our previous study showed that BLA Ppp1r1b+ neurons provide monosynaptic projections to the Prkcd+ neurons located in the lateral subnuclei of CeA (CeL), which are also activated during fear extinction retrieval (Kim et al., 2017). Therefore, the fear extinction behavior can also be modulated via BLA Ppp1r1b+-CeA monosynaptic projection or BLA Ppp1r1b+-ITC-CeA di-synaptic projection. The infralimbic region (IL) of the prefrontal cortex is another critical cite for fear extinction (Burgos-Robles et al., 2007; Laurent and Westbrook, 2009; Vidal-Gonzalez et al., 2006) and its connections with BLA Ppp1r1b+ neurons may also be involved in this process (Cho et al., 2013; Kim et al., 2016; Likhtik et al., 2005; Quirk et al., 2006; Sotres-Bayon et al., 2012).
Intrinsic reinforcement signals during fear extinction learning
How Ppp1r1b+ cells are activated for the acquisition of fear extinction memory is another question that remains to be elucidated. Reinforcement learning theories would predict that acquisition of extinction memory is instructed by negative prediction error, in which the predictive value of the CS with respect to the occurrence of the US is compromised (McNally et al., 2011; Schultz, 2016). Given there is no obvious external US in the case of fear extinction, this stimulus may be provided internally and preferentially to Ppp1r1b neurons in the BLA. The identification of the agent and the circuit that provide this stimulus would be of fundamental importance.
Accumulating evidence suggests that dopamine reinforces fear extinction learning by signaling the omission of expected aversive US in both animals and humans (de Jong et al., 2019; Kalisch et al., 2019; Matsumoto and Hikosaka, 2009; Raczka et al., 2011; Ray Luo, 2018; Salinas-Hernandez et al., 2018). In the mesolimbic dopaminergic system, amygdala receives dopaminergic inputs from the ventral tegmental area (VTA) and substantia nigra (SNc) in the midbrain (Brinley-Reed and McDonald, 1999; Pezze and Feldon, 2004). Future studies are needed to investigate how dopamine functions as an instructive signal onto BLA neurons for fear extinction learning.
In conclusion, our study provides the empirical evidence for the key concept derived from learning theory that an omission of the expected punishment is rewarding(Felsenberg et al., 2018; Kim et al., 2006; Nasser and McNally, 2012), pinpointing potential targets for effective interventions of fear-related disorders (Shalev et al., 2017; Stein and Sareen, 2015).
Funding
This work was supported by the RIKEN Brain Science Institute, the Howard Hughes Medical Institute, and the JPB Foundation.
Author contributions
X.Z., J.K. and S.T. contributed to the study design. X.Z. contributed to the data collection and interpretation. X.Z. conducted the surgeries, behavior experiments and histological analyses. X.Z. and S.T. wrote the paper. All authors discussed and commented on the manuscript.
Competing interests
The authors declare no competing financial interests. Correspondence and requests for materials should be addressed to Susumu Tonegawa.
Data and material availability
AAV9-c-Fos-tTA, AAV9-TRE-ChR2-EYFP, AAV9-TRE-eArchT-mCherry, and AAV9-TRE-EYFP were developed at the Massachusetts Institute of Technology by the group of S.T.; virus plasmids are available through a material transfer agreement. The Rspo2-Cre transgenic mouse line was developed at RIKEN BSI by the group of Shigeyoshi Itohara and is available through a material transfer agreement. All data is available in the main text or the supplementary materials.
Acknowledgments
We thank X. Zhou, K. Marshall, B. Bane, F.J. Bushard and W. Yu for technical assistance; C. Sun and C.J. MacDonald for discussion on the calcium imaging data analysis; M. Pignatelli, S. Muralidhar, C. Sun and C.J. MacDonald for comments and discussion on the manuscript; and all members of the Tonegawa laboratory for their support.