Reactivation of hedonic but not sensory representations in human emotional learning

Learning which stimuli in our environment co-occur with painful or pleasurable events is critical for survival. Previous research has established the basic neural and behavioural mechanisms of aversive and appetitive conditioning; however, it is unclear what precisely is learned. Here we examined what aspects of the unconditioned stimulus (US) – sensory and hedonic – are transferred to the conditioned stimulus (CS). To decode the content of brain activation patterns elicited during appetitive (soft touch) and aversive (painful touch) conditioning of faces, a novel variation of representational similarity analysis (RSA) based on theoretically driven representational patterns of interest (POIs) was applied to fMRI data. Once face associations were learned through conditioning, globally the CS reactivated US representational patterns showing conditioning-dependent reactivation. More specifically, in higher order brain regions, the CS only reactivated hedonic but not sensory aspects of the US – suggesting that affective conditioning primarily carries forward the valence of the experience rather than its sensory origins.


Introduction
POIs for each ROI. Multiple regression was used to obtain beta coefficients for each POI 125 (Kryklywy, Ehlers, et al., 2021). D. RSA was performed separately on early, mid, and late CS-126 only trials to examine reactivation of US information over the course of conditioning. (Bayesian) 8 multiple regression was used in order to quantify the US model reactivation by CS-only data 128 globally and BIC identified components were used to quantify which US aspects were reactivated 129 by CS.

151
The goal of the current study was to determine the extent and content of US pattern 152 reactivation by the CS once associations were learned. This was achieved by comparing patterns 153 of information representation identified during CS-only conditions early, mid, and late in 154 conditioning to previously identified representational patterns observed in CS-US paired trials.

155
The extent was assessed as both a global reactivation of US representational patterns, and the 156 reactivation of specific informative patterns of interest (POIs, Figure 1b), which further provided 157 information about the content of reactivation.

158
After initial preprocessing and first-level analysis of fMRI data, representational 159 similarity analysis (RSA) was conducted on the patterns of BOLD activity within predefined

162
Representational information in BOLD responses to the US was identified by using Bayesian 163 Information Criterion (BIC) to fit predefined POIs -idealized similarity matrices modelling 164 specific information content (e.g., the valence of the touch stimulus) -to the observed US 165 similarity data ( Figure 1c). The representation of appetitive and aversive US, independent from 166 CS-only trials, are focus of a different study (Kryklywy, Ehlers, et al., 2021).

167
To identify the extent to which US patterns were reactivated by CS following learning, a 168 reconstructed measure of US activation similarity (rUS) was fit to CS similarity data from early-, 169 mid-, and late-conditioning trials. rUS were constructed by summing the scaled contribution of 170 each POI contributing to overall similarity in an ROI (identified though hypothesis-driven PCM; 171 see (Kryklywy, Ehlers, et al., 2021). Next, to examine the degree to which each specific US-172 defined POI was comparable to the pattern of CS activation -the content driven reactivation -separate Bayesian linear models with the combination of US-defined POIs as predictors, and 174 rUS and CS representational patterns as outcome variables were fitted in order to compare the 175 (beta) weight of each predictor for the CS representational pattern to that of the US (see Figure   176 1d).

177
A reactivation of the following US-defined POIs is interpreted as reactivation of hedonic  Primary Somatosensory Cortex (S1): Globally, the rUS pattern predicted the CS 188 representation in mid and late conditioning, while the association was at trend level for early 189 conditioning (p = .069) (see Table 1). Thus, there was some increase in the reliability of global significantly predicted by V1 rUS during early, mid, and late conditioning, indicating no effect 198 of conditioning (see Table 1). Likewise, the reactivation of the POI 'Experimental Task' was 199 consistent with a contribution to US representational patterns during all conditioning phases (see  representational patterns (see Figure 2). Taken together these results indicate no effect of 206 affective pairings on representational patterns in these visual regions. suggesting overall a slightly delayed response to learning (see Table 1). The analysis of 211 component reactivation, however, revealed that only 'Touch Valence' showed an effect 212 consistent with US data and only early in conditioning (see Figure 2).Thus, whereas the 213 amygdala was the only ROI to recapitulate the valence of the US with appetitive and aversive at   Anterior cingulate cortex (ACC): As for the vmPFC, ACC rUS could not predict ACC-225 CS representational patterns in early conditioning, but was highly predictive for both mid and 226 late conditioning (see Table 1). These results indicate that ACC may represent affective 227 associations acquired though classical conditioning. US-consistent component reactivation for 228 'Experimental Task' (similarity between all conditions within a task) by CS was observed mid 229 and late conditioning. In addition, while 'Aversive Pressure' reactivation by CS data did not 230 reach the same levels as the US, it becomes apparent that an increase in reactivation from early 231 to mid and late conditioning is induced by conditioning (see Figure 2).

Insula (anterior and posterior):
Here again, no significant relationship between the 233 anterior insula CS representational patterns and anterior insula rUS patterns was found during 234 early conditioning, but was observed during mid and late conditioning. Similarly, the posterior 235 insula rUS pattern was not predictive of the CS representational pattern during early 236 conditioning, but was during mid and late conditioning (see Table 1). This suggests that global 237 activation patterns in anterior and posterior insula represent conditioned affective associations.

238
The anterior insula showed clear conditioning effects, with US-consistent contribution of 239 'Experimental Task' and 'Aversive Pressure' in mid and late conditioning. The pattern of results 240 suggests a bias for the anterior insula to carry forward negative hedonic information during 241 conditioning. We also observed conditioning independent-reactivation of 'Negative Events' 13 (both aversive pressure and absence of pleasurable touch) in early conditioning. On the other 243 hand, the posterior insula only showed a US-consistent contribution of 'Experimental Task' in 244 mid and late conditioning. This might represent the differences between the specific tactile and 245 facial stimuli between the appetitive and aversive conditioning task (see Figure 2). Overall, the 246 current data indicates some distinct patterns of reactivation in anterior and posterior insula 247 supporting the notion that these subregions play functionally distinct roles in representing 248 information about conditioned cues.

Discussion
In this study we examined whether, with emotional learning, the initially neutral conditioned stimulus (CS) reactivates patterns of activation elicited by the unconditioned stimulus. We further probed whether neural population codes for the CS represent non-hedonic sensory and/or hedonic aspects of the appetitive or aversive unconditioned stimulus. In other words, we asked if what is carried forward in conditioning is the entire sensory and hedonic construct that a US encompasses, or whether only hedonic aspects of the appetitive or aversive stimulus become associated with the cue. We used representational similarity analysis (RSA) with a theoretically based version of pattern component modeling (PCM) on fMRI data to decode the content of neural representations observed over the course of aversive and appetitive classical conditioning. The data revealed that, for higher-order regions of interest, a significant amount of variance in the CS data could be explained by patterns elicited by the US, indicating US pattern reactivation by CS. Critically, this effect was only found after affective pairing had been encountered, and not prior to the learning process. By employing pattern component modeling (PCM) and determining the best combination of patterns of interest (POIs) for different ROIs, we were further able to model the distinct content of information that is represented by each brain region for the US and the degree to which it was reactivated by the CS. In primary sensory regions, the CS primarily reactivated components of the US that do not rely on learning.
In contrast, several brain regions implicated in emotional learning showed reactivation of those components representing hedonic value of the USproviding evidence for a dominance of stimulus-response learning.
Global US representation pattern reactivation by CS. We first wanted to establish whether, through conditioning, the representation of an initially neutral stimulus changes to This pattern indicates that reactivation is not a result of conditioned associations but rather of experimental phase overlapping features inherent to the task (e.g., the stable visual properties of the stimuli). In contrast, amygdala, vmPFC, ACC as well as anterior and posterior insula predicted CS representational patterns from US data only after conditioned associations had developed. While one previous study (Onat & Büchel, 2015) has found preliminary evidence for such an effect for fear conditioning in the insula, the present study shows that this finding can be generalized to both appetitive and aversive associative learning, and to other brain regions. Thus, in this study we provide strong evidence suggesting that the basic mechanisms observed behaviorally in conditioning -that is, that the CS elicits the same response as the US (Maren, 2001;Martin-Soelch et al., 2007;Pavlov, 1927) -are also represented in the brain.

US component reactivation by CS.
After having established that, globally, the CS reactivates US representational patterns after associative learning, we next focused on the  Task' indicating pain task vs. brush task) reactivation that did not depend on conditioning, consistent with the fact that, the visual input associated with each task did not change over the course of learning.
In contrast, component reactivation in the vmPFC and, to some degree, in the amygdala and the anterior insula, provide answers to the question of what aspects of the US become attached to the CS. A substantial body of literature has delineated complementary roles for the amygdala and vmPFC in establishing conditioning (Schoenbaum & Roesch, 2005). In the current study, in the vmPFC, representations of both appetitive and aversive touch (POIs: 'Aversive Pressure', 'Appetitive Brush') were reactivated. Of note, reactivation of appetitive touch was observed mid conditioning, while aversive touch was observed during late conditioning only.
This might indicate that the association with appetitive touch developed more quickly than with aversive pressure but that at the same time habituation to the appetitive brush is faster than to aversive pressure (Triscoli et al., 2014). with somewhat inconsistent results. A recent meta-analysis (Kurth et al., 2010) revealed that most functions investigated have been associated with activation in the anterior portion, especially emotional processing. Only sensorimotor processing has been exclusively mapped onto the posterior insula while pain processing has been shown to involve the entire insula (Craig, 2002). In the light of these meta-analysis findings, the current results of reactivation of negative information representation in the anterior insula, especially without reactivation of nonspecific sensory input (reactivation POI: 'Non-specific Touch') strongly suggest reactivation of hedonic but not sensory US components. In contrast, the task-based (reactivation POI: 'Experimental Task') in the posterior insula could be explained by the differences in sensory input between appetitive and aversive tasks, i.e. female faces paired with appetitive brush and male faces paired with aversive pressure respectively. In summary, after demonstrating US component reactivation by CS with conditioning, the current findings show that several brain regions that have been previously associated with conditioning are biased to reactivate components that carry hedonic information instead of purely tactile information. Thus, the pattern of results observed here suggests that what is carried forward in conditioning and becomes associated with the CS is primarily the affective attachment with the stimulus rather than the pure sensory experience. In other words, when we are exposed to a conditioned stimulus we re-experience the pleasant or unpleasant feelings elicited by the US rather than the sensory stimulation. Taken together, the current analysis provides support for the notion of stimulusresponse conditioning in which a CS reproduces the unconditioned response rather than the perceptual experience of the US.
In conclusion, we demonstrated for the first time that, in conditioning, a conditioned stimulus reactivates the pattern of activation initially elicited by the unconditioned stimulus in several brain regions. The results further show that it is primarily the hedonic components of the initial experience, rather than discriminative sensation, that is reproduced when we encounter a cue that predicts it. Thus, when we encounter a cue signaling pain or pleasure, what we carry forward is the emotional meaning we attach to it rather than the sensory experience.

Participants
Data from 71 young, healthy participants (age: 21.1 ± 2.8 years, 41 females) was included in the analysis. Initially, 107 participants were recruited from Cornell University to participate in a brain imaging study of appetitive and aversive classical conditioning tasks. A number of participants had to be excluded for the following reasons: 20 participants had missing data (imaging run, stimulus onset files, motion correction files) while multi-echo preprocessing described below failed for 16 participants. All participants gave written, informed consent and had normal or corrected-to-normal vision. Participants were pre-screened for a history of anxiety and depression as well as other psychopathology, epilepsy and brain surgery in addition to general suitability for fMRI data collection. Pre-screening was followed up in person by an additional interview to ensure inclusion criteria were met. Due to the fact that this study was conducted as part of a larger research program, all participants were genotyped.
The neural representation of the appetitive and aversive US as well as the development of the PCM derivative and other methodical details have been described in (Kryklywy, Ehlers, et al., 2021).

Stimulus and apparatus
Six faces were chosen from the Karolinska directed emotional faces, comprising three male and three female exemplars each with a neutral expression (Goeleven, De Raedt, Leyman, & Verschuere, 2008). These faces were used as the conditioned stimuli (CS) in a classical conditioning paradigm. The (US) consisted of either an aversive pressure delivered to the right thumb, or an appetitive brush stroke to the participant's forearm. Aversive pressure stimuli were delivered using a custom designed hydraulic device, similar to those used in previous studies (Giesecke et al., 2004;López-Solà et al., 2010), capable of transmitting controlled pressure to 1 cm 2 surface placed on the subjects' right thumbnail. In individual calibration sessions, it was ensured that the pressure intensity was aversive but not excessively painful. Appetitive brush strokes were manually applied to the left forearm lasting ~4s. Individual subjective responding to brush stimuli were recorded in a separate session prior to all experimental scanning, with only participant who responded positively to the manipulation invited to participant in the scanning session.

Stimulus ratings
As a measure of subjective stimulus assessment and conditioning, participants were asked to rate the likeability and trustworthiness of the faces used as CS+ and CS-stimuli on a scale from 1-100 (1) before and (2) after conditioning as a measure of conditioning. CS Discrimination scores ([CS+ minus CS-]) before and after conditioning were calculated and compared using t-tests. Due to technical difficulties, stimulus ratings were only available for 69 of the 71 participants included in the analysis.

Experimental tasks
While undergoing functional MR scanning, participants completed two unique conditioning tasks with nearly identical structure modeled after Visser and colleagues (2015).
These tasks differed from each other only in the nature of the tactile unconditioned stimulus (US; see above), and the gender of the face stimuli. In each task, participants completed seven CSonly blocks interleaved with six CS-US paired blocks (see Figure 2a). Single blocks of either the CS-only or the CS-US pairing entailed one presentation of each of the three male or female face stimuli used in that task. The order of the CS+ and the CS-faces was randomized within each CS-US block. Individual trials started with an initial fixation period (19500 ms) followed by the presentation of a face stimulus (4000 ms). The fixed and long interstimulus interval was included in the experimental design to reduce intrinsic noise correlations (Visser et al., 2013). During CSonly trials, all faces were presented without tactile stimulation (see Figure 1d). During CS-US paired trials, two of three facial stimuli were paired with tactile stimulation, thus creating two CS+ and one CS-face stimuli (see Figure 2b). The US was delivered from the midpoint after the visual stimulus presentation (2000 ms post-onset), and remained for the duration of the visual presentation (2000 ms). Face pairings were randomly assigned for each participant but held constant across the duration of the experiment.

Acquisition
Scanning was conducted on a 3 Tesla GE Discovery magnetic resonance scanner using a 32-channel head coil at Cornell University. For each subject, a T1-weighted MPRAGE sequence was used to obtain high-resolution anatomical images (repetition time (TR) = 7 ms, echo time (TE) = 3.42 ms, field of view (FOV) 256 x 256 mm slice thickness 1 mm, 176 slices).
The functional tasks were acquired with the following multi-echo (ME) EPI sequence: TR = 2000 ms, TE1 = 11.7 ms, TE2 = 24.2 ms and TE3 = 37.1 ms, flip angle 77°; FOV 240 x 240 mm. A total of 102 slices was acquired with a voxel size of 3 x 3 x 3 mm. Pulse and respiration data were acquired with scanner-integrated devices.

Preprocessing
Multi-echo independent component analysis (ME-ICA, meica.py version 3.2 beta1) was used to denoise the multi-echo fMRI data. An optimally combined (OC) dataset was generated from the functional multi-echo data by taking a weighted summation of the three echoes, using an exponential T2* weighting approach (Posse et al., 1999). Multi-echo principal components analysis was first applied to the OC dataset to reduce the data dimensionality.
Spatial independent component analysis (ICA) was then applied and the independent component time-series were fit to the pre-processed time-series from each of the three echoes to generate ICA weights for each echo. These weights were subsequently fitted to the linear TE-dependence and TE-independence models to generate F-statistics and component-level κ and ρ values, which respectively indicate blood-oxygen-level-dependent (BOLD) and non-BOLD weightings (Kundu et al., 2012). The κ and ρ metrics were then used to identify non-BOLD-like components to be regressed out of the OC dataset as noise regressors (Kundu et al., 2013).

Regions of interest
To assess tactile (aversive pressure, appetitive brush) and hedonic representations in neural patterns, eight bilateral regions of interest (ROIs) were generated from the standard anatomical atlas (MNI_caez_ml_18) implemented in the Analysis of Functional NeuroImages (AFNI) software package (Cox, 1996): primary somatosensory cortex (S1), primary/secondary visual cortex (V1) were selected as the primary sites of tactile and visual information respectively. In addition, ventral visual structures (VVS) were chosen due to their role in visual classification (Kanwisher et al., 1997;Kravitz et al., 2013). Amygdala, ventromedial prefrontal cortex (vmPFC), anterior cingulate cortex (ACC) and insula were further selected for their hypothesized roles in affect processing (Anderson & Phelps, 2002;Craig, 2002;Winecoff et al., 2013) and pain representations (Kragel et al., 2018;Orenius et al., 2017). The insula was further divided into an anterior and posterior portion due to its functional and anatomical subdivisions (Nieuwenhuys, 2012) for a total of eight ROIs.

Representational similarity analysis
Data analysis of the fMRI data was conducted using Analysis of Functional NeuroImages (AFNI) software (Cox, 1996). Regressor files of interest were generated for all individual trials across the experiment, modelling the time course of each stimulus presentation during each run (36 total events). The relevant hemodynamic response function was fit to each regressor to perform linear regression modeling. This resulted in a β coefficient and t value for each voxel and regressor. To facilitate group analysis, each individual's data were transformed into the standard brain space of Montreal Neurological Institute (MNI).
In order to identify the representational pattern elicited by the experimental stimuli, representational similarity analysis (RSA) was performed using the Python package PyMVPA (Hanke et al., 2009

Pattern Component Modeling
In order to characterize the content of CS (and US) representation in key regions of interest, we developed a theory-guided implementation of Pattern Component Modeling (PCM) (Diedrichsen et al., 2018;Kriegeskorte & Kievit, 2013;Kryklywy, Ehlers, et al., 2021;Kryklywy, Forys, et al., 2021). The details are described in Kryklywy, Ehlers, et al., 2021. In brief, we created 13 patterns of interest (POIs) to represent dissociable correlation patterns that would be observed in the experimental data if it would contain perfect representation of distinct theoretically-derived constructs ( Fig 1C). POIs were constructed for 1) Experimental Task, 2) Non-Specific Touch, 3) Specific Touch, 4) Appetitive Brush, 5) Aversive Pressure, 6) Touch Valence, 7) Positive Events, 8) Negative Events, 9) All Valence, 10) Salience, 11) Face Stimulus, 12) Violation of Expectation and 13) Temporal Adjacency (see Figure 1c). In order to determine the POI combinations that best explained the observed correlation in the US data in each ROI, a Bayesian Information Criterion (BIC) analysis and multiple regression implemented in our R package 'PCMforR' (Kryklywy, Forys, et al., 2021) were conducted. A reconstructed US (rUS) pattern was built from identified POIs in order to reduce noise. The rUS pattern was first used as predictor for the CS pattern in early, mid and later conditioning to determine the global reactivation of US patterns by CS-only data. Subsequently, the US identified POIs were used as predictors in Bayesian linear models for both rUS and CS data in order to compare the contribution (beta weight) of each POI between rUS and CS data. For that purpose the R package 'BayesFactor' (Morey et al., 2018) was used. The Bayesian linear model was estimated with 1,000,000 iterations allowing us to extract mean beta weights for each POI and their 95 % credible intervals (CrIs). In order to determine whether the contribution of each POI to the CS data is comparable to that of the US data, we adapted an approach developed to assess the robustness of replications (LeBel et al., 2018) that has recently also been employed in a Bayesian framework (Kuhn et al., 2021). While we are not comparing replication attempts, we have adapted the measure of consistency described previously (LeBel et al., 2018) in such a way that consistency between US and CS data is assumed when the beta weight point estimate obtained from CS data of any given POI is included in the credible interval of the beta weight obtained from US data for the same POI.