Pain-related learning signals in the human insula

Pain is not only a perceptual phenomenon, but also a preeminent learning signal. In reinforcement learning models, prediction errors (PEs) play a crucial role, i.e. the mismatch between expectation and sensory input. In particular, advanced learning models require the representation of different types of PEs, namely signed PEs (whether more or less pain was expected) to specify the direction of learning, and unsigned PEs (the absolute deviation from an expectation) to adapt the learning rate. The insula has been shown to play an important role in pain intensity coding and in signaling surprise. However, mainly unsigned PEs could be identified in the anterior insula. It remains an open question whether these PEs are specific to pain, and whether signed PEs are also represented in the insula. To answer these questions, 47 subjects learned associations of two conditioned stimuli (CS) with four unconditioned stimuli (US; painful heat or loud sound, of one low and one high intensity each) while undergoing functional magnetic resonance imaging (fMRI) and skin conductance response (SCR) measurements. CS-US associations reversed multiple times between intensities and between sensory modalities, generating frequent PEs. SCRs indicated comparable nonspecific characteristics of the two modalities. fMRI analyses focusing on the insular and opercular cortices contralateral to painful stimulation showed that activation in the anterior insula correlated with unsigned intensity PEs. Importantly, this unsigned PE signal was similar for pain and aversive sounds and also modality PEs, indicating an unspecific aversive surprise signal. Conversely, signed pain intensity PE signals were modality-specific and located in the dorsal posterior insula, an area previously implicated in pain intensity processing. Previous studies have identified abnormal insula function and abnormal learning as potential causes of pain chronification. Our findings link these results and suggest one potential mechanism, namely a misrepresentation of learning relevant prediction errors in the insular cortex.


27
Apart from its role in signaling tissue damage, pain is increasingly considered to be a preeminent teaching signal [1,2] in 28 the context of reinforcement learning models [3]. For example, delta rule learning models in classical fear conditioning, 29 such as the , almost exclusively employ pain as unconditioned stimulus (US). In this and 30 similar models, the value of predictive cues (conditioned stimuli, CS) is updated by the difference between the expected 31 and the experienced outcome, i.e. a prediction error (PE). In this case the PE needs to be signed and signals the direction 32 of the difference between expectation and event, i.e. whether the outcome is better or worse than expected. In the case 33 of an aversive event like painful stimulation, this is relevant for shaping future behavior. Reinforcement learning 34 particularly relies on these valences, and different neuronal correlates have been reported for aversive compared to 35 appetitive PEs [5][6][7][8]. This has important clinical implications, as pathological learning mechanisms [1,9] have been 36 reported in chronic pain. 37 However, PEs can also be computed as unsigned [10][11][12]. An unsigned PE simply indicates the presence of an 38 unexpected event regardless of its valence. Unsigned PEs are therefore conceptually related to constructs like surprise 39 or salience, and may contain information concerning the urgency of behavioral change [13]. Computational models of 40 learning can include either type of PE,or both [4,10,[14][15][16] -for example, the Pearce-Hall model incorporates the 41 unsigned PE as a factor to increase the learning rate after highly incongruent (surprising) events [14,17], whereas a 42 hybrid-model contains both terms [10,17,18]. 43 Previous studies investigating PEs in the context of aversive learning have observed signal changes in the anterior insula 44 related to unsigned PEs [6,12,[19][20][21]. Unfortunately, in many studies, a signed PE signal is non-orthogonal to stimulus 45 expectation, which poses a problem with a short interval between CS and US, and the low temporal resolution of 46 functional magnetic resonance imaging (fMRI). Consequently, these studies were suboptimal to investigate signed PEs. 47 Granted that unsigned PEs resemble a surprise signal, they could plausibly involve similar regions for all surprising 48 events, independent of the stimulus sensory modality. Crucially, the representation of unsigned pain PEs in the anterior 49 insula [12,19] raises the question of whether these are specific to pain, or simply related to aversive events. 50 To further investigate the existence of signed PEs and the modality-specificity of unsigned PEs, as well as the underlying 51 neuronal mechanisms, we used a Pavlovian transreinforcer reversal learning paradigm [22,23]. This involves two visual 52 stimuli as CS, and two intensities of painful heat or loud sounds as US (for brevity, these are referred to as "pain" and 53 "sound" forthwith). Across sensory modalities, stimuli were chosen to be roughly comparable in salience as indicated by 54 similar skin conductance responses (SCR) [24]. Reversals occurred between US intensity but within US modality (e.g. CS 55 predicting low pain will next predict high pain), or within US intensity but between US modality (e.g. CS predicting loud 56

73
In two sessions with 64 trials each, 47 subjects learned associations of two conditioned stimuli (fractal pictures; CS) with 74 individually calibrated unconditioned stimuli (US; two painful heat intensities and two loud sound intensities) (Figure 1a, 75 b). In each trial, either CS appeared, followed by symbols of all four US, from which subjects selected the US they 76 expected ( Figure 1c). One of the US was then applied. CS/US associations were deterministic, but importantly, 77 associations frequently reversed and had to be relearned over the course of the experiment (Figure 2). Reversals 78 occurred unannounced after a randomized number of trials. Reversals could occur along the modality dimension or the 79 intensity dimension, but not both simultaneously (e.g., no low heat to high sound reversals). See Materials  subjects were asked to choose which US they expected to follow. The US was then applied and rated in terms of its painfulness (for 87 pain)/unpleasantness (for sound). EDA, electrodermal activity; CS, conditioned stimuli; US, unconditioned stimuli.   110 aggregate ratings of all pain and sound trials. Circles with error bars show the mean ± standard errors over all subject means. Subject 111 means are displayed as smaller circles. Violin plots aggregate over subject means. The grey dashed line is the "intended" rating as 112 per calibration (VAS25 for low, VAS75 for high intensities). (B) Performance pre and post reversals, aggregated over all subjects.

113
Circles indicate the performance during (peri)reversal trials, first averaged within and then between subjects (mean ± standard 114 errors). The dashed horizontal line marks chance level (25%, i.e. 1 of 4 options). The dashed vertical line indicates contingency 115 reversal, with relative trial number 0 as the reversal trial. Note that no difference arose between trials preceding and following 116 modality versus intensity reversals (also see Figure 2 for aspects concerning contingency reversals). Furthermore, the steep increase 117 in performance after trial number 0 indicates, on average, rapid learning of the new contingency.

119
Behavioral results: Learning performance 120 The next behavioral question was whether the subjects learned the CS/US contingencies. Figure 3b depicts mean 121 performance in predicting the US currently associated with the CS, in relation to the reversals of the association. 122 Combining reversal types and comparing performance at the single trials prior reversal, at reversal, and after reversal, 123 A B 8 we find pre-reversal performance to be above chance level (t[79] = 13.8, p ≈ 0), at reversal performance below chance 124 (t[79] = -15.9, p ≈ 0), and post-reversal performance back above chance (t[79] = 19.5, p ≈ 0). 125

126
The major question concerning SCR results were whether any differences between the US arose, and how the different 127 PE types would be reflected in this psychophysiological measure of nonspecific characteristics or processes like arousal, 128 salience, or surprise. SCR following sound has a faster onset than that following heat pain stimuli (Figure 4a; see 129 Materials and Methods concerning the different response windows). The average amplitude of pain-related SCR was 130 higher than the average of sound-related SCR, but this difference only showed a trend towards significance (main effect 131 modality, t[4399] = -1.7228, p = 0.08499). Instead, the difference is subsumed by a larger difference between low and 132 high stimuli in the pain modality, as compared to that in the sound modality (modality*intensity, Further investigating SCR differences following PEs, we first distinguished SCR when subjects correctly predicted the US 138 from trials when either an intensity PE or modality PE was made ( Figure 4c). The following statistics include all trials -139 not just reversals -where an incorrect prediction was made. As shown in the first block (grey bars), over all US and 140 controlling for modality and intensity, SCR following unsigned intensity PEs are larger than those following no PE 141 (intPE>noPE, t[4397] = 4.336, p = 2 x 10 -05 ), while SCR following modality PEs are even larger (modPE>noPE, t[4397] = 142 12.345, p = 2 x 10 -34 ; modPE>intPE, t[4397] = 6.398, p = 2 x 10 -10 ). 143 Notably, we performed an adjunct analysis on whether the direction of intensity PEs (i.e. signed intensity PEs) had an 144 impact. We obtained mean SCR differences per subject between no PE and intensity PE trials for each modality and 145 intensity separately, thereby accounting for higher intensity-related base SCRs; next, we contrasted these (now signed) 146 PE-related differences between the low and high intensity. For pain, results indicate no effect (PE-related SCR difference 147 for low pain mean±SE 0.036±0.052, for high pain 0.0922±0.0622, paired t-test t[36] = -0.725, p = 0.4731), while for 148 sound, a more ambiguous yet non-significant result arose (PE-related SCR difference for low sound mean±SE 149 0.060±0.054, for high sound 0.199±0.054, paired t-test t[35] = -1.931, p = 0.0616). 150 In four consequent analyses, we investigated differences in SCR following PEs in all US separately, meaning that all 151 intensity PEs are now signed. Results indicate that the intPE>noPE effect of the global analysis is driven by this contrast 152 in the high sound US (light blue bars, t[1119] = 4.732, p = 3 x 10 -6 ); it does not reach significance following any other US. 153 9 Conversely, modality PEs are followed by larger SCR in all US (all modPE>noPE p < 0.001; smallest effect modPE>intPE 154 t[1090] = 2.045, p = 0.041079). 155 Figure 4d shows the average perireversal trial effect on SCR, over all US. It shows a large increase in SCR during both 156 modality and intensity reversals; note that this analysis does not consider actual subject expectation, just the position 157 related to the reversal trial. SCR is highest during the reversal trial, and rapidly reaches a lower plateau even one trial 158 later. Comparing the pre-reversal trial to immediate post-reversal (trials -1 to +1), SCR is not significantly different if a 159 modality reversal occurred (p = 0.54704); this is also the case if an intensity reversal occurred (p = 0.071164).

211
Having ascertained strictly stimulus-related effects, our next analysis included an investigation of unsigned intensity PEs 212 within and between either modality ( Figure 6). The guiding question here was whether any differences and 213 commonalities between the modalities would emerge. Since we used the actual expectation queried from subjects, 214 "prediction error" here means that subjects explicitly expected one intensity but received the other. Consequently, the 215 unsigned PE implies some extent of surprise. 216 C 13 In both modalities, widespread activation was observed. However, conjunction analyses revealed that the majority of 217 the observed activation actually overlapped between the modalities (green in Figure 6). The anterior insula constituted 218 the dominant cluster of this overlap, with symmetric bilateral peaks (XYZMNI = 34.6/23.5/-1.5, T = 5.8, p[corr. wb.] = 1 x 219 10 -04 ); whole brain-significant frontal (medial and lateral), temporal and parietal activation was also observed 220 (Supporting Figure 8). 221 222 223 Figure 6. Brain activation following unsigned intensity prediction errors in pain (red/yellow) and sound (blue), including overlaps as 224 per conjunction analyses (green). Peak activation following either modality is located in the anterior insula (aIns1) and is subsumed in 225 the common activation. Activations are overlaid on an average brain surface; for display purposes, activations in the whole brain

231
Two aspects were of particular interest to us considering unsigned intensity PE results: First, that brain activation related 232 to unsigned intensity PEs ( Figure 6) was distinct from the intensity-related activation ( Figure 5). Second, the fMRI signal 233 of the common activation in the anterior insula clearly indicated that modality PEs are likewise encoded in this area. 234 Modality prediction errors 235 Following these two observations, we proceeded to investigate the nature of the overlap between the two types of PE. 236 Like with unsigned intensity PEs, we observed widespread activation following each modality PE separately (Figure 7). 237 14 Likewise, all unimodal activation is subsumed in the conjunction analysis, which indicates a large dorsal anterior insula 238 cluster in our region of interest T = 5.4,p[corr.] = 5 x 10 -05 ). Beyond this region, widespread 239 common activation is observed, for example, in the superior parietal lobule, precuneus, temporo-parietal junction, 240 middle frontal gyrus and frontal operculum, and medial orbital gyrus (Supporting Figure 10). 241 242 243 Figure 7. Brain activation following modality prediction errors in pain (red/yellow) and sound (blue) activation, including overlaps as 244 per conjunction analyses (green). As with unsigned intensity PEs, peak activation following modality PEs in either modality is located 245 in the anterior insula (aIns1) and is largely subsumed in the common activation. Activations are overlaid on an average brain surface;

252
Overlap of unsigned prediction errors 253 As a next step, we wanted to more formally assess the apparent overlap between both types of unsigned PEs. To do so, 254 we simply computed the conjunction between unsigned intensity and modality PE (Figure 8). This analysis corroborated 255 the anterior insula peak determined by separate analyses above. Furthermore, activation extended dorsally through the 256 middle frontal gyrus and also included medial prefrontal areas adjacent to the dorsal anterior cingulate cortex.

266
Signed intensity prediction errors 267 After ascertaining the effects for unsigned PEs for both intensity and modality, the final question for our fMRI data 268 referred to differences and commonalities following signed intensity PEs, i.e. correlations of brain activation with higher-269 than-expected intensity ( Using a Pavlovian learning paradigm with frequent reversals within and across aversive modalities in combination with 294 SCR recordings and high resolution fMRI, we were able to investigate signed and unsigned representations of PEs in the 295 human brain. The data showed an unsigned representation of intensity PEs in the anterior insula indistinguishable for 296 pain and aversive sounds, supporting a role of the anterior insula in coding unspecific arousal or salience. In addition, the 297 same part of the anterior insula also strongly activated for PEs concerning stimulus modality. Most importantly, we 298 could identify a circumscribed part of the dorsal posterior insula representing a signed PE for pain only, collocated with 299 areas processing pain intensity per se. 300 The parallel assessment of SCR, behavioral ratings for both expectation and outcome, as well as fMRI recordings allowed 301 us to investigate PEs in a multimodal fashion. Previous studies investigated PEs using cue-based pain paradigms 302 [12,19,21,32]. In these paradigms, a cue predicts a pain intensity with a certain probability. However, the probability also 303 determines the number of trials in which a PE occurs. This can lead to unbalanced designs in which certain PEs occur 304 much more frequently than others. In addition, the fixed association of a specific cue with an outcome risks that specific 305 features of the cue influence PE processing. Adopting a Pavlovian transreinforcer paradigm ameliorates these 306 shortcomings, and requires frequent relearning of contingencies and thus generates frequent PEs [22,23]. By defining a 307 Markovian transition structure, we also controlled the nature of reversals; we confined our experiment to within-308 intensity/between-modality, and between-intensity/within-modality reversals. Finally, introducing two CS in our task 309 increased task difficulty. 310 We explicitly included expectation ratings, which allowed us to use the difference between the US and its expectation as 311 a rating-derived PE [22]. Compared to model-derived PEs, this can account for within-subject differences in learning and 312 can also capture PEs in erratic behaviors difficult to model in formal reinforcement learning models. 313 Although we aimed to perfectly match salience between stimulus modalities, high intensity painful stimuli lead to higher 314 SCR activation compared to low pain or either sound intensity (Figure 4), even though average SCR amplitudes between 315 modalities were not statistically different. Technically, this is related to the fact that we were not able to increase sound 316 pressure levels above a certain level [33] to avoid harm for the volunteers. However, the fMRI signal changes in the 317 anterior insula for unsigned intensity PEs were similar for pain and sound, suggesting that the residual differences in SCR 318 did not affect our results ( Figure 6, Figure 7, Figure 8). In addition, previous accounts [34] have indicated that higher 319 salience enhances memory performance. We tested this and observe no such effect: learning performance did not 320 substantially differ between any of the US groups (Supporting Figure 14). 321 We have replicated findings concerning pain-related activation in the dorsal posterior insula/parietal operculum and 322 sound-related activation in the superior temporal gyrus [24]. Previously, these areas showed a clear effect of pain and 323 sound stimulation, respectively, but a crucial intensity-related increase in activation that is shallower or absent in non-324 noxious intensities. In contrast to the previous study, we see a stronger correlation of the BOLD response to sound 325 ratings, possibly owing to the higher intensities employed here. 326 Also in agreement with previous studies, we observed an unsigned intensity PE for pain in the anterior insula [12,19,21]. 327 The novel contribution is the fact that stimuli in different modalities (i.e. pain and aversive sounds) [24] lead to the same 328 activations in the anterior insula, with similar magnitudes. To our surprise, strong activation in the anterior insula was 329 also observed for modality PEs (expect pain and receive sound, and vice versa). fMRI signals for unsigned intensity PEs 330 and modality PEs were very similar in magnitude. This disconfirms our hypothesis that at the level of the insula, modality 331 PE carries less difference in salience between the expected and the real outcome, as compared to an unsigned intensity 332 PE. Rather, it seems that surprise from unexpected sensory modalities is as much a source of anterior insula activation as 333 from unexpected intensities. Our findings suggest that modality and unsigned intensity PEs are largely modality-neutral, 334 and support findings that the anterior insula is richly interconnected part of the salience and attentional network 335 involved in decision-marking, error recognition and generally the guidance of flexible behavior [35][36][37][38][39]. Indeed, the 336 large-scale activation following modality PEs and unsigned intensity PEs themselves does not correspond to any single 337 network description, but seems to involve all of the above; possibly, different dynamics are at play over the course of 338 the stimulation, which do not allow for the disentangling of single networks. In fact, recent meta-analytic evidence of 339 resting-state functional connectivity points to the existence of a pain-related network centered on the anterior insula 340 [40]. The activation associated with both pain-related (posterior insula) activation, and that associated with PE-related 341 (anterior insula) activation correspond well with connectivity gradients observed along the posterior-anterior axis [41-342 43]. 343 It is known that SCR predominantly shows arousal and similar effects, but is relatively insensitive concerning valence 344 [25][26][27]44,45]. Here, SCR following unsigned or signed intensity PEs was little different from SCR following no PEs, while 345 SCR following modality PEs was much higher. This might indicate that modality PEs provide a highly salient a teaching 346 signal even in the absence of intensity differences (Supporting Figure 3). 347 A signed representation of an intensity PE for pain is a crucial teaching signal in reinforcement learning, as it is important 348 to dissociate a low threat from a high threat stimulus. Such a representation for pain could plausibly be located in an 349 area adjacent the anterior insula part representing unsigned intensity PEs and modality PEs. Alternatively, this 350 representation could be located closer to representations of pain intensity: Coding of signed intensity PEs within areas 351 coding for stimulus intensity per se was observed using a similar Pavlovian transreinforcer paradigm in the olfactory 352 domain [23]. Indeed, our data show that a signed intensity PE for pain is represented in a part of the dorsal posterior 353 insula [24,28]. Interestingly, we also identified a similar representation of a signed intensity PE for aversive sounds in or 354 adjacent to primary auditory cortices [46,47], namely the middle temporal gyrus and temporal operculum. It also seems 355 indicative of the more general involvement of the insula in pain perception [48] that the signed intensity PE in pain has 356 little to none sound-related activation at all, whereas the signed intensity PE in sound includes some pain intensity-357 related activation. 358 At most, the clear spatial dissociation of intensity PEs for pain and sounds furthermore indicates a specificity of the 359 signal; at least, it stands in marked contrast with the large overlap of activation for unsigned intensity and modality PEs 360 in the anterior insula. Powerful learning models can utilize both a signed PE to update their predictions and an unsigned 361 19 PE to update their learning rate [10,17,18]. Our results provide a neuronal basis for these models as we were able to 362 reveal the simultaneous representation of both a signed and unsigned PE signal in spatially distinct regions of the insula. 363 Due to the task-inherent structure, signed pain intensity PEs can be correlated with actual pain intensity [49]. This 364 collinearity can be remedied by orthogonalizing regressors in the general linear model used for fMRI analysis. However, 365 this arbitrarily assigns the shared variance to either of the two correlated regressors, depending on the order of the 366 serial orthogonalization [50]. Therefore, we refrained from any orthogonalization in our analysis and thus only reveal 367 areas that show unique variance tied to the regressors, including the signed intensity PEs for pain. 368 In conclusion, our data provides clear evidence of anterior insula-centered, modality-independent unsigned PEs, not 369 only concerning mismatched stimulus intensities across modalities, but also across sensory modalities themselves. 370 Equally important, signed intensity PEs were associated with activation in or adjacent to sensory areas highly dedicated 371 to unimodal processing. Neuronal data from both sources are the basis for reinforcement learning and further enhance 372 our understanding of the functional synergies within the insula. Importantly, pathological learning mechanisms [1,9] and 373 abnormalities in anterior insula-related function have been reported in chronic pain [40,51]. Our data therefore offers 374 the possibility that a misrepresentation of PEs constitutes a potential mechanism in pain persistence.  Overview of the experiment 400 The sequence of measurements and timings of the protocol are displayed in Figure 1, while aspect pertaining to CS 401 characteristics as well as contingencies are displayed in Figure 2. The experiment lasted about 2.5 h. The experiment 402 followed a full cross-over design, with every subject participating in all conditions. Subjects learned associations of 403 conditioned stimuli (CS) and unconditioned stimuli (US; painful heat or loud sound). These associations eventually 404 changed in an unforeseeable manner and then had to be relearned. The experiment was run in a single visit, but split 405 into two sessions to reduce subject fatigue and carry-over effects. Prior to the experimental sessions, subjects were 406 calibrated according to their pain and sound sensitivity. At the start and the end of the experiment, subjects filled out 407 psychological questionnaires outside the scanner. Electrodermal activity was measured throughout the experimental 408

sessions. 409
Unconditioned stimuli 410 Heat stimuli were delivered using a CHEPS thermode (Medoc, Ramat-Yishai, Israel) attached to the volar forearm. Basic 411 stimulus parameters included a 32°C baseline temperature and 10°C/s rise and fall rates. Sound stimuli were delivered 412 using MR-compatible headphones (MR confon, Magdeburg, Germany). A pure sound (frequency 1000 Hz, sampling rate 413 22050 Hz) was generated during runtime using MATLAB. 414

415
Prior to the experiment proper, subjects underwent US calibration to determine two intensities at VAS 25 and VAS 75 416 for both modalities (heat and sound). During the experiment, only these four stimuli were used. All stimuli lasted 3s at 417 plateau, except for four 10s long, low-intensity preexposure stimuli used for familiarization and pre-heating of the skin. 418 Heat and sound stimuli were presented and rated in an analogous fashion. Like in a previous study comparing neuronal 419 responses to the two modalities [24], we used the descriptor "painfulness" for heat, while we used the descriptor 420 "unpleasantness" for sound. After calibration, all stimuli were above the respective pain and unpleasantness thresholds 421 and were therefore displayed on simple 0 to 100 visual analogue scales (VAS) for both modalities. 422 21 For heat, anchors were displayed for "minimal pain" (0) and "unbearable pain" (100). Pain was defined as the presence 423 of sensations other than pure heat intensity, such as stinging or burning [54]. 424 For sound, subjects were instructed to rate between anchors labelled "minimally unpleasant" (0) and "extremely 425 unpleasant" (100). Unpleasantness was defined as a bothersome quality of the sound emerging at a certain loudness. 426 During the calibration procedure performed in the running MR scanner, two stimulus intensities each were obtained for 427 the heat and sound modality (low/high pain and low/high noise). Heat stimuli ranged from 43 to 49°C, sound stimuli 428 ranged from 89.1 through 103.0 dBA. Calibration was constrained such that subjects had to reach a certain 429  minimum physical intensity (43°C for heat, 20% system volume for sound, n=1 received 10%) 430  minimum physical difference between the VAS 25 and 75 stimuli (1.5°C for heat, 15% system volume for sound; 431 n=1 received 1°C, n=8 received 10%) 432 If either condition was not met, physical intensities were automatically adjusted to the minimum (e.g., if subject 433 reported VAS 25 for 41°C, temperature was raised to 43°C). Furthermore, to ensure discriminability within stimulus 434 modalities, subjects had the calibrated US played back to them and were explicitly asked three questions, namely that 435 both intensities of the respective modality 436  were painful (for heat) or unpleasant (for sound) 437  were perspectively tolerable throughout repeated trials in two sessions 438  were easily discriminable. 439 If either question was answered in the negative, the calibrated intensities were adjusted, but never below the minimum 440 requirements listed above. 441 Learning protocol 442 Learning the CS-US associations was designed as a Pavlovian transreinforcer reversal learning task [22,23]. Two CS would 443 independently predict one of four US, namely two intensities of painful heat and two intensities of unpleasant sound. 444 Subjects were presented with one of the two CS (Figure 2c and d) and then asked to choose which of the four US they 445 believed to be preceded by it (symbols in Figure 2b). After making their choice, they would actually be exposed to one of 446 the four US (see Figure 1c for trial structure). If they were correct, no further learning was required; if not, they would 447 have the opportunity to learn the correct association for the next occurrence of the CS. They would then rate their pain 448 or unpleasantness on a 0-100 visual analogue scale (VAS), as during US calibration. Both CS signified an independent 449 sequence of associations with the US. Both CS were randomly drawn for each subject from a library of eight fractal 450 pictures (Figure 2a). Which of the two CS was presented in each trial was fully randomized, as were the US for the 451 respective initial associations, and the display order of the US prediction rating. 452 22 Crucially, after a number of trials with deterministic CS-US association, the association underwent an unannounced 453 reversal either in terms of intensity (previously low US intensity would now be high, or vice versa), or modality (previous 454 pain US would now be a sound US, or vice versa) (Figure 2c and d). The number of trials that an association was upheld 455 was randomly determined from [3, 3, 4, 5] (i.e. 3.75 trials on average). After each reversal, subjects therefore made an 456 error in predicting the following US, and subsequently had to learn the new association. As reversals on both dimensions 457 were precluded, each session included eight reversals per CS to cover all possible reversals. Task performance was 458 assessed by the percentage of correct predictions. 459 Psychological questionnaires 460 Prior to and immediately after the experiment, subjects filled out several questionnaires assessing state and trait 461 psychological constructs. These are listed in Supporting Electrodes were connected to Lead108 carbon leads (BIOPAC Systems, Goleta, CA, USA). The signal was amplified with 465 an MP150 analog amplifier (also BIOPAC Systems). It was sampled at 1000 Hz using a CED 1401 analog-digital converter 466 (Cambridge Electronic Design, Cambridge, UK) and downsampled to 100 Hz for analysis. 467 Analysis was performed using the Ledalab toolbox for MATLAB [55]. Single subject data were screened for artifacts 468 which were removed if possible by using built-in artifact correction algorithms. Of 47 subjects, 1 was excluded due to 469 equipment malfunction, 9 due to skin conductance non-responsiveness. From the remaining 37 subjects, a total of 101 470 of 6016 segments (1.7%) were excluded due to unsalvageable artefacts. Using a deconvolution procedure, we computed 471 the driver of phasic skin conductance (skin conductance responses, SCR). Stimulus phase response windows were offset 472 between the two stimulus modalities [24] -we attribute an earlier onset following acoustic stimulation to reduced 473 latency from the delivery system and neuronal transmission. To determine response windows, we obtained the times 474 for average peaks of the respective modality, and selected the data range ±1.25 s: For pain, response windows were set 475 between 2.42 s and 4.92 s, and between 1.15 s and 3.65 s for sound. SCR segments were log-and z-transformed within 476 subjects to reduce the impact of intra-and interindividual outliers [25]. Subsequently, segments were averaged within 477 subjects for several conditions corresponding to the behavioral performance of subjects (e.g. intensity PE following low 478 painful stimulation, or high painful stimulation). SCR was used because it is an objective measure of general sympathetic 479 activity, and therefore a measure of arousal, stimulus salience and several associated psychological processes 480 [25,26,45,56,57]. It is routinely used in assessing painful [12,24,58] as well as acoustic stimulation [59]. 481 23 fMRI acquisition and preprocessing 482 Functional and anatomical imaging was performed using a PRISMA 3T MR Scanner (Siemens, Erlangen, Germany) with a 483 20-channel head coil. An fMRI sequence of 56 transversal slices of 1.5 mm thickness was acquired using T2*-weighted 484 gradient echo-planar imaging (EPI; 2001 ms TR, 30 ms TE, 75° flip angle, 1.5x1.5x1.5 mm voxel size, 1 mm gap, 485 225x225x84 mm field of view, simultaneous multislice imaging with a multiband factor of 2, and an acceleration factor 486 of 2 with generalized autocalibrating partially parallel acquisitions reconstruction). Additionally, a T1-weighted MPRAGE 487 anatomical image was obtained for the entire head (voxel size 1x1x1 mm, 240 slices). 488 For each subject, fMRI volumes were realigned to the mean image in a two-pass procedure, and non-linearly co-489 registered to the anatomical image using the CAT12 toolbox for SPM (Christian Gaser & Robert Dahnke,. In short, this novel non-linear coregistration segments both the mean EPI and the 491 T1 weighted image and performs a nonlinear spatial normalization of the segmented tissue classes from the mean EPI 492 using the segmented tissue classes from the T1 scan as a template. Finally, individual brain surfaces were generated, 493 using CAT12. 494 General statistical approach 495 Unless otherwise noted, analyses except the fMRI analyses were performed using linear mixed models with random 496 intercept using trial-by-trial parameters. In the case of mixed (within/between) descriptive statistics, standard errors 497 were calculated using the Cousineau-Morey approach [60]. The significance level for analyses of behavioral and 498 psychophysiological data was set to p = 0.05. 499 Analysis of imaging data 500 Subject-level analyses were performed on the 3D (volume) data in native space without smoothing, as required for 501 surface mapping. We computed a general linear model with a canonical response function to identify brain structures 502 involved in the processing of each stimulus modality, and corresponding to various predictions and PEs inherent in the 503 protocol. Realignment (motion) parameters were included as nuisance variables, to further mitigate motion-related 504 artifacts. 505 A general linear model was set up with one regressor for stimulus main effects in each modality (heat or sound), and a 506 parametric modulator each for pain or unpleasantness (using behavioral ratings). An additional three parametric 507 modulators for each modality were entered for modality PEs and intensity PEs: Modality PEs were entered unsigned due 508 to their non-parametric nature, whereas intensity PEs were entered both unsigned (absolute) and signed. All parametric 509 modulators were z-scored within subjects and sessions. In either model, global or sequential orthogonalization between 510 regressors were turned off to preserve only the unique (non-shared) variance components [23,50]. This approach allows 511 for the interpretation of consecutively entered parametric modulators even if correlations to previous regressors exist. 512

24
We opted for surface-based analyses of fMRI data to enhance discrimination between modalities processed in adjacent 513 brain regions [24]; for an example of pseudo-overlap detected across the sylvian fissure, see Supporting Figure 5 (row 3), 514 particularly in slices -28 through -16. Results from subject-level analyses were mapped to brain surfaces obtained via the 515 CAT12 segmentation procedure. The mapped subject-level results were then resampled to correspond to cortical 516 surface templates, and smoothed with a 6 mm full width-half maximum 2D kernel. Group-level within-subjects analyses 517 of variance were performed including the mapped contrasts. The original, unmapped contrasts were used for volume-518 based group-level analyses to assess subcortical activation. Volume results were then warped using DARTEL 519 normalization and smoothed with a 6 mm full width-half maximum 3D kernel. Volume-based results are provided in the 520 supporting information and referenced where relevant. 521 Contrasts employed for any of the analyses were either performed against low-level baseline (e.g. Pain>0), as a 522 conjunction of a differential modality contrast and one against low-level baseline (e.g. Pain>Sound ∧ Pain>0), or as a 523 conjunction of both modalities (e.g. Pain ∧ Sound). 524

525
As laid out above and because pain is the modality of interest in this study, we focused the analyses on the contralateral 526 (right) periinsular cortices as regions of interest used for small volume correction of significance level [12,19,24] Table 1. Effects of modality and intensity, by prediction error type. Parameters obtained from linear mixed models with 738 random subject intercept. Differences between the conditions are largest in trials with no prediction error, and smallest in trials with 739 modality prediction error (cf. Supporting Figure 1