Aversive Learning Changes Face-Viewing Strategies — as Revealed by Model-Based Fixation-Pattern Similarity Analysis

To avoid costly situations animals must be able to rapidly predict imminent threat based on past experience and present noisy sensory evidence. We aimed to characterize to what extent active exploration strategies, can be adaptively tuned to achieve this goal. We measured how eye-movement patterns on 8 faces, organized along a circular similarity continuum, were modified after aversive learning and generalization. Using model-based Fixation Similarity Analysis, we characterized how similarity relationships between exploration strategies were modified after volunteers learnt to pair one face (CS+) with a mild electric shock. Initially, viewing patterns reflected the circular physical similarity structure of different faces, indicating that eye-movements were guided by subtle differences between faces. Following aversive learning, the similarity structure of exploration patterns became elliptical and stretched along the adversity gradient defined by the CS+ and the most dissimilar neutral face (CS-), indicating that exploration patterns on these faces were most dissimilar. These findings suggest that the need to predict adversity introduces substantial remodelling of exploration patterns and influences entire sensorimotor loops by selecting fixation locations that can help in categorizing stimuli as aversive or safe.


Introduction
To avoid costly situations animals must be able to rapidly predict future adversity based on information actively harvested with their sensorimotor systems.However, sensory samples are noisy and the environment is complex, therefore newly encountered situations are never exactly the same as previously experienced ones.
Hence, for aversive learning to be effective a careful balance between generalization and selectivity is needed [1,2].While generalization makes it possible to promptly deploy defensive mechanisms when similar adverse situations are encountered anew [3,4], selectivity ensures that only truly aversive stimuli are recognized as aversive [5][6][7].When learning is tested using stimuli that lie on a well-controlled similarity continuum, this antagonism results in smoothly decaying response amplitudes with increasing dissimilarity to the adversity predicting stimulus, thereby producing the canonical generalization profile [2,8,9].In real-world situations however adversity predictions must be based on sensory samples collected through active exploration of information embedded in complex environments [10,11].In humans, a central part of active exploration involves eye-movements [11][12][13][14][15] which can rapidly determine what information is available in a scene for recognizing adversity [16].However, little is known about how aversive learning and its generalization influence the active exploration strategies that are fundamental to collect the necessary evidence for adversity predictions.
Here we used eye-movements as a model sensorimotor loop and compared exploration strategies before and after an aversive learning during viewing of faces.Using faces as stimuli offers an ideal test bed for investigating changes in active exploration strategies through learning.On the one hand, active viewing of faces is a key ability during daily social interactions [17,18], where detecting of minute differences in the configuration of facial elements is crucial for inferring the identity or emotional content of a face [19,20].On the other hand, the universal spatial configuration of facial elements makes it easily possible to generate stimuli that are calibrated to form a well-controlled perceptual similarity continuum required for testing aversive generalization [7,21].Therefore these key features make it possible to use an ecologically relevant task, while precisely controlling stimulus properties to generate a perceptual continuum to probe exploration strategies during aversive generalization.During aversive learning one randomly chosen face along this continuum (CS+) was paired with a mild electric shock (UCS) through a simple Pavlovian procedure, therefore introducing an adversity gradient based on physical similarity.This allowed us to investigate how exploration strategies deployed to collect sensory information were modified during aversive generalization.
Exploration strategies of faces are typically investigated by counting the number of fixations at predefined regions of interest [22,23].Modifications in exploration strategies can therefore be characterized by relative changes in the number of fixations within these regions.Therefore this approach relies upon consistent changes in fixation counts across participants within the regions of interests.However, aversive learning might also influence fixation selection not by targeting different regions of the face, but by optimizing saccadic endpoints in order to provide relevant information more precisely to better predict adversity.This could lead to small shifts in exploration patterns that do not result in differences for fixation counts within predefined regions of interest.We therefore complemented count-based analyses with a variant of representational similarity analysis [24] that we term "fixation-pattern similarity analysis" (FPSA) which considers exploration patterns as multivariate entities [25,26].FPSA assesses between-condition (dis-)similarity of eye-movement patterns for individual participants, therefore eliminating the requirement for arbitrary regions of interests (Fig. 1A).One peer-reviewed) is the author/funder.All rights reserved.No reuse allowed without permission.
The copyright holder for this preprint (which was not .http://dx.doi.org/10.1101/125682doi: bioRxiv preprint first posted online Apr. 15, 2017; advantage of this approach is that it is sensitive to fine-grained changes in exploration patterns independent of whether these are accompanied with changes in fixation counts.Furthermore, FPSA has the added benefit that it can cope with idiosyncratic facial exploration strategies [25][26][27] as long as the similarity structure of exploration patterns changes consistently across observers. Model-based FPSA allowed us to formulate parametric hypotheses about how aversive learning might induce changes in the similarity relationship between exploration patterns when one face along the circular continuum started to predict adversity (Fig. 1B-E, top and bottom panels).The circular similarity continuum allowed us to further decompose hypotheses about the similarity relationships onto specific and unspecific orthogonal components [28], which were centered on either the CS+/CS-or +90°/-90° faces, respectively (Fig. 1B-E, middle panels).We formulated four mutually non-exclusive hypotheses about possible changes in fixation patterns with key diagnostic values for different exploration strategies.First, if fixation selection mechanism during viewing of faces operates based on salient low-level features [10,29], we would expect exploration patterns to track the circular similarity relationships between faces (Fig. 1B).Therefore, the bottom-up saliency hypothesis predicts that a circular relationship between exploration patterns is already present before aversive learning has taken place.This would be characterized by low dissimilarity between neighboring faces (1 st offdiagonal) and high dissimilarity between the opposing faces separated by 180° (4 th off-diagonal, Fig. 1B   face stimuli (numbered 1 to 8) calibrated to span a circular similarity continuum across two dimensions (gender and identity; see also SFig. 1 for stimuli).A pair of most dissimilar faces was randomly selected as CS+ (red border) and CS-(cyan border; see color wheel for color code).The similarity relationships among the 8 faces and the resulting exploration patterns are depicted as two 8 by 8 matrices.Physical similarity (top right panel) between all pair-wise combination of faces were calibrated (see methods and SFig.2A) to have a perfect circular similarity, characterized by highest similarity (blue) between neighbors, and lowest similarity (yellow) for opposing pairs (see also SFig. 2 for calibration).Fixation Similarity Analysis summarizes the similarity relationship between the 8 exploration patterns as a symmetric 8x8 matrix (bottom right panel). 4 th and 8 th columns (and rows) are aligned with the CS+ and CS-, respectively.(B-E) Multidimensional scaling representation of four theoretical similarity relationships between exploration maps through learning.Each colored node represents one exploration pattern (same color scheme; red: CS+; cyan: CS-), where internode distances are proportional to dissimilarity between exploration patterns, depicted in bottom panels as 8 by 8 matrices.These matrices are further decomposed onto two orthogonal components (middle row), centered either on the CS+/CS-(specific component) or +90°/-90° faces (unspecific component).In (B), equal contribution of components results in a circularly similar exploration patterns.In (C), a stronger contribution results in a better global separation of all exploration patterns (denoted by radial arrows second column; shaded nodes depict the first hypothesis).In (D), stronger contribution of the specific component results in a biased separation of exploration patterns specifically along the adversity gradient defined between the CS+ and CS-nodes.In (E), a Gaussian covariance component centered on the CS+ face can specifically increase the similarity of exploration patterns for faces similar to the CS+, resulting in circularly shifted nodes (circular arrows) while preserving the global circularity of the similarity relationships.(same colormap as in A).
Aversive learning might lead to heightened arousal and/or an increased contribution of low-level image features on eye-movements with the objective to collect increased sensory evidence.This would result exploration strategies to mirror more closely the similarity relationships between faces (Fig. 1C) and lead to a globally increased dissimilarity between all exploration patterns.Increased arousal hypothesis therefore predicts a stronger but equal contribution of the underlying specific and unspecific components (Fig. 1C, middle panel), leading to a better separation of all exploration patterns globally.Alternatively, eye-movements may reflect a categorization process for faces as threatening or safe taking place during aversive generalization [30,31].To achieve this, exploration strategies can be tailored to collect relevant information that jointly predicts adversity and safety.Such a fixation strategy would preferentially target locations that are discriminative of the CS+ and CS-faces.This would lead exploration patterns to become more similar for faces sharing similar features with the CS+ and CS-faces, while simultaneously predicting an increased dissimilarity between these two sets of exploration patterns (Fig. 1D, middle panel).Therefore, the 'adversity categorization' hypothesis would lead an increase of dissimilarity between exploration patterns distributed specifically along the task-relevant adversity gradient without influencing exploration patterns for intermediate faces.As a fourth possible scenario, aversive learning might result in the deployment of a new sensorimotor strategy for the adversity predicting face, thereby leading to a localized remodelling of the similarity relationships specifically around the adversity predicting CS+ face (Fig. 1E).This is supported by univariate behavioral readouts such as autonomic skin-conductance responses [21], verbal ratings of subjective adversity [32] and startle responses [33] that shows the canonical generalization profiles consisting of gradually decaying responses with increasing dissimilarity to the CS+ stimulus.Therefore, based on univariate generalization profiles, the adversity-tuning hypothesis predicts an increased similarity of exploration patterns around the CS+ face, that would decay proportionally with the increasing dissimilarity to the CS+ face.
In sum, using FPSA we analyzed the similarity relationships between exploration patterns during viewing of faces.FPSA provided us insights on how exploration strategies changed following an aversive peer-reviewed) is the author/funder.All rights reserved.No reuse allowed without permission.
The copyright holder for this preprint (which was not .http://dx.doi.org/10.1101/125682doi: bioRxiv preprint first posted online Apr. 15, 2017; learning and its generalization.First, aversive learning changed exploration patterns in subtle ways that were not captured by fixation counts.Second, before learning exploration patterns showed an approximately circular similarity structure that followed the physical stimulus similarity structure.Third, after learning the similarity structure changed specifically along the adversity gradient, indicating that CS+ and CS-exploration patterns were modified, while the similarity between other faces remained largely unchanged. peer-reviewed) is the author/funder.All rights reserved.No reuse allowed without permission.
The copyright holder for this preprint (which was not

Results
We created 8 face stimuli that were characterized by subtle differences in facial elements that altogether spanned a circular similarity continuum defined by two dimensions (gender and identity; see SFig. 1 for stimuli).
We carefully calibrated the degree of similarity between all pairwise combinations of these faces using a biologically plausible model of the primary visual cortex [34] tuned to human contrast sensitivity (see SFig.We confronted participants (n = 61) with this stimulus continuum before and after an aversive associative learning procedure (Fig. 2A) while measuring eye-movements during viewing of faces.During the conditioning phase, only the CS+ and CS-faces were presented and the CS+ face was partially reinforced with an aversive outcome (UCS, mild electric shock in ~30% of trials).The CS-corresponding to the face most dissimilar to the CS+ (separated by 180°) and was not reinforced.During the subsequent generalization phase, all faces were presented and the CS+ continued to be partially reinforced to prevent extinction of the previously learnt association.To ensure comparable arousal states between baseline and generalization phases, we administered UCSs also during the baseline period, however they were fully predictable as their occurrence was indicated by a shock symbol (Fig 2A).
We used autonomic skin-conductance responses (during each phase; Fig. 2B) and subjective ratings of UCS expectancy (at the end of each phase; Fig. 2C) as univariate behavioral readouts to monitor aversive learning.As expected, aversive manipulation had a profound effect on these measurements.Skin-conductance responses (SCR) recorded during the conditioning phase were on average 2.8 times higher for the CS+ face than CS-(Figure 2B, middle panel, paired t-test, p < .001).Explicit UCS expectancy ratings gathered at the end of the conditioning phase were also highest for the CS+ face (Figure 2C, middle panel).This showed that the CS+ face gained an aversive quality during the conditioning phase as shown by both verbal as well as autonomic recording modalities.In the subsequent generalization phase, amplitudes in both measurements decayed with increasing dissimilarity to CS+ face (Figure 2B-C, right panel) leading to a adversity-tuned profile which was well captured by a circular Gaussian curve (centered on the CS+ face, comparison to flat null model, p < .001,log-likelihood ratio test) in both modalities.We ruled out that aversive associations were already present before learning (comparison of flat null model and Gaussian model p = .54,log-likelihood ratio test; black horizontal lines in Fig. 2B-C).In summary, these univariate adversity selective measurements confirmed that learning was successfully established and transferred towards other stimuli which were perceptually similar, providing evidence for aversive generalization following learning.Whether eye-movements during viewing of complex stimuli such as faces also exhibit learning induced changes, and if so whether these generalize around the adversity predicting face is an open question.We first investigated this question using a fixation count-based approach.To this end, we computed fixation density maps (FDMs) for every volunteers and face separately (Fig. 3A for FDMs from two volunteers) and evaluated fixation probability within the 4 different regions of interest (left and right eyes of the face, nose and mouth [22,23]; ROIs shown in Fig. 3B as insets).We reasoned that if aversive learning has a specific influence on exploration of faces, this would result in a bell-shaped modulation of fixation counts around the CS+ face similar to SCR and verbal ratings.In line with previous report [27], left and right eyes together with the nose region were the most salient locations across the baseline and generalization phases, and attracted ~84% of all fixation density, whereas the mouth region had only a marginal contribution with ~3% of directed density.Aversive learning increased the number of fixations directed at the nose (+4%) and mouth (+0.6%) regions at the expense of left (-3.5%) and right (-2.8%) eyes (Fig. 3B).We modeled the difference of percentages (∆%) through learning with a categorical factor that coded for ROI identity (ROI), a second factor that indicated the absolute angular distance to the CS+ face (AngularDistance) and a third factor consisting of their interaction (∆% ~ ROI + AngularDistance + ROI*AngularDistance).While fixation densities were significantly modulated by ROI (F = 20.18;p < .001),neither angular distances to the CS+ face (F = 0.3; p = .55)nor their interaction (F = 1.889; p = 0.12) did have any significant contribution.We complemented this linear approach with a nonlinear model, and tested how well a circular Gaussian can explain fixation counts along the similarity continuum separately for each ROIs.Model comparison on percentage changes (Fig 3B, black lines and curves) favoured the flat null model for all regions (p > 0.05, log-likelihood test), with the exception of the mouth region.Here the Gaussian model was favoured, however marginally over the null model (p = .012,log-likelihood ratio test).Therefore using the fixation count-based approach, we were able to show a specific adversity-related effect at the mouth region with increasing similarity to adversity predicting stimulus, which however exhibited only a slight increase in fixation density.However at locations that accounted for most of the fixation density were accompanied with unspecific changes that were independent of the adversity gradient.Despite the lack of a adversity specific effect at the most salient locations, a careful examination of single-subject FDMs revealed fine-grained patterning on exploration patterns that lawfully changed along the dissimilarity continuum (Fig. 3A).Notably, these differences existed within the ROIs that fixation count analysis didn't report any difference.This suggests that multivariate analysis methods [26] might therefore be more appropriate for investigating the impact of aversive learning on fixation patterns during viewing of faces.We thus investigated multivariate information content within FDMs and tested whether eye-movements deployed for the exploration of the CS+ face could be differentiated from the CS-face beyond what could be already accounted by physical differences.We evaluated how accurately a cross-validated linear classifier could discriminate FDMs on these faces before and after learning, expecting decoding accuracy to increase if aversive learning led to a differentiation of exploration patterns for CS+ vs. CS-(using a 50% holdout cross-validation with 1000 random splits of FDMs into test and training sets).The average classification performance of CS+ (59.8 ± 1.8 %; mean ± SEM across subjects) was significantly better than classification accuracy obtained before learning (52.7 ± 1.2 % classification performance; paired t-test, p < .001) as well as chance-level decoding (based on label permutation: 50.0 ± 0.1 %).Furthermore, classification accuracy decayed according to the typical bell-shaped curve with decreasing similarity to the CS+ face (see SFig. 3 for change in accuracy with increasing dissimilarity to CS+), suggesting a gradual deployment of the adversity specific exploration strategy with increasing similarity to the adversity predicting face.In order to understand whether this increased decoding peer-reviewed) is the author/funder.All rights reserved.No reuse allowed without permission.
performance was driven from the mouth region, which previous fixation count-based analysis detected adversitytuned density profile at the group-level, we repeated the same analysis but this time excluding the data from the mouth region.This yielded very similar results both in terms of classification accuracy and smooth decay with increasing dissimilarity, therefore excluding the possibility that decoding performance was solely driven by the adversity tuned density in the mouth region found at the group-level.Altogether these results show that exploration patterns during viewing of CS+ and CS-faces were associated with detectable differences emerging specifically with learning that could not be explained by physical differences between stimuli, or adversitytuning present in the fixation counts.This corroborates the notion that aversive learning was associated with new exploration strategies providing strong evidence on observable modification of behavior, complementing thus SCR and verbal reports.
peer-reviewed) is the author/funder.All rights reserved.No reuse allowed without permission.
The copyright holder for this preprint (which was not .http://dx.doi.org/10.1101/125682doi: bioRxiv preprint first posted online Apr. 15, 2017; However, these results cannot disentangle different hypotheses about different exploration strategies, as this requires precise characterization of all pairwise similarity relationships between exploration patterns.To gain further insights, we therefore applied a variant of representational similarity analysis [24], which we termed Fixation Similarity Analysis (FPSA).FPSA allowed us to parametrically test different hypotheses on how aversive learning could change the pairwise similarity relationships between exploration patterns, rather than the exploration patterns per se.We computed a dissimilarity matrix consisting of all pairwise comparisons of FDMs for individual volunteers (using 1 -Pearson correlation) and averaged these after separately aligning them to each volunteer's CS+ face (shown always at the 4 th column and row Fig 4A).Furthermore, in order to gather an intuitive understanding learning-induced changes in the similarity geometry we used multidimensional scaling (jointly computed on the 16x16 matrices).Multidimensional scaling intuitively summarizes complicated similarity matrices by transforming observed dissimilarities as closely as possible onto distances between different nodes, therefore making it easily understandable at a descriptive level.
Already during the baseline period the dissimilarity matrix was highly structured (Fig. 4A).In agreement with a circular similarity geometry and the MDS depiction (Fig. 4B) lowest dissimilarity values (1.04 ± .01;M ± SEM) were found between FDMs of neighboring faces (i.e.first off-diagonal), whereas FDMs for faces separated by 180° exhibited significantly higher dissimilarity (1.21 ± .01 (paired t-test, t(60) = 7.03, p < .001).We investigated to what extent the observed dissimilarity structure before learning could be accounted by physical aspects of the stimulus set using the bottom-up saliency model (shown in Figure 1B).Bottom-up saliency model uses a theoretically circular similarity matrix (consisting of equally weighted sums of specific and unspecific components) as a linear predictor, therefore it estimates the global dissimilarity between exploration patterns.The circular bottom-up saliency model performed significantly better compared to a null model consisting of a constant similarity for all pairwise FDMs comparisons (for bottom-up model adjusted r 2 = .09;log-likelihood-ratio test for the alternative null model: p < 10 -5 ; BIC NullModel = -1529.3,BIC BottomUp = -1650; see STable 1 for the results of model fitting).We additionally fitted the bottom-up model for every volunteer separately.Model parameters at the aggregate level was significantly different than zero (Fig. 4C; w Circle = .063± 0.008, M ± SEM; t = 7.89, p < 10 -5 ) indicating that prior to learning exploration strategies mirrored the similarity structure physically present in the stimulus set.This provides evidence that fixation selection strategies are, at least to some extent, guided by physical stimulus properties.
Following aversive learning, (Figure 4A, right panel) we observed significant changes when comparing baseline and generalization dissimilarity values element-by-element (Figure 4A, indicated by asterisks) providing evidence for learning-induced changes in the similarity relationships.The same bottom-up saliency model was again significant (adjusted r 2 = .33;p < 10 -5 , log-likelihood ratio test), albeit in comparison to baseline phase performed notably better (BIC BottomUp = -1650 for the baseline vs. BIC BottomUp = -2715.5 for the generalization phase; see STable 2 for model fitting results).Critically, we found a significant increase in the model parameter from baseline to generalization phase (w Circle = 0.092 ± 0.01; t = 9.13, p < 10 -5 ; Fig. 4C compare two leftmost bars) suggesting that dissimilarity between FDMs globally increased.Supporting this result, we found a significant interaction between the circular model and a factor that categorically differentiated pre-and post-learning periods (p = 0.009 for the interaction term).Overall, these results are compatible with the view that aversive learning led to a better separation of exploration patterns globally, in agreement with the heightened arousal model (Figure 1C), which predicted an increased contribution of the bottom-up saliency to the similarity of exploration patterns as shown by larger model parameters.
However, MDS method suggested that the pattern separation might have occurred mainly along the adversity gradient defined by the CS+ and CS-faces, whereas the separation along the orthogonal direction did not exhibit any noticeable changes (Fig. 4B).We thus extended the circular bottom-up model to capture independent variance along the two different directions using the adversity categorization model (Fig. 1D).
Model comparison indicated that this model performed better (BIC BottomUp = -2715 vs. BIC AdversityCateg.= -2897.3during the generalization phase; adjusted r 2 = .44;see STable 3 and 4 for fitting results with the adversity categorization model on baseline and generalization phases, respectively).Notably this difference was accompanied by a nearly two times stronger contribution of the specific component (w Specific = 0.12 ± 0.014, t = 21.034;w Unspecific = 0.063 ± 0.01, t = 11.07; Figure 4C).Furthermore this difference was highly significant (p = 3.2 x 10 -4 , t = -3.81,pair-wise t-test) indicating that the similarity relationship across the 8 faces were mainly modified along the adversity gradient.This provides strong evidence that aversive learning introduced changes in the scanning behavior specifically along the task-relevant adversity direction.
The remodelling of the similarity geometry along the adversity gradient can also be accompanied with exploration strategies that are specifically deployed for the adversity predicting face, resulting in localized changes in the similarity geometry around the CS+ face.We tested this hypothesis by augmenting the previous model with a two-dimensional Gaussian peaking on the CS+ and slowly decaying with increasing dissimilarity (Figure 1E).The model comparison procedure favored the simpler adversity categorization model over the augmented adversity tuning model (BIC AdversityCateg.= -2897.3vs. BIC AdversityTuning.= -2864.4during the generalization phase; adjusted r 2 = .44;see STable 5, 6 for fitting results with adversity tuning model in baseline and generalization phases, respectively).Hence the increase in the number of predictor variables did not result in a significant reduction in explained variance.In line with this result, the parameter estimates for the twodimensional Gaussian predictor were not significantly different than zero neither in baseline or generalization phases (w Gaussian = 0.015 ± 0.04 in baseline, p = .72,t = 0.35; w Gaussian = 0.07 ± 0.04 in generalization, p = 0.12, t = 1.56; Figure 4C).Furthermore, pair-wise differences between parameter estimates did not reach significance neither (p = 0.37, t = 0.89).We therefore conclude that further improvements of the adversity categorization model to include adversity-specific changes did not result in a better understanding of the adversity-induced changes in the similarity geometry of exploration strategies.

Discussion
We investigated how flexible and adaptive are active exploration strategies during generalization of an aversive learning with faces.Univariate readouts from autonomous recordings, as well as subjective verbal reports exhibited as expected the canonical generalization profiles, characterized by smoothly decaying responses with increasing dissimilarity to the CS+ face.Multivariate analyses of exploration patterns indicated that these observations were accompanied by decodable changes in the behavior, in line with an emergence of new exploration strategies with aversive learning.Furthermore, these gradually increased with the similarity to the CS+ face, resulting in adversity-tuned multivariate exploration patterns.Notably, fixation count-based approach did not detect specific effects of aversive learning on basic facial regions that were most often fixated.
Using FPSA, we showed that aversive learning remodeled exploration patterns by increasing the dissimilarity specifically along the adversity gradient, while jointly merging scanning strategies for faces that were most representative for safety and threat.This was shown by the unequal contribution of orthogonal similarity components in the adversity categorization model, with the specific component having a stronger weight.
Extending this model to further take into account changes in similarity relationships around the CS+ face, as predicted by the univariate bell-shaped generalization profiles, did not lead to a significant improvement.These results provide evidence for an internal cognitive process that is involved in the adversity-based categorization during aversive generalization, and argues against for the emergence of an adversity-specific exploration strategy, as predicted by univariate generalization profiles.
Humans move their eyes typically around three times per second.Hence, in comparison to autonomic responses-which inform about aggregate cognitive evaluations-and verbal evaluations such as ratings-which can interfere with the on-going task-eye-movements provide a large bandwidth of information that can be used to unveil rapidly unfolding cognitive processes [35].However, in stark contrast with the universal organization of facial elements on a face, exploration patterns during viewing of faces are typically highly variable across individuals [25,26,36], therefore complicating their straightforward use.FPSA, in the same spirit as representational similarity analysis in fMRI [37,38], MEG [39] or EEG [40] investigates between-condition similarity of eye-movement patterns within the specific action repertoire of an individual, where averaging across volunteers would have a destructive effect.FPSA has been previously used for controlling eyemovement-related artefacts during neuronal recordings [40].However, in this report FPSA played a central role to overcome methodological difficulties related to strong inter-individual differences.Furthermore, the key contribution of the FPSA was providing insights on how active exploration strategies were remodeled with aversive learning that could not have been predicted based on univariate generalization profiles.
Significant contribution of the bottom-up model prior to aversive learning suggests that exploration patterns reflected the physical similarity of the stimulus set.This is interesting, as it shows that fixation selection process closely tracks subtle differences in low-level local features during viewing of faces and indicates that landing points are selective for facial elements.In contrast, a holistic viewing strategy would have predicted a lack of dependence between similarity of physical features and exploration patterns.Viewing of faces can therefore be better understood by a local mechanism for fixation selection, rather than a completely holistic strategy.However, following aversive learning the bottom-up strategy was insufficient to explain observed changes in the similarity geometry.Rather, results observed here could be reconciled with a model that biases fixation selection process with respect to facial features associated with both adversity and safety.This would effectively result in sensory evidence to be harvested from the most informative locations from stimuli along the adversity gradient.Thereby, increasing the information efficiency downstream and help the categorization process of faces as either safe or dangerous.
The effect of aversive learning on overt attention is well known.For example, fearful faces are more salient and attract more fixations than neutral ones [41][42][43].Elementary visual features can benefit higher priority when they predict adversity, to the point of distracting an on-going task [16].Most importantly, their strength gradually increases with similarity to the adversity predicting features [16].These oculomotor saliency gradients indicate that stimuli predicting adversity benefit higher priority in the sensorimotor processing.Based on these results, one could argue that facial elements that bear resemblance to the adversity predicting face could gain higher priority and therefore would be fixated more often in a proportional manner with their similarity to the shocked face.This would normally lead to the generalization of fixation counts across facial elements that are most often fixated.However, this prediction was only partially validated at the mouth region, which was the single region where we observed an adversity-tuned fixation probability.It is therefore possible that saliency gradients observed with elementary features do not straightforwardly generalize to more complex situations involving for example faces simultaneously defined with multiple features.
The key contribution of the FPSA method was to exploit information present in pair-wise relationships between exploration patterns, and thereby achieve selectivity with respect to different hypotheses outlined here.
Notably, these hypotheses could not be differentiated with univariate generalization profiles.In neuronal recordings, gradients of activity during generalization has been successfully used to characterize selectivity of aversive representations for example in different groups of subjects [44,45] or in different brain regions [7,46].However, univariate gradients exploit only a limited extent of the available information about representations along the adversity gradient.Therefore, it will be highly informative to test these hypotheses using neuronal recordings with representational similarity analysis during the emergence of aversive representations.It is now interesting to speculate that different hypotheses we outlined here can map onto different brain systems, which are then integrated for the production of overt behavior.Therefore combined fMRI and eye-movement recordings can shed light on how the central nervous system can reorganize at the systems level following an aversive learning.
peer-reviewed) is the author/funder.All rights reserved.No reuse allowed without permission.
introduced independent variance on the faces.
To present these stimuli we resized them to 1000x1000 pixels (originals: 400x400) using bilinear interpolation, and slightly smoothed with a Gaussian kernel of 5 pixels with full-width at half maximum of 1.4 pixels to remove any possible pixel artifacts that could potentially lead participants to identify faces.Faces were then normalized to have equal luminance and root-mean-square contrast.The gray background was set to the same luminance level ensuring equal brightness throughout of the experiment.Stimuli are available in [48].
Faces were presented on a 20" monitor (1600 x 1200 pixels, 60 Hz) using Matlab R2013a (Mathworks, Natick MA) with psychophysics toolbox [49,50].The distance of the participants' eyes to the stimulus presentation screen was 50 cm.The center of the screen was at the same level as the participants' eyes.Faces spanned horizontally ~17° and vertically ~30°, aiming to mimic a typical face-to-face social situation.

Experimental paradigm
The fear conditioning paradigm (similar to [7]) consisted of baseline, conditioning and test (or generalization) phases (Fig. 2C-D).Four equivalent runs with exactly same number of trials were used during baseline (1 run) and generalization phases (3 runs) consisting of 120 trials per run (~10 minutes).Every run started with an eye-tracker calibration.Between the runs participants took a break and continued with the next run in a self-paced manner.We avoided having more than 1 runs in the baseline period in order not to induce fatigue in participants.In all three phases, subjects were instructed to fixate fixation crosses thoroughly and to press a button when an oddball stimulus appeared on the screen.This consisted of a blurred unrecognizable face.
At each run during the baseline and generalization phases, 8 faces were repeated 11 times, UCS trials occurred 5 times and one oddball was presented.We presented 26 null trials with no face presented but otherwise the same trial structure (see below sequence optimization).In order to keep arousal levels comparable to the generalization phase, UCSs were also delivered during baseline, however they were fully predictable by a shock symbol therefore avoiding any face to UCS associations.During the conditioning phase, participants saw only the CS+ and the CS-faces among the 8 faces (and the null trials).These consisted of 2 most dissimilar faces separated by 180° on the circular similarity continuum and randomly assigned for every participant in a balanced manner.The conditioning was 124 trials long (~10 minutes) and CS+ and CS-faces were repeated 25 times.CS+ faces were additionally presented 11 times with the UCSs, resulting in a reinforcement rate of ~30 %.The same reinforcement ratio was used during the subsequent generalization phase in order to avoid extinction of the learnt associations.Participants were instructed that the delivery of UCSs during baseline would not be associated with faces, however in the following conditioning and test phases they were instructed that shocks would be delivered after particular faces have been presented.Faces were presented using a rapid-event design with a stimulus onset asynchrony of 6 seconds and stimulus duration of 1.5 seconds.The presentation sequence was optimized using a modified m-sequence with 11 different conditions [51,52] (8 faces, UCS, oddball, null).An m-sequence is preferred as it balances all transitions from condition n to m (thus making the sequence as unpredictable as possible for the participant) while providing an optimal design efficiency (thus making deconvolution of autonomic skin conductance responses more reliable).In order to achieve the required reinforcement ratio (~30%), we randomly pruned UCS trials and transformed them to null trials.Similarly oddball trials were removed to have an overall rate of ~1%.This resulted in a total of 26 null trials.This deteriorated the efficiency of the m-sequence, however was a good compromise as the resulting sequence was still much more efficient than a random sequence.Resulting from the intermittent null trials, SAOs were 6 or 12 seconds approximately exponentially distributed.
Face onsets were preceded by a fixation-cross which appeared randomly outside of the face either on the left or right side along an imaginary circle (r = 19.6°,+/-15° above and below the horizontal center of the image).The side of fixation-cross was balanced across conditions.The calibrated electric shock was delivered as unconditioned stimulus (UCS) before the offset of the CS+ face.

Calibration and Delivery of Electric stimulation
Mild electric shocks were delivered by a direct current stimulator (Digitimer Constant Current Stimulator, Hertfordshire UK), applied by a concentric electrode (WASP type, Speciality Developments, Kent UK) that was firmly connected to the back of the right hand and fixated by a rubber glove to ensure constant contact with the skin.Shocks were trains of 5-ms pulses at 66Hz, with a total duration of 100 ms.The intensity of the electric shock applied during the experiment was calibrated for each participant before the start of the experiment.Participants underwent a QUEST procedure [53] presenting UCSs with varying amplitudes selected by an adaptive algorithm and were required to report whether a given trial was "painful" or "not painful" in a binary fashion using a sliding bar.The subjective pain threshold was the intensity that participants would rate as "painful" with a probability of 50%.The QUEST procedure was repeated twice to account for sensitization/habituation effects, thus obtaining a reliable estimate.Each session consisted of 12 stimuli, starting at an amplitude of 1mA.The amplitude used during the experiment was 2 times this threshold value.Before starting the actual experiment, participants were asked to confirm whether the resulting intensity was bearable.If not the amplitude was incrementally reduced and the final amplitude was used for the rest of the experiment.
UCSs were administered before the offset of the stimulus.

Eye Tracking and Fixation Density Maps (FDM)
Eye tracking was done using an Eyelink 1000 Desktop Mount system (SR Research, Ontario Canada) recording the right eye at 1000 Hz.Participants placed their head on a headrest supported under the chin and forehead to keep a stable position.Participants underwent a 13 point calibration / validation procedure at the beginning of each run (1 Baseline run, 1 Conditioning run and 3 runs of Test).The average mean-calibration error across all runs was Mean = 0.36°, Median = .34°,SD = 0.11.91% of all runs had a calibration better than or equal to .5°.Fixation events were identified using commonly used parameter definitions [54] (cognitive configuration: saccade velocity threshold = 30° / second, saccade acceleration threshold = 8000° per second 2 , motion threshold = .1°).Fixation density maps were computed by spatially smoothing (Gaussian kernel of 1° of FWHM) a 2D histogram of fixation locations, and were transformed to probability densities by normalizing to unit sum.FDMs included the center 500x500 pixels, including all facial elements where fixations were mostly concentrated (~95% of all fixations).

Shock Expectancy Ratings and Autonomic Recordings
After baseline, conditioning and generalization phases, participants rated different faces for subjective shock expectancy by answering the following question, "How likely is it to receive a shock for this face?".Faces

Classification with Linear Support Vector Machine
We used single trial FDMs for validation of linear support vector machines that were trained to classify exploration patterns obtained during viewing of CS+ and CS-conditions.As the generalization phase had more trials than the baseline phase (3 runs vs. 1 run), we took precautions to make a fair comparison between these phases so that differences between number of trials do not invalidate comparison of accuracies between the baseline and test phases.To this end, we trained and tested a linear SVM classifiers [57] always within a given run, and averaged classification accuracy for the generalization phase across the three runs.We trained a classifier on randomly drawn 50% of the CS + and CS-trials, and tested on the remaining 50% and averaged classification performance across 1000 repetitions using this procedure.To reduce the dimensionality, FDMs were first downscaled 10 times resulting in a vector of 2500 pixels.We further reduced dimensionality by projecting FDMs onto their principal components using two different approaches.We identified the number of N principal components corresponding to the elbow where the slope of eigenvalues was levelling off substantially (N = 14, 19, 17, 20 for the 4 runs).We also repeated the analysis simply using N principal components that explained 90% of total variance in each run (N = 63, 69, 74, 75).Both approaches yielded very similar classification results, therefore we report numbers from the first approach in the results section, but present results obtained with both approaches (SFig.3).Principal components were computed excluding Null trials, UCS trials and oddballs.Loadings on these principal components were scaled using the inverse of the square root of eigenvalues, thus effectively whitening their contributions to the training.

Fixation Similarity Analysis
Fixation similarity analysis was conducted on single participants.Condition specific FDMs (8 faces per baseline and generalization phases) were computed by collecting all fixations across trials on a single map which were then normalized to unit sum.We corrected FDMs by removing the common mean pattern (done separately for baseline and generalization phases).We used 1 -Pearson correlation as the similarity metric.This resulted in a 16x16 similarity matrix per subject.Statistical tests were done after Fisher transformation of correlation values.
The multidimensional scaling was conducted on the baseline and generalization phases jointly using the 16x16 similarity matrix as input (mdscale in MATLAB).The node coordinates for the baseline condition were linearly transformed to map onto a perfect circle as close as possible using Procrustes mapping, and the same transformation was applied on coordinates of the generalization phase.This has no consequence on MDS results, as proportions between nodes stay unmodified under linear transformations.Importantly, as the similarity metric is extremely sensitive to signal to noise ratio [28] present in the FDMs, we took care that the number of trials between test and baseline phases were exactly the same in order to avoid differences that would have been caused by different signal to noise ratios.To account for unequal number of trials during the baseline ( 11repetitions) and test (3 runs x 11 = 33 repetitions) phases, we computed a similarity matrix for each run separately in the generalization phase.These were later averaged across runs for a given participant.This ensured that FDMs of the baseline and generalization phases had similar signal to noise ratios, therefore not favoring the generalization phase for having more trials.
We generated 3 different models based on a quadrature decomposition of a circular similarity matrix.A circular similarity matrix of 8x8 can be obtained using the term M⊗M, where M is a 8x2 matrix in form of peer-reviewed) is the author/funder.All rights reserved.No reuse allowed without permission.
The copyright holder for this preprint (which was not .http://dx.doi.org/10.1101/125682doi: bioRxiv preprint first posted online Apr. 15, 2017; [cos(x) sin(x)], and the operator ⊗ denotes the outer product.x represents angular distances from the CS+ face, is equal to 0 for CS+ and π for CS-.Therefore, while cos(x) is symmetric around the CS+ face, sin(x) is shifted by 90°.For the bottom-up saliency and increased arousal models (Fig. 1B and C) we used M⊗M as a predictor together with a constant intercept.For the tuned exploration model depicted in Figure 1D, we used cos(x) ⊗cos(x) and sin(x)⊗sin(x) to independently model ellipsoid expansion.Together with the intercept this model comprised 3 predictors.Finally the aversive generalization model (Fig. 1E) was created using the predictors of the tuned exploration model in conjunction with a two-dimensional Gaussian centered on the CS+ face (in total 4 predictors).We tested different widths for the Gaussian and took the one that resulted in the best fit.This was equal to 65° of FWHM and similar to the values we observed for explicit ratings and SCR responses.
All linear modeling was conducted using non-redundant and vectorized forms of the symmetric similarity matrices.For a 8x8 similarity matrix this resulted in a vector of 28 entries.To compare different similarity models, we used mixed-effects models where intercept and slope contributed both as fixed-and random-effects (fitlme in Matlab).We selected mixed-effect models as these performed much better than models defined uniquely with fixed-effects on intercept and slope.To do model selection we used Bayesian information criteria (BIC) as it compensates for an increase in the number of predictors between different models.For statistical tests on the parameter estimates, these 3 models were fit for every subject individually (fitlm in Matlab) and statistical tests were conducted in a pairwise manner using t-test.

Data Sharing
The dataset used in this manuscript has been published as a dataset publication [54].The code to conduct all the presented analysis, stimuli as well as the figures presented in this manuscript is publicly available [48] and has been developed with Matlab 2016b (MathWorks, Natick MA).This code can be used to download the data set and conduct all analyses presented in this paper.

Figure 1 .
Figure 1.Model-Based Fixation Similarity Analysis.(A) 8 exploration patterns (colored frames) overlaid on 8face stimuli (numbered 1 to 8) calibrated to span a circular similarity continuum across two dimensions (gender and identity; see also SFig. 1 for stimuli).A pair of most dissimilar faces was randomly selected as CS+ (red border) and CS-(cyan border; see color wheel for color code).The similarity relationships among the 8 faces and the resulting exploration patterns are depicted as two 8 by 8 matrices.Physical similarity (top right panel) between all pair-wise combination of faces were calibrated (see methods and SFig.2A) to have a perfect circular similarity, characterized by highest similarity (blue) between neighbors, and lowest similarity (yellow) for opposing pairs (see also SFig. 2 for calibration).Fixation Similarity Analysis summarizes the similarity relationship between the 8 exploration patterns as a symmetric 8x8 matrix (bottom right panel). 4 th and 8 th columns (and rows) are aligned with the CS+ and CS-, respectively.(B-E) Multidimensional scaling representation of four theoretical similarity relationships between exploration maps through learning.Each colored node represents one exploration pattern (same color scheme; red: CS+; cyan: CS-), where internode distances are proportional to dissimilarity between exploration patterns, depicted in bottom panels as 8 by 8 matrices.These matrices are further decomposed onto two orthogonal components (middle row), centered either on the CS+/CS-(specific component) or +90°/-90° faces (unspecific component).In (B), equal contribution of components results in a circularly similar exploration patterns.In (C), a stronger contribution results in a better global separation of all exploration patterns (denoted by radial arrows second column; shaded nodes depict the first hypothesis).In (D), stronger contribution of the specific component results in a biased separation of exploration patterns specifically along the adversity gradient defined between the CS+ and CS-nodes.In (E), a Gaussian covariance component centered on the CS+ face can specifically increase the similarity of exploration patterns for faces similar to the CS+, resulting in circularly shifted nodes (circular arrows) while preserving the global circularity of the similarity relationships.(same colormap as in A).
2 for calibration).The similarity relationship between all pair-wise faces conformed to a near perfect circular organization (Fig 1A, top right panel), such that dissimilarity varied with angular difference between faces (lowest for left and right neighbors and highest for opposing faces) with equidistant angular steps.

Figure 2 .
Figure 2. Univariate Characterization of Aversive Learning (A) On every trial, one out of 8 faces was presented for 1.5 seconds (conditions coded by colored frames, see color wheel).The fixation cross indicating the start of a trial was randomly placed outside of the face on either the left or right side.For each volunteer, a pair of most dissimilar faces was randomly selected as the CS+ (red) and CS-(cyan).During baseline, UCSs (indicated by shock sign) were completely predictable by a triangular signboard.During conditioning and generalization, CS+ face was paired with an aversive outcome in ~30% of trials.Null trials (gray) were presented resulting in a SOA of 6 or 12s.(B) Group-level z-scored skin-conductance responses (n = 51) and (C) subjective ratings of UCS expectancy (n = 61) for baseline, conditioning and generalization phases for individual faces (same color code).Responses are aligned to the CS+ for each volunteer separately.For baseline and generalization phases, the winning model (Gaussian vs. Null model) is depicted as either a black line (null model) or curve (Gaussian model).Asterisk depicts significant differences between responses to CS+ and CSstimuli.(***: p < .001,t-test).Error bars denote SEM.

Figure 3
Figure 3 Impact of Aversive Learning on Fixation Counts at Four Different Regions of Interest.(A) Fixation density maps (FDMs) of two volunteers preferentially fixating on the left (top row) or right eye (bottom row) during the generalization phase.FDMs for the 8 faces are aligned to individual CS+ face (colored frame), and smoothed with a Gaussian kernel of 1 visual degrees.To emphasize differences between conditions, average pattern is removed from single-conditions for each volunteer separately (dark blue: less fixation than average, yellow: more fixation than average).(B) Percentage change in fixation density for different faces (colored bars) within 4 different regions of interests (black contours in inset).Y-axis shows the difference between generalization and baseline phases (values > 0 represent more fixation during generalization).Lines or curves indicate the winning model Gaussian model vs. Flat null model).Errorbars: SEM.

Figure 4 .
Figure 4. Multivariate analysis of Exploration Patterns and Fixation Similarity Analysis (A)Dissimilarity matrices of exploration patterns for baseline (left panel) and generalization phases (right panel).Fourth and eight columns (and rows) are aligned with each volunteer's CS+ and CS-faces, respectively.Asterisks on the upper half denote significant differences in dissimilarity values for the corresponding element between baseline and generalization phases.(B) Multidimensional representational similarity analysis conducted jointly on 16x16 dissimilarity matrix comprising baseline and generalization phases (not shown).Distances between nodes are proportional to the dissimilarity between corresponding FDMs (open circles: baseline; filled circles: generalization phase; same color scheme as previous).(C) depicts parameter estimates for the 3 tested models (left: bottom-up saliency; middle: adversity categorization; right: univariate generalization) fitted to individual volunteers (M ± SEM; w circle : weight for the circular component, which is the sum of equally weighted specific and unspecific components; w specific /w unspecific : weight of two components centered on the CS+/CS-and +90°/-90° faces, respectively; w Gauss : weight for component derived from univariate generalization profiles; **: p < .01;***: p < .001,paired t-test).
-reviewed) is the author/funder.All rights reserved.No reuse allowed without permission.