Decoding social knowledge in the human brain

The present functional MRI study addressed how the brain maps different aspects of social information. We focused on two key dimensions of social knowledge: affect and likableness. Thirty participants were presented with audio definitions, half referring to affective (e.g. empathetic) and half to non-affective concepts (e.g. intelligent). Orthogonally, half of the concepts were highly likable (e.g. sincere) and half were socially undesirable (e.g. liar). We used a support vector machine to delineate how both concept dimensions are represented in a set of 9 a priori brain regions defined from previous meta-analyses on semantic and social cognition. We show that average decoding in semantic regions (e.g. lateral temporal lobe, inferior frontal gyrus, and precuneus) outperformed social ROIs (e.g. insula and anterior cingulate), with the lateral temporal lobe containing the highest amount of information about the affect and likableness of social concepts. We also found that the insula had a bias towards affect while the likableness dimension was better represented in anterior cingulate cortex. Our results do not support a modular view of social knowledge representation. They rather indicate that the brain representation of social concepts implicates a distributed network of regions that involves ‘domain-specific’ social cognitive systems, but with a greater dependence on language-semantic processing.

head coil allowed participants to communicate with the researchers between sequences. 136 2.3 Experimental procedure 137 We selected 36 social concepts from the list reported by Anderson (1968), for which 138 we developed short audio definitions controlling for sentence length. We categorized all 139 social concepts following a 2x2 factorial design using the concept dimensions of affect 140 and likableness. First, half of the concepts referred to affective states, making an explicit 141 mention to the emotions of oneself or others (see left panel in Table 1), while the other 142 half involved non-affective mental states, referring to interpersonal behavior that does not explicitly involve any emotional content or state (see right panel in Table1). Second, half 144 of the concepts involved highly likable interpersonal behavior (see upper half in Table 1), whereas the other half described highly unlikable social behavior (see bottom half in Table   146 1). We kept the number of concepts in each category equivalent, with 9 social concepts 147 in each of the four subcategories (e.g. high affect and low likableness). Each trial began 148 with a fixation period of 250 ms followed by a blank screen for 500 ms (see Figure 1B). 149 Then, participants listened to the definition of a social concept for 3500 ms (e.g. 'She 150 gets sad when seeing someone suffering and tries to ease their pain'; see Table 1 for 151 the complete list of social concept definitions), followed by another period of 2000 ms 152 in which they were instructed to mentally simulate a person of their own choice (e.g. a 153 relative, acquaintance, or famous character) behaving as described in the definition. 154 All 36 social concepts were presented in each functional run, with concept order ran-155 domized between runs. A run lasted approximately six and a half minutes. To facilitate 156 the estimation of the peak of the HRF across the different trials, we included an additional 157 jitter so that the time between the offset of the current stimulus and the onset of the next 158 audio definition varied between 6 and 8 seconds. The jitter followed a pseudo-exponential 159 distribution resulting in 50% of trials with an ITI of 6 s, 25% of 6.5 s, 12.5% of 7 s and so  Before and after the MRI scanning session, we asked participants to rate the affect and lik- 164 ableness of each concept definition on a scale from 0 to 100. We used these measurements 165 to analyze the test-retest reliability of self-ratings of the concept definitions. 166 2.5 MRI data preprocessing 167 We first converted all MRI data from DICOM to NIfTI format using MRIConvert (http://l 168 cni.uoregon.edu/downloads/mriconvert). We then preprocessed the MRI data using FEAT 169 6 (fMRI Expert Analysis Tool) from the FSL suite (FMRIB Software Library; v5.0.9). We 170 removed the first 10 volumes of each functional run to ensure steady-state magnetization. 171 We used FSL's brain extraction tool 2.1 (BET) to remove non-brain tissue (Smith 2002) 172 and ICA-based Automatic Removal of Motion Artifacts (ICA-AROMA) to identify and 173 remove motion-related artifacts (Pruim et al. 2015). We applied spatial smoothing to the 174 data using a Gaussian kernel of 3 mm FWHM and a high-pass filter with a cutoff of 90 175 s (estimated using FEAT's "Estimate High Pass Filter Tool" based on the analysis of the 176 frequency content of the design). All functional images were coaligned to a reference 177 volume from the first run for each participant.  2018). These were bilateral since the results from that meta-analysis on the functional 192 connectivity architecture of the social brain showed a strong coupling of the connectivity 193 maps between bilateral regions. Social ROIs included the insula (Ins), and anterior (ACC) 194 and posterior cingulate cortices (PCC). Moreover, we included the anterior temporal lobe and social information processing. Finally, we included the primary visual cortex (V1) 197 as a control region. We used FreeSurfer v.6.0.0 for automatic segmentation of the struc-198 tural images. We then obtained masks for each a priori ROI in anatomical space using 199 FreeSurfer's LookUp Table. After visual inspection of the anatomical masks, we trans-

TABLE 1
Definitions of social concepts. All concepts used in the experiment followed a 2x2 factorial design. Half of the concepts made an explicit mention to the emotions of oneself or others, while the other half referred to interpersonal behavior that does not explicitly involve any emotional content or state. Second, half of the concepts involved highly likable interpersonal behavior, whereas the other half described highly unlikable social behavior. 204 We conducted multivariate pattern analysis as implemented in the Python libraries scikit-   After preprocessing the MRI data, we used the output generated from PsychoPy during 224 the experimental task to label the relevant scans with an attribute for each binomial clas-225 sification (i.e. high vs. low affect; high vs. low likableness) for each subject. We then 226 removed invariant features (i.e. voxels whose BOLD activity did not vary throughout the 227 length of a functional run) and stacked the data from all 8 functional runs after z-score 228 normalization and linear detrending (see Figure 1D). Finally, we generated examples for   Figure 1E). 247 We then used Principal Component Analysis (PCA) with the default settings as pro-248 vided by scikit-learn to reduce the dimensionality of the data and, thus, the chances of

298
As mentioned above, we used two cross-validation procedures for the binomial classifi-299 cation analyses. In the partition-level CV, we used independent partitions of the scans for training and testing (i.e. training occurred within partitions containing 80% of the data; then, we tested in the remaining 20%). Nonetheless, scans in both sets could refer to the 302 same concepts. In the second cross-validation procedure, the training and testing parti-303 tions did not include scans corresponding to the same concepts, hence providing a better 304 estimate of out-of-sample generalization.

305
A set of paired t-tests confirmed that the decoding accuracy of both affect and likable-306 ness was significantly above the empirical chance level in all ROIs when using partition-307 level CV (see Figure 3A). We then repeated the decoding analysis with the second, item-308 level CV procedure that allowed testing the generalizability of the brain representations of  Table 2 (see   315   Table 4 in the supplementary materials for results using item-level CV). 316 We then used two repeated-measures ANOVAs with ROI as a factor to analyze signif-317 icant differences in decoding accuracy among ROIs for each classification problem.

318
On the one hand, using the partition-level CV we found a main effect of ROI for  Table 5 & Table 7 in the supplementary materials for all post hoc tests of the classification of likableness when using partition-level and item-level CV, respectively).

345
In sum, we found that our selection of semantic and social regions contains detailed 346 information on the meaning of social concepts that allows a linear classifier to distinguish 347 between subcategories based on their affect and likableness. These results were supported 348 by a second, more stringent cross-validation procedure, which yielded results almost iden-349 tical to the typical 5-fold CV procedure using stratified random partitions of all brain scans 350 (see Figure 5 in the supplementary materials for a comparison of the distribution of all de-351 coding accuracies across concept dimensions and cross-validation procedures).

352
To further evaluate whether our ROIs showed a bias towards the decoding of the af-353 fect or likableness of social knowledge, we performed a repeated-measures ANOVA with 354 two factors: ROI and concept dimension (i.e. affect vs. likableness). We found a main 355   TABLE 2 Binomial classification of the affect and likableness of social knowledge. Correct classification rates, chance performance, and summary statistics for the binomial classification problems of affect (high vs. low; top) and likableness (high vs. low; bottom) using a partition-level cross-validation.  Table 3). Moreover,  Table 8 for decoding accuracy and summary statistics of all ROIs using

394
As a final verification check, we addressed whether the neural pattern underlying each 395 concept dimension of social knowledge was distinct. Accordingly, we tested whether an 396   TABLE 3 Comparison between the classification accuracies of the affect and likableness of social knowledge. Correct classification rates and summary statistics for the contrast between the classification of likableness (high vs. low) vs. affect (high vs. likableness) using a partition-level cross-validation.  Table 9). The present fMRI study investigated how social knowledge is represented in the brain.

405
In particular, we asked whether the affect and likableness of interpersonal behavior are 406 relevant features that the brain uses to construct such representations. We used multi-407 variate classification analyses from BOLD signals in semantic (LTL, IFG, and Prec) and 408 social-cognitive processing regions (Ins, ACC, and PCC) to understand how these key 409 dimensions of social concepts are represented in the human brain. The main goal of this 410 study was to test whether activity patterns in functionally distinct brain regions contain 411 information that is detailed enough to allow a linear classifier to distinguish between sub-412 categories of social concepts defined a priori. 413 We found that most of these regions can decode above chance the affect and lik- that both may rely to some extent in domain-specific processing. 433 We also evaluated whether the concept dimensions of affect and likableness played 434 different roles in the brain representation of social knowledge. We found that the ACC 435 decoded the likableness of interpersonal behavior significantly better than its affect when 436 using a partition-level CV. This is congruent with the self-reported ratings we obtained 437 from the participants, which indicate that ratings of likableness were more concentrated  529   TABLE 4 Binomial classification of the affect and likableness of social knowledge. Correct classification rates, chance performance, and summary statistics for the binomial classification problems of affect (high vs. low; top) and likableness (high vs. low; bottom) using an item-level cross-validation.

A Supplementary materials
FIGURE 5: Distributions of decoding accuracies of social concepts. (A) Classification accuracy of both the affect and likableness of social knowledge was significantly above chance in all ROIs using a partition-level cross-validation procedure. A repeatedmeasures ANOVA showed an interaction effect between ROI and concept dimension was driven by the ACC (p < .001), which shows a preference for the classification of likableness. The shaded area indicates the average, empirically-estimated chance level (M = 0.53). (B) Same information but using an item-level cross-validation procedure. Note that the interaction effect between ROI and dimension was driven by the Ins (p < .001) instead of the ACC when using the latter cross-validation procedure, showing a preference for the classification of affect.

TABLE 5
Post hoc comparisons of classification accuracies in each pair of ROIs for the main effect of ROI in the binomial classification of affect (high vs. low) using a partition-level crossvalidation.

TABLE 6
Post hoc comparisons of classification accuracies in each pair of ROIs for the main effect of ROI in the binomial classification of affect (high vs. low) using item-level crossvalidation.

TABLE 7
Post hoc comparisons of classification accuracies in each pair of ROIs for the main effect of ROI in the binomial classification of likableness (high vs. low) using item-level crossvalidation.

TABLE 8
Comparison between the classification accuracies of the affect and likableness of social knowledge. Correct classification rates and summary statistics for the contrast between the classification of likableness (high vs. low) vs. affect (high vs. likableness) using an item-level cross-validation.

TABLE 9
Decoding accuracies across concept dimensions. An SVC trained to discriminate the affect dimension (high vs. low) did not generalized to classify examples in the likableness condition, and vice versa, using the item-level CV.