Abstract
The perception of gender and age of unfamiliar faces is reported to vary idiosyncratically across retinal locations such that, for example, the same androgynous face may appear to be male at one location but female at another. Here we test spatial heterogeneity for the recognition of the identity of personally familiar faces. We found idiosyncratic biases that were stable within subjects and that varied more across locations for low as compared to high familiar faces. These data suggest that like face gender and age, face identity is processed, in part, by independent populations of neurons monitoring restricted spatial regions and that the recognition responses vary for the same face across these different locations. Moreover, repeated exposure to the same face in different portions of the visual field due to repeated and varied social interactions appears to lead to adjustment of these independent face recognition neurons so that the same familiar face is eventually more likely to elicit the same recognition response across widely separated regions.
Significance statement In this work we tested spatial heterogeneity for the recognition of personally familiar faces. We found retinotopic biases that varied more across locations for low as compared to highly familiar faces. The retinotopic biases were idiosyncratic and stable within subjects. Our data suggest that, like face gender and age, face identity is processed by independent populations of neurons monitoring restricted spatial regions and that recognition may vary for the same face at these different locations. Unlike previous findings, our data show how the effect of learning modifies the representation of face identity in retinotopically-organized visual cortex. This new perspective has broader implications for understanding how learning optimizes visual processes for socially salient stimuli.
Acknowledgments
We would like to thank the Marten’s Family Fund for its support.
Introduction
We spend most of our days interacting with acquaintances, family and close friends. Because of these repeated and protracted interactions, the representation of personally familiar faces is rich and complex, as reflected by stronger and more widespread neural activation in the distributed face processing network, as compared to responses to unfamiliar faces (Gobbini and Haxby, 2007; Taylor et al., 2009; Gobbini, 2010; Natu and O’Toole, 2011; Bobes et al., 2013; Sugiura, 2014; Ramon and Gobbini, 2017; Visconti di Oleggio Castello et al., 2017a). Differences in representations of familiar as compared to unfamiliar faces is also reflected in faster detection of familiar faces and more robust recognition (Burton et al., 1999; Gobbini et al., 2013; Ramon et al., 2015; Visconti di Oleggio Castello and Gobbini, 2015; Visconti di Oleggio Castello et al., 2017b). Thus, despite the subjective feeling of expertise with faces in general (Diamond and Carey, 1986), our visual system seems to be optimized for the processing of familiar faces. The mechanisms underlying the prioritized processing of familiar faces are still a matter of investigation (Guntupalli and Gobbini, 2017; Ramon and Gobbini, 2017; Young and Burton, 2017).
The advantage for familiar faces could originate at different stages of the visual processing stream. In a study measuring saccadic reaction time, correct and reliable saccades to familiar faces were recorded as fast as 180 ms when unfamiliar faces were distractors (Visconti di Oleggio Castello and Gobbini, 2015). At such short latencies it is unlikely that a viewpoint-invariant representation of an individual face’s identity drives the saccadic response. To account for facilitated, rapid detection of familiarity, we have previously hypothesized that personally familiar faces may be recognized quickly based on diagnostic, idiosyncratic features, which become highly learned through extensive personal interactions (Visconti di Oleggio Castello and Gobbini, 2015; Visconti di Oleggio Castello et al., 2017b). In a study of perception of gaze direction and head angle, changes in eye gaze were detected around 100ms faster in familiar than in unfamiliar faces (Visconti di Oleggio Castello and Gobbini, 2015). These data provide support for the hypothesis that facial features that are diagnostic for identity are processed more efficiently for familiar as compared to unfamiliar faces. Detection of these features may occur early in the visual hierarchy, allowing an initial, fast differential processing for personally familiar faces.
Processes occurring at early stages of the visual system can show idiosyncratic retinotopic biases (Greenwood et al., 2017). Afraz et al. (2010) reported retinotopic biases for perceiving face gender and age that vary depending on stimulus location in the visual field and were specific to each subject. These results suggest that neurons in higher-order face areas have restricted receptive fields or, equivalently, that diagnostic facial features for gender and age are encoded in retinotopic visual cortices. Here we reasoned that diagnostic visual features that play a role in visual processes for individuating faces may show idiosyncratic retinotopic biases and that these biases may be tuned by repeated interactions with personally familiar faces. Such biases may affect recognition of the identities presented in different parts of the visual field and may be modulated by the familiarity of those identities.
We tested this hypothesis by presenting participants with morphed stimuli of personally familiar individuals that were briefly shown at different retinal locations. In two separate experiments we found that participants showed idiosyncratic biases for specific identities in different visual field locations, and these biases were stable on retesting after weeks. Importantly, the range of the retinal biases was inversely correlated with the reported familiarity of each target identity, suggesting that prolonged personal interactions with the target individuals reduced retinal biases. These findings provide additional support for the hypothesis that asymmetries in the processing of personally familiar faces can arise at stages of the visual processing hierarchy where there is still retinotopic coding.
Materials and Methods
Stimuli
Pictures of the faces of individuals who were personally familiar to the participants (graduate students in the same department) were taken in a photo studio room with the same lighting condition and the same camera. Images of two individuals were used for Experiment 1, and images of three individuals were used for Experiment 2. All individuals portrayed in the stimuli signed written informed consent to use their pictures for research and in publications.
The images were converted to grayscale, resized and centered so that the eyes were aligned in the same position for the three identities, and the background was manually removed. These operations were performed using ImageMagick and Adobe Photoshop CS4. The resulting images were matched in luminance (average pixel intensity) using the SHINE toolbox (function lumMatch) (Willenbockel et al., 2010) after applying an oval mask, so that only pixels belonging to the face were modified. The luminance-matched images were then used to create morph continua (between two identities in Experiment 1, see Figure 2; and among three identities in Experiment 2, see Figure 3) using Abrosoft Fantamorph (v. 5.4.7) with seven percentages of morphing: 0, 17, 33, 50, 67, 83, 100 (see Figures 2, 3).
Experiment 1
Paradigm
The experimental paradigm was similar to that by (Afraz et al., 2010). In every trial participants would see a briefly flashed image in one of eight locations at the periphery of their visual field (see Figure 1). Each image was shown for 50 ms at a distance of 7˚ of visual angle from the fixation point, and subtended approximately 4˚ x 4˚ of visual angle. The images could appear in one of eight locations evenly spaced by 45 angular degrees. For Experiment 1, only the morph ab was used (see Figure 1). Participants were required to maintain fixation on a central red dot subtending approximately 1˚ of visual angle.
After the image disappeared, participants reported which identity they saw using the left (identity a) and right (identity b) arrow keys. There was no time limit for responding, and participants were asked to be as accurate as possible. After responding, participants had to press the spacebar key to continue to the next trial.
Participants performed five blocks containing 112 trials each, for a total of 560 trials. In each block all the images appeared twice for every angular location (8 angular locations × 7 morph percentages × 2 = 112). This provided ten data points for each percentage morphing at each location, for a total of 70 trials at each angular location.
Before the experimental session participants were shown the identities used in the experiment (corresponding to 0% and 100% morphing, see Figure 2), and practiced the task with 20 trials. These data were discarded from the analyses. Participants performed two identical experimental sessions at least four weeks apart.
Participants sat at a distance of approximately 50 cm from the screen, with their chin positioned on a chin-rest. The experiment was run using Psychtoolbox (Kleiner et al., 2007) (version 3.0.12) in MATLAB (R2014b). The screen operated at a resolution of 1920x1200 and a 60Hz refresh rate.
Subjects
We recruited six subjects for this experiment (three males, including one of the authors, MVdOC). The sample size for Experiment 1 was not determined by formal estimates of power, and was limited by the availability of participants familiar with the stimulus identities. After the first experimental session, two participants (one male, one female) were at chance level in the task, thus only data from four subjects (two males, mean age 27.50 ± 2.08 SD) were used for the final analyses.
All subjects had normal or corrected-to-normal vision, and provided written informed consent to participate in the experiment. The study was approved by the Dartmouth College Committee for the Protection of Human Subjects.
Experiment 2
Paradigm
Experiment 2 differed from Experiment 1 in the following parameters (see Figures 1, 3): 1. three morph continua (ab, ac, bc) instead of one; 2. images appeared in four angular locations (45˚, 135˚, 225˚, 315˚) instead of eight; 3. images were shown for 100 ms instead of 50 ms to make the task easier.
All other parameters were the same as in Experiment 1. Participants had to indicate which of the three identities they saw by pressing the left (identity a), right (identity b), or down (identity c) arrow keys.
Participants performed ten blocks containing 84 trials each, for a total of 840 trials. In each block all the images appeared once for every angular location (4 angular locations × 7 morph percentages × 3 morphs = 84). We used 70 data points at every angular location to fit the model for each pair of identities. Thus, we used the responses to different unmorphed images for each pair of identities, ensuring independence of the models.
Before the experimental session participants were shown the identities used in the experiment (corresponding to 0% and 100% morphing, see Figure 3), and practiced the task with 20 trials. These data were discarded from the analyses. Participants performed two experimental sessions at least four weeks apart.
Subjects
Ten participants (five males, mean age 27.30 ± 1.34 SD) participated in Experiment 2, five of which were recruited for Experiment 1 as well. No authors participated in Experiment 2. The sample size (n = 10) was determined using G*Power3 (Faul et al., 2007, 2009) to obtain 80% power at α = 0.05 based on the correlation of the PSE estimates across sessions in Experiment 1, using a bivariate normal model (one-tailed).
All subjects had normal or corrected-to-normal vision, and provided written informed consent to participate in the experiment. The study was approved by the Dartmouth College Committee for the Protection of Human Subjects.
Familiarity and contact scales
After the two experimental sessions, participants completed a questionnaire designed to assess how familiar each participant was with the identities shown in the experiment. Participants saw each target identity, and were asked to complete various scales for that identity. The questionnaire comprised the “Inclusion of the Other in the Self” scale (IOS) (Aron et al., 1992; Gächter et al., 2015), the “Subjective Closeness Inventory” (SCI) (Berscheid et al., 1989), and the “We-scale” (Cialdini et al., 1997). The IOS scale showed two circles increasingly overlapping labeled “You” and “X”, and participants were given the following instructions: Using the figure below select which pair of circles best describes your relationship with this person. In the figure “X” serves as a placeholder for the person shown in the image at the beginning of this section, and you should think of “X” being that person. By selecting the appropriate number please indicate to what extent you and this person are connected (Aron et al., 1992; Gächter et al., 2015). The SCI scale comprised the two following questions: Relative to all your other relationships (both same and opposite sex) how would you characterize your relationship with the person shown at the beginning of this section?, and Relative to what you know about other people's close relationships, how would you characterize your relationship with the person shown at the beginning of this section? Participants responded with a number between one (Not close at all) and seven (Very close) (Berscheid et al., 1989). The We-scale comprised the following question: Please select the appropriate number below to indicate to what extent you would use the term “WE” to characterize you and the person shown at the beginning of this section. Participants responded with a number between one (Not at all) and seven (Very much so). For each participant and each identity we created a composite “familiarity score” by averaging the scores in the three scales.
We also introduced a scale aimed at estimating the amount of interaction or contact between the participant and the target identity. The scale was based on the work by (Idson and Mischel, 2001), and participants were asked to respond Yes/No to the following six questions: Have you ever seen him during a departmental event?, Have you ever seen him during a party?, Have you ever had a group lunch/dinner/drinks with him?, Have you ever had a one-on-one lunch/dinner/drinks with him?, Have you ever texted him personally (not a group message)?, and Have you ever emailed him personally (not a group email)? The responses were converted to 0/1 and for each participant and for each identity we created a “contact score” by summing all the responses.
For each subject separately, to obtain a measure of familiarity and contact related to each morph, we averaged the familiarity and contact scores of each pair of identities (e.g., the familiarity score of morph ab was the average of the scores for identity a and identity b).
Psychometric fit
For both experiments we fitted a group-level psychometric curve using Logit Mixed-Effect models as implemented in lme4 (Bates et al., 2015). For each experiment and each session, we fitted a model of the form
where k indicates the subject, n is the number of angular locations (n = 8 for the first experiment, and n = 4 for the second experiment), Ii is an indicator variable for the angular location, βi are the model fixed-effects, and zi are the subject-level random-effects (random intercept). From this model, we defined for each subject the Point of Subjective Equality (PSE) as the point x such that logit(x) = 0.5, that is for each angular location
Thus, the PSE for subject k at angular location i can be decomposed in a population-level PSE and a subject-specific deviation from the population level, indicated with PSEp and ΔPSEk respectively.
In Experiment 2 we fitted three separate models for each of the morph continua. In addition, prior to fitting we removed all trials in which subjects mistakenly reported a third identity. For example, if an image belonging to morph ab was presented, and subjects responded with c, the trial was removed.
Code and data availability
Code and data for both experiments are available at [link removed: will be made public after publication].
Results
Experiment 1
In this experiment, participants performed a two-alternative forced-choice (AFC) task on identity discrimination. In each trial they saw a face presented for 50 ms, and were asked to indicate which of the two identities they just saw. Each face could appear in one of eight stimulus locations. Participants performed the same experiment with the same task a second time, at least 33 days after the first session (average 35 days ± 4 days standard deviation).
Participants showed stable and idiosyncratic retinal heterogeneity for identification. The PSE estimates for the two sessions were significantly correlated (see Table 1 and Figure 2B), showing stable estimates, and the within-subject correlations of ΔPSEs (see Methods) was significantly higher than the between-subject correlation (correlation difference: 0.87 [0.64, 1.10], 95% BCa confidence intervals (Efron, 1987); see Table 2), showing that the biases were idiosyncratic (see Figure 2A for example fits for two different subjects).
Experiment 2
In this experiment, participants performed a similar task as in Experiment 1, with the following differences. First, each face was presented for 100 ms instead of 50 ms in order to make the task easier; second, each face could belong to one of three morphs, and participants were required to indicate which of three identities the face belonged to; third, each face could appear in four retinal locations instead of eight (see Figure 1). Each participant performed another experimental session at least 28 days after the first session (average 33 days ± 8 days SD).
We found that participants exhibited stable biases across sessions for the three morphs (see Table 1 and Figure 3). Interestingly, within-subjects correlations were higher than between-subjects correlations for the two morphs that included the identity c (morphs ac and bc), but not for morph ab (see Table 2), suggesting stronger differences in spatial heterogeneity caused by identity c. To test this further, we performed a two-way ANOVA on the PSE estimates across sessions with participants and angular locations as factors. The ANOVA was run for each pair of morphs containing the same identity (e.g., for identity a the ANOVA was run on data from morphs ab and ac), and the PSE estimates were transformed to be with respect to the same identity (e.g., for identity b we considered PSEbc and 100 - PSEab). We found significant interactions between participants and angular locations for identity b (F(27, 120) = 1.77, p = 0.01947) and identity c (F(27, 120) = 3.34, p = 3.229e-06), but not identity a (F(27, 120) =1.17, p = 0.2807), confirming that participants showed increased spatial heterogeneity for identities b and c. Moreover, inspecting the PSE estimates for each individual subjects (Figure 4A) revealed lower variance across retinal locations of the biases for morph ab than the other two morphs.
The variance of the average ΔPSE estimates across sessions for each subject was significantly correlated with the reported familiarity of the identities (r = -0.56 [-0.71, -0.30], t(28) = -3.59, p = 0.001248), showing that the strength of the retinal bias for identities was inversely modulated by personal familiarity (see Figure 4B). Because the amount of personal familiarity was correlated with the amount of contact with a target identity (r = 0.45 [0.17, 0.68], t(28) = 2.65, p = 0.01304), we tested whether a linear model predicting ΔPSE with both contact and familiarity as predictors could fit the data better. Both models were significant, but the model with two predictors explained more variance as indicated by higher R2: R2 = 0.45, adjusted R2 = 0.40 for the model with both Familiarity and Contact scores (F(2, 27) = 10.82, p = 0.0003539), and R2 = 0.32, adjusted R2 = 0.29 for the model with the Familiarity score only (F(1, 28) = 12.88, p = 0.001248). Importantly, both predictors were significant (see Table 3), indicating that familiarity modulated the variance of the ΔPSE estimates in addition to modulation based on the amount of contact with a person. After adjusting for the contact score, the variance of the ΔPSE estimates and the familiarity score were still significantly correlated (rp = -0.42 [-0.61, -0.16], t(28) = -2.42, p = 0.02235).
Discussion
Afraz et al. (2010) reported spatial heterogeneity for recognition of facial attributes such as gender and age, suggesting that relatively independent neural populations tuned to facial features might sample different regions of the visual field. Prolonged social interactions with personally familiar faces lead to facilitated, prioritized processing of those faces. Here we wanted to investigate if this learning of face identity through repeated social interactions also affects these local visual processes, by measuring spatial heterogeneity of identity recognition. We measured whether face identification performance for personally familiar faces differed according to the location in the visual field where face images were presented. We found that participants exhibited idiosyncratic, retinotopic biases for different face identities that were stable across experimental sessions. Importantly, the variability of the retinotopic bias was reduced with increased familiarity with the target identities. These data support the hypothesis that familiarity entails learning visual features that affect processing in visual areas with a retinotopic organization (Visconti di Oleggio Castello et al., 2017a).
These results extend the reports of spatial heterogeneity in visual processing to face identification. Similar biases exist for high-level judgments such as face gender and age (Afraz et al., 2010), as well as shape discrimination (Afraz et al., 2010), crowding, and saccadic precision (Greenwood et al., 2017). Afraz et al. (2010) suggested that neurons in IT exhibit biases that are dependent on retinal location because their receptive field sizes are not large enough to provide complete translational invariance, and stimuli in different locations will activate a limited group of neurons. In this work, we show that these perceptual biases for face processing not only exist for gender and age judgments (Afraz et al., 2010), but also for face identification and that these biases are affected by learning. Retinotopic organization is stronger in earlier visual areas, suggesting that high-level judgments of gender, age, and identity may be biased by variability of feature detectors in visual areas such as the occipital face area, or even earlier.
In this work, we showed that the extent of variation in biases across retinal locations was inversely correlated with the reported familiarity with individuals, suggesting that a history of repeated interaction with a person may tune the responses of neurons to that individual in different retinal locations, generating more homogeneous responses. Repeated exposure to the faces of familiar individuals during real-life social interactions results in a detailed representation of the visual appearance of a personally familiar face. Our results showed that both ratings of familiarity and ratings of amount of contact were strong predictors for reduced retinotopic bias. This supports our hypothesis that facilitated processing might be mediated by the development or tuning of detectors for diagnostic fragments of personally familiar faces that may exist in the visual pathway in areas that still have localized analyses and have a retinotopic organization (Gobbini et al., 2013; Visconti di Oleggio Castello et al., 2014, 2017b; Visconti di Oleggio Castello and Gobbini, 2015).
Future research can investigate the mechanism that generates these biases and how learning reduces them. However, our results suggest that prioritized processing for personally familiar faces may exist at relatively early stages of the visual processing hierarchy, as shown by the local biases reported here. We hypothesize that these differences may be one of the mechanisms that underlies the known behavioral advantages for perception of personally familiar faces (Burton et al., 1999; Gobbini and Haxby, 2007; Gobbini, 2010; Gobbini et al., 2013; Visconti di Oleggio Castello et al., 2014, 2017b; Ramon et al., 2015; Visconti di Oleggio Castello and Gobbini, 2015; Chauhan et al., 2017; Ramon and Gobbini, 2017).
Footnotes
Conflict of Interest: The authors declare no competing financial interests.