Fearful expressions enhance recognition memory: Electrophysiological evidence
Introduction
The interaction between emotion, perception and memory is a central topic in cognitive neuroscience. Several studies have investigated the effect of emotions on encoding, i.e. the processes that store data in memory, and retrieval, i.e. the recall of information from long term memory. Behavioral, electrophysiological, lesion and brain imaging studies have provided abundant evidence on the influence of emotion on memory (Adolphs et al., 2000, Bradley et al., 1992, Cahill and McGaugh, 1998, Dolan, 2002, Phelps, 2004) showing that emotional stimuli are more efficiently encoded, consolidated, and retrieved than neutral stimuli (LaBar & Cabeza, 2006). In keeping with this evidence, emotional arousing stimuli with negative value, such as stories (Cahill et al., 1996, Heuer and Reisemberg, 1990), pictures (Hamann, et al., 1999), words (Kensinger & Corkin, 2004), real-life events (e.g., D'Argembeau et al., 2003, Talarico et al., 2004), events depicted in film clips (Burke et al., 1992, Christianson et al., 1991), are better remembered than similar material lacking emotional content (for a review, see Kensinger, 2004, LaBar and Cabeza, 2006, Phelps, 2004). In a similar vein, electrophysiological evidences (Maratos and Rugg, 2001, Inaba et al., 2005) have shown that a better memory performance for negative words, compared with neutral and positive, is associated to larger old/new effects. Similarly, studies investigating the electrophysiological correlates of memory performance for pleasant vs. unpleasant scenes (mainly from IAPS) reported an early (200–500 ms) fronto-central enhancement of the old/new effect only for unpleasant stimuli (Schaefer et al., 2011, Van Strien et al., 2009). Despite the wide literature on memory for emotional pictures with positive and negative valence there is a paucity of studies that systematically investigate the effect of facial expression on recognition memory, i.e. the memory retrieval process that occurs through the re-experiencing of information and involves a process of matching the external data with the memory traces. Recognition memory for faces is a crucial topic for social neuroscience since the processing and the encoding of emotional signals conveyed by faces is used to form impressions, to evaluate the intentions of others and to adapt future behavior (Adolphs, 2003; for review Vuilleumier & Pourtois, 2007). In particular, first emotional impressions about a person have been shown to affect face recognition memory (D'Argembeau and Van der Linden, 2007, Todorov et al., 2007), with the activation of several brain regions when faces were remembered as foes rather than friends (Vrticka, Andersson, Sander, & Vuilleumier, 2009). Consistently, negative facial expressions, that represent a potential danger, are more efficiently processed than positive expressions (Öhman et al., 2001, Williams et al., 2006). As far as temporal information is concerned, event related potentials (ERPs), characterized by an excellent temporal resolution, showed that a rapid extraction of emotional signals begins early and unfolds over a large temporal window and across several processing stages (Eimer & Holmes, 2007). Differences between emotional and neutral faces have been reported on fronto-central components beginning approximately 120 ms after face onset. These findings suggest that some information about the affective meaning of visual stimuli can be extracted rapidly, probably by fronto-temporal limbic structures (Eimer and Holmes, 2002, Kawasaki et al., 2001), before the construction of a detailed structural representation is computed (~ 170 ms, Bentin, Allison, Puce, Perez, & McCarthy, 1996). In addition, consistently with the view that signals of potential threat are given precedence in the neural processing streams, an early enhanced frontal positivity for fear, but not happiness, has been described (Williams et al., 2006). Moreover, several studies (Campanella et al., 2002, Olofsson et al., 2008, Schupp et al., 2004) have found that the emotional content of faces is also processed at later processing stages, as reflected by the Late Positive Potential (LPP) which consists of a sustained positivity at about 300 ms following stimulus onset and is enhanced for pleasant and unpleasant as compared to neutral stimuli (Hajcak et al., 2009, Schupp et al., 2000). Although the onset latency of P300 and LPP is similar, the shorter duration of P300, as well as its sensitivity to non-emotional manipulations, led to the suggestion that the P300 might reflect the initial allocation of attention to motivationally salient stimuli, whereas the later LPP might be more specifically related to the stimulus significance (Schupp et al., 2004, Foti and Hajcak, 2008, Hajcak et al., 2009). Interestingly, the LPP has been related also to memory encoding (Dolcos & Cabeza, 2002). These findings highlight the important interaction between emotion and perception; however, only few studies have tracked the temporal dynamics of the effect of facial emotion on memory. Moreover, the results of the studies investigating the effect of emotion on memory are scarcely consistent, probably due to the fact that different paradigms have been used. For example, Graham & Cabeza (2001), during a study-test procedure with only happy and neutral faces, found a left lateralized late frontal effect for happy but a right late frontal effect for neutral faces. Furthermore, Johansson, Mecklinger, and Treese(2004) reported an enhancement of the parietal old/new effect for correctly remembered negative faces. However, it is impossible to link the memory effect to a specific type of emotion (i.e. anger, fearful or sadness) since no distinction among negative emotional faces was made. More recently, the simultaneous use of electroencephalography (EEG) and magnetoencephalography (MEG) during the repetition of fearful, happy and neutral faces showed a greater amplitude of the M300 on frontal sites for the first repetition of fearful compared to happy and neutral faces. This effect was attributed to the higher arousing value of fearful relative to happy and neutral stimuli (Morel, Ponz, Mercier, Vuilleumier, & George, 2009).
The aim of the present study was to assess the temporal interaction between facial expression and memory. Specifically, ERPs were used to investigate at what processing stage emotional expressions influence face encoding and recognition memory. To our knowledge there are only few studies that were aimed at characterizing the temporal sequence of the encoding and retrieval of facial expressions. Therefore, our main goal was to tap the whole time course of face recognition to assess when emotions might influence early ERP components, related to perceptual processing, or later ERP components related to memory encoding and retrieval mechanisms. Our hypothesis is that recognition processes of threat-related stimuli, that have to be processed rapidly and remembered for an adaptive behavior, may be enhanced by top-down influences that act not only on memory processes but also on early stages of processing.
The experimental design used was an old/new paradigm with two phases. A study phase in which were presented faces with different expressions (happy, fearful and neutral) and a subsequent test phase in which the participants had to discriminate old from new faces. It is worth stressing that a novel aspect of our study was that during the test phase all faces had neutral expression. The rationale for the use of this procedure was to characterize the processes involved in emotional evaluation during encoding and to investigate the effects of the emotional encoding on a subsequent retrieval. Furthermore, this procedure enabled us to investigate the retrieval of emotional events while controlling for confounding perceptual influences. Considering the saliency of emotional expressions and their importance in social cognition, we predicted that they might influence encoding and recognition beginning at early stages of face perception. Our hypothesis is that fearful faces, since they convey signals of threat and represent relevant stimuli for adaptive behaviors, are better encoded and retrieved. Therefore, we predict that fearful faces might enhance, during encoding, ERPs responses linked to emotions (LPP) and encoding.
To further assess the interaction between emotional encoding and successful memory formation, we investigated the possible differences in the study-phase activity as a function of subsequent-memory performance. ERP responses during encoding were compared for items subsequently remembered relative to items later forgotten, to search for the presence of a differential neural activity based on memory and emotions, i.e. “Dm (difference in memory) effect” (Lucas et al., 2011, Paller and Wagner, 2002).
Furthermore, the emotional salience of fearful stimuli may influence also processing stages involved in information retrieval. If so, the recognition of fearful faces may also enhance later recognition related components such as the mid-frontal effect (FN400) linked to familiarity, implicit memory and priming (Curran, 2000, Curran and Cleary, 2003) and the parietal old/new effect that reflects intentional episodic recollection (Donaldson & Rugg, 1998).
Section snippets
Participants
Eighteen healthy university students (females = 8; age range: 19–28 years) with previous experience with ERP recording participated in the experiment. The subjects were all naïve to the stimuli presented and to the aim of present study. All participants were right-handed (as assessed through the Edinburgh Handedness Inventory (Viggiano, Borelli, Vannucci, & Rocchetti, 2001), had normal or corrected-to-normal vision and no history of mental illness.
Stimuli and procedure
340 grayscale pictures of unfamiliar faces were
Accuracy
No significant differences emerged for main effects or interactions (ps > 0.05). Mean accuracy values were 80.8% (SD = 8.5) for neutral faces, 78.6% (SD = 10.2) for fearful faces and 80.3% (SD = 11.2) for happy faces.
Reaction times (RTs)
As for accuracy, no significant differences emerged: all the main effects and interactions were not significant (ps > 0.05). RTs were 997.9 ms (SD = 148) for neutral faces, 1139.4 ms (SD = 184) for fearful faces and 1052.9 ms (SD = 206) for happy faces.
Test phase
An overview of subjects' recognition memory
Discussion
The aim of the present study was to characterize the temporal dynamics of the interaction between facial emotions and memory. Specifically, ERPs were used to find out when in the time course of face processing, faces encoded with emotional or neutral expressions affect recognition memory.
Acknowledgments
We thank doctor Viviana Orlando for her helpful contribution to this work.
References (106)
- et al.
Topographic organization of projections from the amygdala to the visual cortex in the macaque monkey
Neuroscience
(2003) - et al.
Early adaptation to repeated unfamiliar faces across viewpoint changes in the right hemisphere: Evidence from the N170 ERP component
Neuropsychologia
(2009) - et al.
Mechanisms of emotional arousal and lasting declarative memory
Trends in Neuroscience
(1998) - et al.
Using ERPs to dissociate recollection from familiarity in picture recognition
Brain Research. Cognitive Brain Research
(2003) - et al.
The fN400 indexes familiarity-based recognition of faces
NeuroImage
(2007) - et al.
Recognition memory for new associations: Electrophysiological evidence for the role of recollection
Neuropsychologia
(1998) - et al.
Dissociable neural correlates for familiarity and recollection during the encoding and retrieval of pictures
Cognitive Brain Research
(2004) Effects of face inversion on the structural encoding and recognition of faces. Evidence from event-related brain potentials
Brain Research. Cognitive Brain Research
(2000)Event-related brain potentials distinguish processing stages involved in face perception and recognition
Clinical Neurophysiology
(2000)- et al.
Event-related brain potential correlates of emotional face processing
Neuropsychologia
(2007)
An event-related potential study of encoding in young and older adults
Neuropsychologia
“Did you see him in the newspaper?” electrophysiological correlates of context and valence in face processing
Brain Research
Neural systems for recognition of familiar faces
Neuropsychologia
The time course of repetition effects for familiar faces and objects: An ERP study
Brain Research
Motivated and controlled attention to emotion: Time-course of the late positive potential
Clinical Neurophysiology
Neural evidence of effects of emotional valence on word recognition
International. Journal of Psychophysiology
Temporal dynamics of face repetition suppression
Brain Research Bulletin
Amygdala modulation of parahippocampal and frontal regions during emotionally influenced memory storage
NeuroImage
A direct brainstem-amygdala-cortical ‘alarm’ system for subliminal signals of fear
NeuroImage
Interplay between familiarity and orientation in face processing: An ERP study
International Journal of Psychophysiology
Temporal dynamics of face inversion at encoding and retrieval
Clinical Neurophysiology
Deep and shallow encoding effects on face recognition: An ERP study
International Journal of Psychophysiology
When memory meets beauty: Insights from event-related potentials
Biological Psychology
Some methodological issues in assessing attentional biases for threatening faces in anxiety: A replication study using a modified version of the probe detection task
Behavioral Research Therapy
EEG–MEG evidence for early differential repetition effects for fearful, happy and neutral faces
Brain Research
Affective picture processing: An integrative review of ERP findings
Biological Psychology
Validating neural correlates of familiarity
Trends in Cognitive Science
Observing the transformation of experience into memory
Trends in Cognitive Science
Evidence accumulation in cell populations responsive to faces: An account of generalisation of recognition without mental transformations
Cognition
Human emotion and memory: Interactions of the amygdala and hippocampal complex
Current Opinion in Neurobiology
Contributions of the amygdala to emotion processing: From animal models to human behavior
Neuron
Task modulation of brain activity related to familiar and unfamiliar face processing: An ERP study
Clinical Neurophysiology
Does physical interstimulus variance account for early electrophysiological face sensitive responses in the human brain? Ten lessons on the N170
NeuroImage
Event-related potentials and recognition memory
Trends in Cognitive Science
Human recognition memory: A cognitive neuroscience perspective
Trends in Cognitive Science
Electrophysiological correlates of remembering emotional pictures
NeuroImage
A face to remember: Emotional expression modulates prefrontal activity during memory formation
NeuroImage
Non-spatial attentional effects on P1
Clinical Neurophysiology
Spontaneous retrieval of affective person knowledge in face perception
Neuropsychologia
Valence interacts with the early ERP old/new effect and arousal with the sustained ERP old/new effect for affective pictures
Brain Research
Distributed and interactive brain mechanisms during emotion face perception: Evidence from functional neuroimaging
Neuropsychologia
Cognitive neuroscience of human social behaviour
Nature Reviews Neuroscience
Impaired emotional declarative memory following unilateral amygdala damage
Learning and Memory
Lesions of the human amygdala impair enhanced perception of emotionally salient events
Nature
Time course and specificity of event-related potentials to emotional expressions
Neuroreport
Threat-related attentional bias in anxious and nonanxious individuals: A meta-analytic study
Psychological Bulletin
Electrophysiological studies of face perception in humans
Journal of Cognitive Neuroscience
Structural encoding and identification in face processing: ERP evidence for separate mechanisms
Cognitive Neuropsychology
Meaningful processing of meaningless stimuli: The influence of perceptual experience on early visual processing of faces
Cognition
The face-specific n170 component is modulated by emotional facial expression
Behavioral and Brain Functions
Cited by (70)
Sensitivity of late ERP old/new effects in source memory to self-referential encoding focus and stimulus emotionality
2023, Neurobiology of Learning and MemoryEffect of retrieval reward on episodic recognition with different difficulty: ERP evidence
2023, International Journal of PsychophysiologyThe priming effects of emotional vocal expressions on face encoding and recognition: An ERP study
2023, International Journal of PsychophysiologyHuman face and gaze perception is highly context specific and involves bottom-up and top-down neural processing
2022, Neuroscience and Biobehavioral ReviewsIs there a self-positivity bias for destination memory? Behavioral and ERP evidence
2021, Acta Psychologica