Elsevier

Acta Psychologica

Volume 139, Issue 1, January 2012, Pages 7-18
Acta Psychologica

Fearful expressions enhance recognition memory: Electrophysiological evidence

https://doi.org/10.1016/j.actpsy.2011.09.015Get rights and content

Abstract

Facial expressions play a key role in affective and social behavior. However, the temporal dynamics of the brain responses to emotional faces remain still unclear, in particular an open question is at what stage of face processing expressions might influence encoding and recognition memory. To try and answer this question we recorded the event-related potentials (ERPs) elicited in an old/new recognition task. A novel aspect of the present design was that whereas faces were presented during the study phase with either a happy, fearful or neutral expression, they were always neutral during the memory retrieval task.

The ERP results showed three main findings: An enhanced early fronto-central positivity for faces encoded as fearful, both during the study and the retrieval phase. During encoding subsequent memory (Dm effect) was influenced by emotion. At retrieval the early components P100 and N170 were modulated by the emotional expression of the face at the encoding phase. Finally, the later ERP components related to recognition memory were modulated by the previously encoded facial expressions.

Overall, these results suggest that face recognition is modulated by top-down influences from brain areas associated with emotional memory, enhancing encoding and retrieval in particular for fearful emotional expressions.

Introduction

The interaction between emotion, perception and memory is a central topic in cognitive neuroscience. Several studies have investigated the effect of emotions on encoding, i.e. the processes that store data in memory, and retrieval, i.e. the recall of information from long term memory. Behavioral, electrophysiological, lesion and brain imaging studies have provided abundant evidence on the influence of emotion on memory (Adolphs et al., 2000, Bradley et al., 1992, Cahill and McGaugh, 1998, Dolan, 2002, Phelps, 2004) showing that emotional stimuli are more efficiently encoded, consolidated, and retrieved than neutral stimuli (LaBar & Cabeza, 2006). In keeping with this evidence, emotional arousing stimuli with negative value, such as stories (Cahill et al., 1996, Heuer and Reisemberg, 1990), pictures (Hamann, et al., 1999), words (Kensinger & Corkin, 2004), real-life events (e.g., D'Argembeau et al., 2003, Talarico et al., 2004), events depicted in film clips (Burke et al., 1992, Christianson et al., 1991), are better remembered than similar material lacking emotional content (for a review, see Kensinger, 2004, LaBar and Cabeza, 2006, Phelps, 2004). In a similar vein, electrophysiological evidences (Maratos and Rugg, 2001, Inaba et al., 2005) have shown that a better memory performance for negative words, compared with neutral and positive, is associated to larger old/new effects. Similarly, studies investigating the electrophysiological correlates of memory performance for pleasant vs. unpleasant scenes (mainly from IAPS) reported an early (200–500 ms) fronto-central enhancement of the old/new effect only for unpleasant stimuli (Schaefer et al., 2011, Van Strien et al., 2009). Despite the wide literature on memory for emotional pictures with positive and negative valence there is a paucity of studies that systematically investigate the effect of facial expression on recognition memory, i.e. the memory retrieval process that occurs through the re-experiencing of information and involves a process of matching the external data with the memory traces. Recognition memory for faces is a crucial topic for social neuroscience since the processing and the encoding of emotional signals conveyed by faces is used to form impressions, to evaluate the intentions of others and to adapt future behavior (Adolphs, 2003; for review Vuilleumier & Pourtois, 2007). In particular, first emotional impressions about a person have been shown to affect face recognition memory (D'Argembeau and Van der Linden, 2007, Todorov et al., 2007), with the activation of several brain regions when faces were remembered as foes rather than friends (Vrticka, Andersson, Sander, & Vuilleumier, 2009). Consistently, negative facial expressions, that represent a potential danger, are more efficiently processed than positive expressions (Öhman et al., 2001, Williams et al., 2006). As far as temporal information is concerned, event related potentials (ERPs), characterized by an excellent temporal resolution, showed that a rapid extraction of emotional signals begins early and unfolds over a large temporal window and across several processing stages (Eimer & Holmes, 2007). Differences between emotional and neutral faces have been reported on fronto-central components beginning approximately 120 ms after face onset. These findings suggest that some information about the affective meaning of visual stimuli can be extracted rapidly, probably by fronto-temporal limbic structures (Eimer and Holmes, 2002, Kawasaki et al., 2001), before the construction of a detailed structural representation is computed (~ 170 ms, Bentin, Allison, Puce, Perez, & McCarthy, 1996). In addition, consistently with the view that signals of potential threat are given precedence in the neural processing streams, an early enhanced frontal positivity for fear, but not happiness, has been described (Williams et al., 2006). Moreover, several studies (Campanella et al., 2002, Olofsson et al., 2008, Schupp et al., 2004) have found that the emotional content of faces is also processed at later processing stages, as reflected by the Late Positive Potential (LPP) which consists of a sustained positivity at about 300 ms following stimulus onset and is enhanced for pleasant and unpleasant as compared to neutral stimuli (Hajcak et al., 2009, Schupp et al., 2000). Although the onset latency of P300 and LPP is similar, the shorter duration of P300, as well as its sensitivity to non-emotional manipulations, led to the suggestion that the P300 might reflect the initial allocation of attention to motivationally salient stimuli, whereas the later LPP might be more specifically related to the stimulus significance (Schupp et al., 2004, Foti and Hajcak, 2008, Hajcak et al., 2009). Interestingly, the LPP has been related also to memory encoding (Dolcos & Cabeza, 2002). These findings highlight the important interaction between emotion and perception; however, only few studies have tracked the temporal dynamics of the effect of facial emotion on memory. Moreover, the results of the studies investigating the effect of emotion on memory are scarcely consistent, probably due to the fact that different paradigms have been used. For example, Graham & Cabeza (2001), during a study-test procedure with only happy and neutral faces, found a left lateralized late frontal effect for happy but a right late frontal effect for neutral faces. Furthermore, Johansson, Mecklinger, and Treese(2004) reported an enhancement of the parietal old/new effect for correctly remembered negative faces. However, it is impossible to link the memory effect to a specific type of emotion (i.e. anger, fearful or sadness) since no distinction among negative emotional faces was made. More recently, the simultaneous use of electroencephalography (EEG) and magnetoencephalography (MEG) during the repetition of fearful, happy and neutral faces showed a greater amplitude of the M300 on frontal sites for the first repetition of fearful compared to happy and neutral faces. This effect was attributed to the higher arousing value of fearful relative to happy and neutral stimuli (Morel, Ponz, Mercier, Vuilleumier, & George, 2009).

The aim of the present study was to assess the temporal interaction between facial expression and memory. Specifically, ERPs were used to investigate at what processing stage emotional expressions influence face encoding and recognition memory. To our knowledge there are only few studies that were aimed at characterizing the temporal sequence of the encoding and retrieval of facial expressions. Therefore, our main goal was to tap the whole time course of face recognition to assess when emotions might influence early ERP components, related to perceptual processing, or later ERP components related to memory encoding and retrieval mechanisms. Our hypothesis is that recognition processes of threat-related stimuli, that have to be processed rapidly and remembered for an adaptive behavior, may be enhanced by top-down influences that act not only on memory processes but also on early stages of processing.

The experimental design used was an old/new paradigm with two phases. A study phase in which were presented faces with different expressions (happy, fearful and neutral) and a subsequent test phase in which the participants had to discriminate old from new faces. It is worth stressing that a novel aspect of our study was that during the test phase all faces had neutral expression. The rationale for the use of this procedure was to characterize the processes involved in emotional evaluation during encoding and to investigate the effects of the emotional encoding on a subsequent retrieval. Furthermore, this procedure enabled us to investigate the retrieval of emotional events while controlling for confounding perceptual influences. Considering the saliency of emotional expressions and their importance in social cognition, we predicted that they might influence encoding and recognition beginning at early stages of face perception. Our hypothesis is that fearful faces, since they convey signals of threat and represent relevant stimuli for adaptive behaviors, are better encoded and retrieved. Therefore, we predict that fearful faces might enhance, during encoding, ERPs responses linked to emotions (LPP) and encoding.

To further assess the interaction between emotional encoding and successful memory formation, we investigated the possible differences in the study-phase activity as a function of subsequent-memory performance. ERP responses during encoding were compared for items subsequently remembered relative to items later forgotten, to search for the presence of a differential neural activity based on memory and emotions, i.e. “Dm (difference in memory) effect” (Lucas et al., 2011, Paller and Wagner, 2002).

Furthermore, the emotional salience of fearful stimuli may influence also processing stages involved in information retrieval. If so, the recognition of fearful faces may also enhance later recognition related components such as the mid-frontal effect (FN400) linked to familiarity, implicit memory and priming (Curran, 2000, Curran and Cleary, 2003) and the parietal old/new effect that reflects intentional episodic recollection (Donaldson & Rugg, 1998).

Section snippets

Participants

Eighteen healthy university students (females = 8; age range: 19–28 years) with previous experience with ERP recording participated in the experiment. The subjects were all naïve to the stimuli presented and to the aim of present study. All participants were right-handed (as assessed through the Edinburgh Handedness Inventory (Viggiano, Borelli, Vannucci, & Rocchetti, 2001), had normal or corrected-to-normal vision and no history of mental illness.

Stimuli and procedure

340 grayscale pictures of unfamiliar faces were

Accuracy

No significant differences emerged for main effects or interactions (ps > 0.05). Mean accuracy values were 80.8% (SD = 8.5) for neutral faces, 78.6% (SD = 10.2) for fearful faces and 80.3% (SD = 11.2) for happy faces.

Reaction times (RTs)

As for accuracy, no significant differences emerged: all the main effects and interactions were not significant (ps > 0.05). RTs were 997.9 ms (SD = 148) for neutral faces, 1139.4 ms (SD = 184) for fearful faces and 1052.9 ms (SD = 206) for happy faces.

Test phase

An overview of subjects' recognition memory

Discussion

The aim of the present study was to characterize the temporal dynamics of the interaction between facial emotions and memory. Specifically, ERPs were used to find out when in the time course of face processing, faces encoded with emotional or neutral expressions affect recognition memory.

Acknowledgments

We thank doctor Viviana Orlando for her helpful contribution to this work.

References (106)

  • D. Friedman et al.

    An event-related potential study of encoding in young and older adults

    Neuropsychologia

    (2000)
  • G. Galli et al.

    “Did you see him in the newspaper?” electrophysiological correlates of context and valence in face processing

    Brain Research

    (2006)
  • M.I. Gobbini et al.

    Neural systems for recognition of familiar faces

    Neuropsychologia

    (2007)
  • C. Guillaume et al.

    The time course of repetition effects for familiar faces and objects: An ERP study

    Brain Research

    (2009)
  • G. Hajcak et al.

    Motivated and controlled attention to emotion: Time-course of the late positive potential

    Clinical Neurophysiology

    (2009)
  • M. Inaba et al.

    Neural evidence of effects of emotional valence on word recognition

    International. Journal of Psychophysiology

    (2005)
  • A. Ishai et al.

    Temporal dynamics of face repetition suppression

    Brain Research Bulletin

    (2006)
  • L. Kilpatrick et al.

    Amygdala modulation of parahippocampal and frontal regions during emotionally influenced memory storage

    NeuroImage

    (2003)
  • B.J. Liddell et al.

    A direct brainstem-amygdala-cortical ‘alarm’ system for subliminal signals of fear

    NeuroImage

    (2005)
  • T. Marzi et al.

    Interplay between familiarity and orientation in face processing: An ERP study

    International Journal of Psychophysiology

    (2007)
  • T. Marzi et al.

    Temporal dynamics of face inversion at encoding and retrieval

    Clinical Neurophysiology

    (2011)
  • T. Marzi et al.

    Deep and shallow encoding effects on face recognition: An ERP study

    International Journal of Psychophysiology

    (2010)
  • T. Marzi et al.

    When memory meets beauty: Insights from event-related potentials

    Biological Psychology

    (2010)
  • K. Mogg et al.

    Some methodological issues in assessing attentional biases for threatening faces in anxiety: A replication study using a modified version of the probe detection task

    Behavioral Research Therapy

    (1999)
  • S. Morel et al.

    EEG–MEG evidence for early differential repetition effects for fearful, happy and neutral faces

    Brain Research

    (2009)
  • J.K. Olofsson et al.

    Affective picture processing: An integrative review of ERP findings

    Biological Psychology

    (2008)
  • K.A. Paller et al.

    Validating neural correlates of familiarity

    Trends in Cognitive Science

    (2007)
  • K.A. Paller et al.

    Observing the transformation of experience into memory

    Trends in Cognitive Science

    (2002)
  • D.I. Perrett et al.

    Evidence accumulation in cell populations responsive to faces: An account of generalisation of recognition without mental transformations

    Cognition

    (1998)
  • E.A. Phelps

    Human emotion and memory: Interactions of the amygdala and hippocampal complex

    Current Opinion in Neurobiology

    (2004)
  • E.A. Phelps et al.

    Contributions of the amygdala to emotion processing: From animal models to human behavior

    Neuron

    (2005)
  • B. Rossion et al.

    Task modulation of brain activity related to familiar and unfamiliar face processing: An ERP study

    Clinical Neurophysiology

    (1999)
  • B. Rossion et al.

    Does physical interstimulus variance account for early electrophysiological face sensitive responses in the human brain? Ten lessons on the N170

    NeuroImage

    (2008)
  • M.D. Rugg et al.

    Event-related potentials and recognition memory

    Trends in Cognitive Science

    (2007)
  • M. Rugg et al.

    Human recognition memory: A cognitive neuroscience perspective

    Trends in Cognitive Science

    (2003)
  • A. Schaefer et al.

    Electrophysiological correlates of remembering emotional pictures

    NeuroImage

    (2011)
  • K. Sergerie et al.

    A face to remember: Emotional expression modulates prefrontal activity during memory formation

    NeuroImage

    (2005)
  • M.J. Taylor

    Non-spatial attentional effects on P1

    Clinical Neurophysiology

    (2002)
  • A. Todorov et al.

    Spontaneous retrieval of affective person knowledge in face perception

    Neuropsychologia

    (2007)
  • J.W. Van Strien et al.

    Valence interacts with the early ERP old/new effect and arousal with the sustained ERP old/new effect for affective pictures

    Brain Research

    (2009)
  • P. Vuilleumier et al.

    Distributed and interactive brain mechanisms during emotion face perception: Evidence from functional neuroimaging

    Neuropsychologia

    (2007)
  • R. Adolphs

    Cognitive neuroscience of human social behaviour

    Nature Reviews Neuroscience

    (2003)
  • R. Adolphs et al.

    Impaired emotional declarative memory following unilateral amygdala damage

    Learning and Memory

    (2000)
  • A.K. Anderson et al.

    Lesions of the human amygdala impair enhanced perception of emotionally salient events

    Nature

    (2001)
  • V. Ashley et al.

    Time course and specificity of event-related potentials to emotional expressions

    Neuroreport

    (2004)
  • Y. Bar-Haim et al.

    Threat-related attentional bias in anxious and nonanxious individuals: A meta-analytic study

    Psychological Bulletin

    (2007)
  • S. Bentin et al.

    Electrophysiological studies of face perception in humans

    Journal of Cognitive Neuroscience

    (1996)
  • S. Bentin et al.

    Structural encoding and identification in face processing: ERP evidence for separate mechanisms

    Cognitive Neuropsychology

    (2000)
  • S. Bentin et al.

    Meaningful processing of meaningless stimuli: The influence of perceptual experience on early visual processing of faces

    Cognition

    (2002)
  • V.C. Blau et al.

    The face-specific n170 component is modulated by emotional facial expression

    Behavioral and Brain Functions

    (2007)
  • Cited by (70)

    View all citing articles on Scopus
    View full text