Abstract
Humans have remarkable episodic long-term memory abilities, capable of storing thousands of objects with significant detail. However, it remains unknown how such episodic memory is utilized during the short-term maintenance of information. Specifically, if people have an episodic memory for an item, how does this affect subsequent working memory for that same item? Here, we demonstrate that under these conditions people can quickly and accurately make use of episodic memory and therefore maintain less information in working memory. We assessed how much information is maintained in working memory by measuring neural activity during the delay period of a working memory task using electroencephalography. We find that despite maintaining less information in working memory when episodic memory representations are available, there is no decrement in memory performance. This suggests people can dynamically disengage working memory and instead use episodic memory when episodic memory is available. However, this does not mean that participants always utilize episodic memory when it is available. In a follow-up experiment, we introduced additional perceptual interference into working memory and found participants actively stored items in working memory even when they had existing episodic memories of those items. These results clarify the conditions under which episodic and working memory operate. Specifically, working memory is engaged when new information is encountered or perceptual interference is high. Episodic memory is otherwise rapidly accessed and utilized in lieu of working memory. These data demonstrate the interactions between working memory and episodic memory are more dynamic and fluid than previously thought.
Significance Statement Exploring the interaction between working memory and long-term memory is critical to understanding how people make use of memory in everyday tasks. Surprisingly, it remains unknown how having an existing long-term memory affects performance when you need to later remember the same item in a short-term memory task. Using behavioral and electrophysiology methods, we discover that under certain conditions long-term memories can ‘replace’ working memory representations, eliminating the need to hold items in working memory. These results not only elucidate the conditions under which working and long-term memory operate, but also suggest that under the realistic scenario of working with previously encountered items, our memory systems use existing long-term memories to free up working memory resources for use elsewhere.
Introduction
Visual working memory is an online system used to actively retain and manipulate information over brief periods (Baddeley & Hitch, 1974; Cowan, 2008; Ma, Husain & Bays, 2014). Its most notable characteristic is that it is capacity limited (Vogel, Woodman & Luck, 2001), with individual capacity differences strongly correlating with measures of broad cognitive function, including fluid intelligence and academic performance (Alloway & Alloway, 2010; Fukuda et al., 2013). By contrast, visual episodic long-term memory is the storage of visual information operating through the retrieval of memory traces over any time scale without continued active maintenance (Squire, 2004; Cowan, 2008; Brady, Konkle & Alvarez, 2011). A large body of work has demonstrated humans have remarkable visual long-term memory abilities (Shepard, 1967; Standing, 1973), capable of storing thousands of objects with significant detail (Brady, Konkle, Alvarez, Oliva, 2008).
Generally, visual working memory is studied using simple displays consisting of stimuli such as meaningless colored squares or line orientations to reduce any potential contributions from longterm memory. However, in the real-world visual working memory does not operate in isolation, and in our everyday life we rarely need to hold in mind semantically meaningless stimuli or stimuli that we have never previously encountered. Indeed, existing work has shown the important role of semantic meaning (Brady, Störmer & Alvarez, 2016) and expertise for visual working memory (Curby, Glazek & Gauthier, 2009). However, how existing episodic long-term memory traces for a specific item affect visual working memory is surprisingly unexplored. Imagine a friend asks you to pour some water and points to a red cup to indicate it is theirs. You need to hold in mind which cup is theirs while you fetch the water. But what if an hour before you encoded a memory that your friend’s cup was red? Does this existing episodic memory reduce or eliminate the need to hold this task-relevant information actively in mind, thus alleviating demands on the capacity-limited working memory system? Or when needing to act on information in the immediate future do people use both systems, reducing potential errors but at a cognitive cost of having to engage working memory even when an existing memory could serve to guide their performance? Most research on the relationship between working and longterm memory focuses on working memory as a passageway through which information must pass on the way to long-term memory (Cowan, 2008). However, existing evidence does point to the speed and accuracy of access to episodic long-term memories (e.g., Ericsson & Kintsch, 1995; Wolfe, 2012; Cunningham & Wolfe, 2014). Thus, it is possible that under some conditions participants could be able to rapidly access episodic memories efficiently enough to avoid having to engage working memory.
In the present study, we used electrophysiological recordings to test whether having a previous episodic memory of an item alters or eliminates the need for active maintenance for that same item in working memory. One particularly strong marker of active maintenance in working memory is the contralateral delay activity (CDA), a sustained posterior negativity contralateral to lateralized objects that are being remembered (Klaver et al. 1999; Vogel & Machizawa, 2004). CDA amplitude increases when more items are stored in working memory, and decreases immediately when items are dropped from working memory (Vogel et al., 2005). It’s correlated with behavior, such that it’s larger on correct than incorrect trials (McCollough et al., 2007). And critically, the CDA is not reflective of episodic long-term memory (Carlisle, Arita, Pardo & Woodman, 2011). Thus, the CDA provides a measure to assess whether and how episodic long-term memory might affect the storage of information in visual working memory, since it can be utilized to assess how much information is being held actively in mind. Across two experiments, we utilized the CDA activity to assess conditions under which episodic memory may be utilized in lieu of active maintenance in working memory.
Materials and Methods
Participants
A group of 23 Johns Hopkins University undergraduates participated in Experiment 1, and a separate group of 21 University of California, San Diego undergraduates participated in Experiment 2. The results from three participants in Experiment 1 and one participant in Experiment 2 were excluded due to technical errors and/or excessive artifacts in the EEG signal, leading to a final sample of 20 participants in each experiment. All participants reported normal or corrected-to-normal visual acuity. Participation was voluntary, and in exchange for extra credit in related courses. The experimental protocol was approved by the Johns Hopkins University IRB and the University of California, San Diego IRB.
Procedure
Experiment 1
Experiment 1 began with a long-term memory encoding task. Participants viewed 120 images of real-world objects, one after another for 2 seconds each (500 ms ISI), and were explicitly told to remember each image as best they could. After completing the task, approximately 30 minutes later participant then completed a lateralized visual working memory task.
Participants completed 240 total trials of the working memory task. On each trial, two categorically distinct objects were presented sequentially (500 ms each, 500 ms ISI) in each visual hemifield, followed by a delay interval (900 ms) and then a memory test. Participants were cued to remember the objects on the left or right of fixation and to ignore the other objects in order to allow measurement of the contralateral delay activity (Figure 1A). As a result, the visual display on both hemifields were equated for perceptual information so that any brain differences between the cued and uncued side during the delay interval were due to differences in holding the items in working memory. Objects on the to-be-ignored side of the display were drawn from a separate set of categorically distinct real-world objects from the same stimulus set, which were never shown during long-term memory encoding or on the to-be-remembered side of the working memory display.
(A) General design of sequential working memory task. To allow measurement of the contralateral delay activity, a neural marker of working memory, participants were cued to remember only the objects on either the left or right side of fixation. For half the trials, both study images presented had never been previously encountered (LTM-unavailable condition). For the other half of trials, one of the images had been seen previously in during an episodic long-term memory encoding session (LTM-available condition). Objects were presented sequentially for 500ms each, with a 500ms ISI. After a 900ms delay, a perceptual discrimination 2-AFC test assessed detailed object memory. (B) Behavioral results for Experiment 1. We found no difference in performance based on whether observers had a previous long-term memory representation available or not. Error bars represent within-subject standard error (29).
Results of Experiment 1. (A) Contralateral-minus-ipsilateral waveforms for the LTM Unavailable (black) and LTM Available (red) conditions. The CDA is measured from 300 ms after offset during the delay period (black rectangle, labeled CDA). We observed significantly reduced CDA amplitudes for when participants had a LTM representation available, compared to when they did not. (B) CDA amplitudes across all three conditions. We observed significantly reduced CDA activity for when participants encountered an LTM first, but not second, consistent with the gating hypothesis of working memory. Error bars reflect within-subject standard error.
Results of Experiment 2. When perceptual interference was high, we did observe a slight behavioral advantage for the LTM-first relative to LTM-second condition. This was primarily driven by better performance for when the first item was tested in LTM-first (when an LTM item was available) compared to the LTM unavailable condition. Critically, we failed to find any difference across conditions in active maintenance in working memory (as indexed by the CDA). * represents p = 0.001. Error bars represent within-subject standard error.
The cue indicating which side of the screen to remember was an arrow pointing either left or right that appeared at the center of the screen 1,000 ms before the presentation of the objects. Participants were instructed to keep their eyes in the center of the screen throughout each trial until the test display appeared. Trials with horizontal eye movements were excluded from the analysis. On half of the trials both of the to-be-remembered objects presented were new (i.e. never encountered during the previous long-term memory encoding session), which we term “LTM-unavailable” trials. These trials provided a baseline assessment of working memory performance in the task. For the other half of the trails, one of the objects encountered had been previously seen during the previous long-term memory encoding session, which we term “LTM-available” trials. Of these trials, half of the time the previous item was encountered as the first item in the trial sequence (“LTM-first”), whereas for the other half the previous item was encountered as the second item in the sequence (“LTM-second”).
After the 900-ms delay interval, participants were presented a perceptual discrimination forced-choice test that remained visible until participants made a response. This test consisted of one of the two previously seen items (counterbalanced across all conditions) and a second, similar-looking lure, with both items appearing above or below fixation on the to-be-remembered side. Participants were instructed to indicate which of the two objects had appeared in the previous display. Since the test included a previous item and a similar-looking lure, participants could not rely on gist, categorical, or semantic information in order to identify the correct item. Rather, visual details in memory were necessary at test.
Importantly, in this design, even if a participant recognized a previously encoded object during the working memory task, this was not indicative of which item they would be tested on. In addition, the sequential presentation design ensured that participants could not direct attention solely to the new item, as they would be able to do if the items were presented simultaneously. The sequential nature of the task also allowed us to examine potential distinction between gating and maintenance in working memory (Badre, 2012); in particular to ask whether once working memory was engaged if people tended to use it for all items, or whether each item was treated independently.
Experiment 2
Experiment 2 was designed to measure whether CDA differences between the LTM-unavailable and LTM-available conditions would persist when substantial perceptual interference was added to the working memory task. Experiment 2 was thus exactly the same as Experiment 1, with the following exceptions:
During each working memory trial, rather than seeing two categorically distinct objects, participants were presented two objects from the same category (e.g., two teddy bears) in each visual hemifield. As in Experiment 1, they were told to remember the objects on the left or right of fixation and to ignore the other objects. Objects on the to-be-ignored side of the display were randomly drawn from the same categories of the to-be-remembered side. As a result, the visual display on both hemifields were equated for perceptual information so that any brain differences between the cued and uncued side during the delay were due to differences in working memory, and not perceptual processing. At subsequent test, participants were presented a perceptual discrimination forced-choice test and remained visible until participants made a response. This test consisted of one of the two previously seen items (counterbalanced equally across all conditions) and a third, similar lure from the same category, with both items appearing above or below fixation on the to-be-remembered side.
Electrophysiological Recordings and Analysis
EEG was recorded continuously from 32 Ag/AgCL electrodes mounted in an elastic cap and amplified by an ActiCHamp amplifier (BrainVision). EEG data were sampled at 500 Hz. For Experiment 1 (collected at Johns Hopkins University), data were online referenced to Cz and later were offline referenced to the average of the right and left mastoid. For Experiment 2 (collected at University of California, San Diego), data were online referenced to the right mastoid and later were offline referenced to the average of the right and left mastoid. Eye movements were measured through the two frontal eye channels (FP1 and FP2).
Continuous EEG data were filtered offline with a band pass of 0.01–112 Hz. Trials with horizontal eye movements, blinks, or excessive muscle movements were excluded from the analysis. All EEG and ERP data analyses were conducted through EEGLAB (Delorme & Makeig, 2004) and ERPLAB (Lopez-Calderon & Luck, 2014) toolboxes for Matlab. Timing of the stimulus presentation and event codes were monitored and corrected using a photodiode.
ERPs were time-locked to the onset of the memory display in all experiments, and ERPs from artifact-free epochs were averaged and digitally low- pass-filtered (–3-dB cutoff at 25 Hz) separately for each subject.
ERPs elicited by the memory display were averaged separately for each condition and were then collapsed across to-be-remembered hemifield (left or right) and lateral position of the electrodes (left or right) to obtain waveforms recorded contralaterally and ipsilaterally to the to-be-remembered side. For each participant, mean CDA amplitudes were measured with respect to a 200-ms prestimulus period at four lateralized posterior electrodes (PO3/PO4/PO7/PO8), consistent with existing data on the location of the CDA (McCollough, Machizawa & Vogel, 2007).
For all experiments, the measurement window for the CDA started 300 ms after the offset of the memory display and lasted for 400 ms, until the cue indicating the memory test item. Thus, for both Experiments the CDA amplitude was measured between 1800-2200 ms with respect to the onset of the memory display. The resulting mean amplitudes were statistically compared using paired t-tests and ANOVAs.
Results
Experiment 1 Episodic Memory Replaces Active Maintenance in Working Memory
We first examined participants’ accuracy separately in the LTM-available and LTM-unavailable conditions. This allowed us to examine one aspect of how working memory and episodic longterm memory work together in this task: in particular, if participants engage working memory even in the LTM-available condition and make use of both sources of information (in working memory and long-term memory), they should be more accurate in the memory test in the LTM-available condition. Alternatively, if participants make use of their previous episodic memories for items when these are available, and these representations are utilized in lieu of active maintenance in working memory, performance in the LTM-available condition should be either slightly worse or the same as performance in the LTM-unavailable condition (depending on how strong participants episodic memories are and how adaptive to their own memory strength participants are in choosing whether to invoke working memory). Such a strategy would be beneficial, as it would free up cognitive resources, but it would not necessarily be reflected in any behavioral differences between conditions.
We found that across both conditions, accuracy was similar. In particular, participants averaged 81.2% correct (SEM = 1.6%) in the LTM-unavailable condition and 82.4% correct (SEM = 1.4%) in the LTM-available condition, which were not significantly different from one another, t(19) = 0.92, p = 0.37, Cohen’s dz = 0.20. Furthermore, we found no effect of order in the LTM-available condition. Participants averaged 81.2% correct (SEM = 1.7%) in the LTM-first condition and 83.6% correct (SEM = 1.5%) in the LTM-second condition, t(19) = 1.46, p = 0.16, Cohen’s dz = 0.32. Additionally, in all the LTM-available conditions we found no performance difference for whether the old or new item was tested (all p’s > 0.2). Thus, having previously encoded objects in episodic long-term memory didn’t offer any advantage or disadvantage in performance over maintaining new objects in working memory. This is consistent with either participants not utilizing long-term memory at all in the task, or with the idea that participants are strategically making use of active maintenance in working memory only when they do not have a strong episodic memory available. However, this does provide evidence that observers do not use the two kinds of representations additively.
We next assessed CDA activity, our neural marker of how much information was being actively maintained in working memory. Due to the short time period between the two sequentially presented stimuli and the predictable timing (and thus possibility of preparatory activity), we focused only on the longer 900ms delay interval after the second object was presented and before the test (We used sequential presentations largely to prevent people from distributing attention to only the new items). First, we compared trials containing only new images, our working memory baseline (LTM-unavailable), to trials containing a previously seen episodic image (LTM-available). Consistent with the hypothesis that episodic long-term memory was being used to substitute for active maintenance in working memory, we found that the CDA amplitude was lower when participants encountered a previously seen object (M = 1.20, SEM = 0.21) compared to when both images were new (M = 1.63, SEM = 0.19), t(19) = 2.17, p = 0.04. Combined with the lack of behavioral advantage for the LTM-available condition, this provides evidence that participants used episodic long-term memories in lieu of holding items active in working memory. In particular, participants seem to be using episodic memory to substitute for or replace active maintenance in working memory when they encounter a familiar item.
The design of our experiment, with sequential presentations and thus LTM-first and LTM-second trials, also allows us to examine the role of “gating” in the engagement or lack of engagement of visual working memory. A significant literature has suggested that working memory operates via separate gate and maintenance mechanisms: when the gate is open available information can enter working memory, and when the gate is closed the contents of working memory are sustained while keeping irrelevant information out (Miller & Cohen, 2001; Raghavachari et al., 2001; Badre, 2012; O’Reilly & Frank, 2006). This suggests that engaging visual working memory is a distinct process from maintaining additional information in this memory system once it is engaged. One prediction of this account is that when the first item participants see is ‘new’ (not available in episodic memory), and thus participants must engage visual working memory to maintain it, they may be more likely to hold the second item in working memory even if it is already available in episodic long-term memory. By contrast, if the first item is available in episodic memory, working memory may not be engaged at all until the second item is shown.
To address this, we separately analyzed the CDA in the LTM-first and LTM-second conditions and compared them to the LTM-unavailable condition. We entered all three conditions into a repeated-measures ANOVA and observed a significant effect, F(2, 38) = 4.27, p = 0.02, ηp2 = 0.18. Follow up analyses revealed no difference in CDA amplitude between the LTM-unavailable (M = 1.63, SEM = 0.19) and LTM-second (M = 1.58, SEM = 0.30) conditions, t(19) = 0.84, p = 0.21, but a diminished CDA amplitude in the LTM-first condition (M =0.82, SEM = 0.25), which was significantly different compared to the LTM unavailable condition t(19) = 2.71, p = 0.01. Thus, when participants are initially shown an old image during a visual working memory task, followed by a new item, they have the least engagement of visual working memory resources, consistent with the gating hypothesis; whereas when they must engage working memory for the first item if it is new, there is little distinction in working memory activity based on whether the following item is old or new. These effects were not explained by potential differences in attention during the encoding phase, as analyses of P1 and N2PC components (two well-known neural markers of attention; Woodman & Luck, 1999; Moher et al., 2014) found no significant differences across conditions (all p’s > 0.10).
Broadly, these results suggest that working memory, as indexed by CDA activity, is engaged once new information is encountered. However, if a participant first encounters an object previously encoded in episodic memory, they maintain significantly less in working memory compared to being shown two completely new objects, suggesting they maintain less information about the previously-encoded object in working memory and instead rely on episodic long-term memory. Consistent with this, we find that despite maintaining significantly less information in working memory as indexed by the CDA, participants do not demonstrate any reduction in behavioral performance in this condition. Overall, the results of Experiment 1 suggest a surprisingly adaptive use of visual working memory only when required, and that participants use episodic long-term memory in lieu of working memory when it is available. Our results also provide strong evidence in favor of the ‘gating’ hypothesis -- e.g., that engaging working memory may be a distinct process from putting additional information into visual working memory.
Experiment 2: Working Memory Systems Engage When Perceptual Interference is High
The data from Experiment 1 indicates that participants can dynamically avoid engaging working memory when episodic memory is available, without any detriment to performance. Given these results, what then is the purpose of visual working memory: if episodic memory can be utilized without any apparent costs and with considerably less cognitive demand and capacity limitations, why have a separate, effortful working memory system?
Many researchers have claimed that the main reason we require separate working memory systems is that working memory is considerably less susceptible to inference. For example, consistent with the work of Ericsson and Kintsch (1995), Engle (2002) argued that dealing with interference is the primary function of working memory: “Without the effects of interference, most of the information people know and need to function in the world could be retrieved from long-term memory sufficiently quickly and accurately for them to perform even complex cognitive functions quite well” (Engle, 2002). Thus, the results of our first experiment, where participants quickly and accurately make use of episodic long-term memory instead of engaging visual working memory, may depend crucially on the level of interference participants expect to encounter. To investigate this, in Experiment 2 we introduced substantial perceptual interference into our working memory task by asking participants to remember two objects in memory that were of the same category. Participants then had to make a detailed perceptual comparison involving an object from the previous working memory display and a third similar-looking object.
Consistent with Experiment 1, we again found that across both main conditions accuracy was similar. In particular, participants averaged 88.2% correct (SEM = 0.8%) in the LTM-unavailable condition and 88.6% correct (SEM = 1.0%) in the LTM-available condition, which were not significantly different from one another, t(19) = 0.92, p = 0.37, Cohen’s dz = 0.12. However, we did observe an effect of order in the LTM-available condition. Participants averaged 90.9% correct (SEM = 1.0%) in the LTM-first condition and 86.3% correct (SEM = 1.4%) in the LTM-second condition, t(19) = 3.84, p = 0.001, Cohen’s dz = 0.85. The significant effect was primarily driven by better performance for when the first item was tested in the LTM-first compared to LTM-unavailable condition (M = 90.7% vs 85.8%, respectively), t(19) = 2.76, p = 0.01, Cohen’s dz = 0.62. We failed to find any other significant differences for all remaining contrasts (all p’s>0.20). Thus, it appears that in the LTM-first condition, observers were able to use previously encountered memories for objects to gain a slight performance advantage under very specific conditions (i.e. only for the first item).
We were primarily interested in whether we would observe any differences in CDA activity across conditions when perceptual interference was high. First, we compared trials containing only new images (LTM-unavailable) to trials containing a previously seen episodic image (LTM-available). CDA amplitude was not significantly different according to whether one object had been previously encountered in episodic memory (M = 1.91, SEM = 0.30) compared to when both images were new (M = 1.72, SEM = 0.35), t(19) = 0.95, p = 0.35. Next, all three conditions were entered into a repeated-measures ANOVA. We failed to observe any significant differences, F(2, 38) = 0.54, p = 0.59, ηp2 = 0.03. Therefore, CDA amplitude was not statistically distinguishable between the LTM-unavailable, LTM-second (M = 1.86, SEM = 0.32) and LTM-first (M = 1.97, SEM = 0.33) conditions (p>0.20). This suggests that under conditions where participants expected to encounter sufficient interference to decrease the utility of their episodic long-term memories (Konkle et al. 2010), participants actively maintained the same amount of information in all conditions, regardless of whether or not they had a previous episodic memory representation available. Thus, during conditions of low expected interference (Exp. 1) participants quickly and easily replaced maintenance in visual working memory with access to long-term memory, whereas under conditions of high interference (Exp. 2) they engaged working memory regardless of whether they had long-term memories available. This provides evidence consistent with the idea that one of the main purposes of working memory broadly, and visual working memory specifically, is to provide a memory system that is robust to interference.
Discussion
The present results show that under conditions of low interference, having episodic long-term memory representations available significantly reduces participants’ use of active maintenance in visual working memory. Specifically, we find that working memory is engaged once new information is encountered or when perceptual interference is high. However, when interference is low and participants have a previous episodic long-term memory to rely on, this existing episodic memory can be utilized in a short-term memory test instead of engaging working memory systems. Critically, these episodic long-term memory representations contain enough visual details (see Brady et al., 2008) to be used without impacting performance, even when making fine perceptual discriminations at test. Thus, the present data reveals how episodic longterm memory representations may be used to support memory over brief durations. Overall, this suggests a more intimate and dynamic interaction between working memory and long-term memory than is typically considered, with episodic memory substituting for working memory in a setting that is typical of a real-world task (e.g., where items are not novel and meaningless). The limited capacity of visual working memory may therefore be less of a constraint on performance than is typically assumed, if when performing real-world tasks we are often able to make use of our impressive visual long-term memory capacities rather than holding items actively in visual working memory.
Possible mechanisms supporting dynamic utilization of long-term memory representations
Initially, the flexibility and speed with which long-term memory representations can be utilized in a short-term memory task appears quite surprising. For example, in the LTM-first condition, there must be a process that allows the image to be recognized quickly enough that working memory for that item is not engaged. What are the potential mechanisms that may facilitate this process?
There is significant evidence, via Ericsson and Kintsch (1995), that participants are able to quickly and effectively engage long-term memory systems. This may be especially true when all that is required is familiarity (that is, no source memory is required). Consistent with this idea that after encoding dozens of images into episodic long-term memory participants may have fast and efficient access to whether a given image is familiar or not, research into hybrid visual search has shown participants can hold hundreds of potential targets in episodic long-term memory and efficiently search for any of these items (Wolfe, 2012; Cunningham & Wolfe, 2014).
Previous research has demonstrated familiarity signals for items occur quite rapidly (within 100 ms), much faster than recollection signals required for source memory (at least 500 ms) (Hintzman, Caulton & Levitin, 1998). Similarly, novelty signals -- which may be used in order to assess whether an item may require the engagement of working memory -- are also extremely fast. For example, previous research in monkeys has shown visual novelty can trigger neuronal population firing 70-80 ms after stimulus onset (Li, Miller & Desimone, 1993; Xiang & Brown, 1998; Brown, 2009). In the context of the present task, during a trial of the working memory task, familiarity or novelty is all that is required for an observer to know they can rely on a longterm memory representation during the subsequent test. Thus, even our brief encoding times for individual images (500 ms) are more than enough time for a familiarity or novelty signal to prevent working memory from engaging.
Importantly, this fast access to items in episodic long-term memory may not occur for all previously encoded items, but may require those memories to be held in a special state that allows for fast access, sometimes referred to as activated long-term memory (e.g., Cowan, 2008). In the hybrid search literature, there is some evidence that items that are expected to be used in the task are more accessible or active than other items (e.g., Boettcher, Drew, & Wolfe, 2013). The extent to which items need to be held in an activated long-term memory state to replace working memories remains an open question.
Support for gating hypothesis of working memory
In Experiment 1, we observed that CDA amplitude in the LTM-second condition was analogous to activity observed during LTM-unavailable trials. This is consistent with the “gating” hypothesis of working memory. This theory suggests that in order to balance flexibility and stability, working memory operates via separate gate and maintenance mechanisms: when the gate is open available information can enter working memory, and when the gate is closed the contents of working memory are sustained while keeping irrelevant information out (Miller, Cohen, 2001; Raghavachari et al., 2001; Badre, 2012; O’Reilly & Frank, 2006). Separating maintenance and gating into distinct mechanisms has been shown to be computationally efficient (Hochreiter & Schmidhuber, 1997) and remains an assumption in several prominent and influential working memory models (Braver & Cohen, 2000; O’Reilly & Frank, 2006).
Thus, in the context of the present studies, when the first item presented in a working memory display was one encoded previously in episodic long-term memory, observers were able to utilize long-term memory representations as the working memory gate had not been opened. But when the first item encountered was new and had not been previously encoded, the working memory gate was opened to maintain subsequently encountered information. As a result, in the LTM-second condition, even though observers had a long-term memory available for the second item encountered, they encoded this information into working memory.
Conclusions
Our data illuminate the conditions under which episodic memory and working memory operate. Working memory is engaged when new, previously unstudied information is encountered or when perceptual interference is high. In contrast, when old information is present or perceptual interference is low, episodic memory is utilized in lieu of working memory. These results further our understanding of how and when hand-offs occur between episodic memory and working memory systems. Moreover, our findings demonstrate not only how these systems interact with one another, but also suggest the relationship between working memory and long-term memory is more dynamic and fluid than previously thought.
Acknowledgements
Some of the results were presented at the annual meeting on Object Perception, Attention and Memory (OPAM) in 2017 and the annual meeting of the Vision Sciences Society (VSS) in 2018. For helpful feedback on this work, we would like to thank Viola Störmer and John Wixted.
