Eye movements in real-life search are guided by task-irrelevant working-memory content

Attention is automatically guided towards stimuli that match the contents of working memory. This has been studied extensively using simplified computer tasks, but it has never been investigated whether (yet often assumed that) memory-driven guidance also affects real-life search. Here we tested this open question in a naturalistic environment that closely resembles real life. In two experiments, participants wore a mobile eye-tracker, and memorized a color, prior to a search task in which they looked for a target word among book covers on a bookshelf. The memory color was irrelevant to the search task. Nevertheless, we found that participants’ gaze was strongly guided towards book covers that matched the memory color. Crucially, this memory-driven guidance was evident from the very start of the search period. These findings support that attention is guided towards working-memory content in real-world search, and that this is fast and therefore likely reflecting an automatic process. Significance statement A core concept in the field of visual working memory (VWM) is that visual attention is automatically guided towards things that resemble the content of VWM. For example, if you hold the color red in VWM, your attention and gaze would automatically be drawn towards red things in the environment. So far, studies on such memory-driven guidance have only been done with well-controlled computer tasks that used simplified search displays. Here we address the crucial and open question of whether attention is guided by the content of VWM in a naturalistic environment that closely resembles real life. To do so, we conducted two experiments with mobile eye tracking. Crucially, we found strong memory-driven guidance from the very early phase of the search, reflecting that this is a fast, and therefore likely automatic, process that also driven visual search in real life.

in a naturalistic environment that closely resembles real life. To do so, we conducted two 46 experiments with mobile eye tracking. Crucially, we found strong memory-driven guidance 47 from the very early phase of the search, reflecting that this is a fast, and therefore likely 48 automatic, process that also driven visual search in real life. 4 Searching is a complex task. When you are looking for an object in your environment, 50 you generally keep a prototype of the object in working memory; this prototype can be defined 51 by different stimulus attributes (e.g., shape, color, size, name; Wolfe, 1994). The visual system 52 efficiently directs our attention towards objects that resemble the features stored in working 53 memory, a phenomenon referred to as memory-driven guidance of attention (Bundesen, 1990).  Memory-driven guidance has been extensively investigated in lab studies. For example, 61 in a combined memory-search paradigm (Olivers, Meijer, & Theeuwes, 2006), participants 62 were first asked to keep a color in memory. During the retention interval, they performed a 63 visual search task in which the search display consisted of: a target; a singleton distractor that 64 matched or did not match the memory color; and several gray distractors. Finally, in a memory 65 test, participants indicated the memorized color among three colored disks. Importantly, Olivers 66 and colleagues found that the interference of the singleton distractor, as measured through 67 increased response times (RTs) to the target, was especially high when its color matched the 68 memory color maintained in working memory. 69 Jung and colleagues (2018) used real-world images in a combined memory-search task.

70
Their task was similar to the one used by Olivers et al. (2006), but more closely resembled real-71 world search. (Although it was still a computer task.) Participants were first asked to keep an 72 object in memory (e.g., a coffee cup). Next, they searched for a target in indoor-scenes images 73 (e.g., a photo of a room). In line with previous findings, Jung and colleagues found that search  ). Yet the assumption is that memory-driven guidance is a fundamental mechanism that 79 allows us to search for things in real life. Can we still find robust memory-driven guidance  In the present study, we created a naturalistic search setting that resembles searches in 86 daily life far more closely than has so far been done in previous studies. Participants wore a 87 mobile eye-tracker, and looked for a target word among book covers while maintaining a color 88 in working memory. As a search array, we used real bookshelves with book covers in four 89 different color categories: Blue, Yellow, Red, and Green. We predicted that, if attention is 90 guided towards the color that is maintained in VWM, participants would look more often at 91 books that match the memory color, even though color is irrelevant to the search task. Moreover, 92 if the color of the book cover that contains the target word matches the memory color, the search 93 RT should be faster. Consistent with our prediction, the eye-tracking data for two experiments 94 demonstrated a very strong guidance effect in this real-world search task. There was also a 95 tendency for a memory-driven guidance effect in RTs.

98
A detailed pre-registration of Experiment 2 is available at https://osf.io/nxbzh. All 99 deviations from the preregistration will be mentioned below.       If more than 20% gaze points or video frames (due to technical malfunction) were lost, 166 or if 20% of gaze points deviated so much from a surface that they could not be manually 167 corrected, the trial was excluded from the eye-tracking analysis. (This is slightly different from 168 the preregistration, in which we did not specify the proportion of lost data, and did not include  Eye-tracking analysis 180 We conducted linear mixed effects models (LMER) using the R package lmerTest using  Behavioral analysis 190 We conducted a LMER using Reaction Time as dependent measure, Target Match 191 (Target-match, Target-non-match) as fixed effect, with a random by-participant intercept and a 192 random by-participant slope for the effect of Target.

194
As shown in Figure 2, when participants maintained a color in working memory, they

214
The present study shows that visual working memory guides attention in real-world 215 visual search. In two experiments, participants wore an eye-tracker while performing a 216 combined memory-search task in a naturalistic setting. First, participants memorized a color 217 shade that they had to report later. Next, they searched for a target word among book covers 218 that matched or did not match the color in working memory. We found that participants' gaze 219 was guided towards the book covers that matched the memory color during visual search, as 220 compared to the covers that did not match the memory color. Moreover, this effect was present 221 from the very beginning of the search, remained constant over time, and occurred despite the 222 fact that the memory color was irrelevant for the search task.