The neural mechanisms of active removal from working memory

The ability to frequently update the contents working memory (WM) is vital for the flexible control of behavior. Whether there even exists a mechanism for the active removal of information from working memory, however, remains poorly understood. In this Registered Report we will test the predictions of models for two different (and not mutually exclusive) mechanisms of active removal: adaptation-hijacking and mental-context shifting. We will collect functional magnetic resonance imaging (fMRI) data while subjects perform a novel “ABC-retrocuing” task designed to elicit two modes of removal, active or passive (Shan & Postle, Registered Report). The adaptation-hijacking model posits an adaptation-like modification of perceptual circuits combined with a weak activation of the to-be-removed item. Its predictions will be assessed by using multivariate inverted encoding modeling (IEM) and photic “pings” to assay the state of feature-selective encoding channels and of activity-silent representations under active-removal versus passive-removal conditions. A second model – “working memory episodic memory” (WMEM) – posits that interference from no-longer-relevant information is minimized by making the mental context associated with new information dissimilar from that associated with the to-be-“removed” information. This will be tested by using representational similarity analysis (RSA) to compare the rate of contextual drift under active-removal versus passive-removal conditions.

electroencephalography (EEG) study of delayed recall of orientation, Bae and Luck (2019) were able to decode the 48 orientation of the previous trial's sample after the onset of the current trial's sample. For delayed recall of location, Barbosa,49 Stein et al. (2020) were able to decode the previous trial's sample location (from activity in the PFC of nonhuman primates) 50 from late in the intertrial interval (ITI), just prior to the start of the next trial. A similar pattern of reactivation was also 51 observed in whole-scalp EEG in humans. Simulations with a bump-attractor network model suggested that the reactivation 52 of no-longer relevant information may be due to "nonspecific" activation of a residual neural trace that is "imprinted in 53 neuronal synapses as a latent activity-silent trace" (Barbosa, Stein, et al., 2020). Tellingly, this model did not include an 54 explicit mechanism for removal of no-longer-relevant information; rather, when an item was no longer relevant, activation 55 was simply withdrawn from it, and the bump of activity representing it receded to baseline. Similarly, another formal 56 model of WM performance, the Prefrontal Basal Ganglia Working Memory (PBWM) model, accomplishes the replacement 57 of a no-longer-relevant item with a new one via the "reallocation" of resources away from the former (Chatham and Badre,58 2013). Thus, there is considerable evidence that a default strategy for updating the contents of WM is what we will refer 59 to as the "passive removal" of no-longer-relevant information. 60 In contrast to passive removal, there is also considerable evidence for an active removal mechanism, particularly 61 6 no-overlap conditions. In the no-overlap condition, item B had an attractive serial bias on 1-item recall, consistent with the 106 attractive serial bias observed in several previous studies that we assume were characterized by passive removal of no-107 longer-relevant stimulus information (e.g., Bliss et al., 2017;Fischer & Whitney, 2014;Fritsche et al. 2020). In the overlap 108 condition, in contrast, item B had a repulsive serial bias on 1-item recall, suggesting that it was processed in a very different 109 way. The interpretation that the critical difference between the two conditions was active versus passive removal of the 110 IMI was reinforced by the fact that in both conditions item A exerted comparable levels attractive serial bias on 1-item 111 recall. 112

Do common mechanisms underlie repulsive serial dependence and active removal from WM? 113
Recent work from three independent groups has converged on the view that serial bias effects arise from opposing 114 effects that play out at two levels of processing: at the level of perception, adaptation to recent perceptual events produces 115 repulsion from previous stimuli; at the level of decision making, in contrast, perceptual decisions are attracted toward 116 previous decisions (Pascucci et al. 2019; Fritsche et al. 2020). Although decisional biases are stronger, explaining why short-117 lag serial bias is typically attractive, perceptual adaptation is longer lasting, explaining why longer-lag serial bias (e.g., the 118 influence of the item from five trials previous) can be repulsive (Fritsche et al., 2020). That perceptual adaptation-like 119 effects might also be involved in processing stimuli during WM comes from a simulation of fMRI results from a retrocuing 120 task (Lorenc, Vandenbroucke et al., 2020). The observed effect of a retrocue on the multivariate inverted encoding model 121 (IEM) reconstruction of the IMI was to "flip" it (c.f., Sahan et al., 2020), and this effect was best modeled by an adaptation- Registered Report) literatures has given rise to the idea that is the motivation for this Registered Report: The active removal 126 of information from WM may be implemented via a top-down "hijacking" of an adaptation-like modification of perceptual 7 circuits. More specifically, removal may be accomplished by the co-occurrence of two events. The first is the adaptation-128 like modulation of the gain of the perceptual channels that were engaged by the encoding of the IMI. This putative 129 operation is "adaptation-like" because it is triggered by the onset of the retrocue (not by the perceptual processing of the 130 to-be removed item) and because it is greater in magnitude than true adaptation (the repulsive effects of true adaptation 131 are typically weaker than the attractive influence of recent decisions). The coincident second event is the brief, weak 132 activation of the IMI. Although seemingly paradoxical, the weak activation of the to-be-removed item is predicted by the 133 simulation from Lorenc et al. (2020) and is consistent with other computational accounts of forgetting (e.g., Kim et al.,134 2020). 135 We plan to assess this "hijacked adaptation" idea by collecting fMRI data while subjects perform an ABC-retrocuing 136 task ( Figure 1) while high-contrast task-irrelevant visual stimuli are flashed to "ping" the visual system, so as to assay 137 predicted consequences of this hypothesized mechanism for active removal. Here we will provide a narrative overview of 138 the logic of this experiment, to provide context for the statement of Preregistered Hypotheses. Details follow in the 139

Methods. 140
Based on Shan & Postle (Registered Report), we assume that the IMI undergoes active removal in the overlap 141 condition of the ABC-retrocuing task, but passive removal in the no-overlap condition. As diagrammed in Figure 2, in the 142 overlap condition, we predict that the hypothesized adaptation-hijacking operation will produce a phasic "flipping" of the 143 active representation of the IMI (operationalized as an IEM reconstruction of the IMI with a negative slope) during the first 144 several seconds following the retrocue (c.f., Lorenc et al., 2020), followed by a disappearance of a detectable active trace 145 (i.e., an IEM reconstruction slope not different from 0). This will correspond to successful removal of the IMI. A longer-146 lasting consequence of active removal, however, will be the residual adaptation-like change to the perceptual feature 147 channels that correspond to the orientation of the IMI. This will be revealed in the filtering of the ping-evoked response, 148 which will also produce a transient flipped IEM reconstruction of the IMI. Note that the delay period after the retrocue 149 8 and before the ping will be relatively long (15.25s), so as to be able to dissociate the endogenously generated flipped 150 reconstruction of the IMI that is triggered by the retrocue from the flipped reconstruction predicted to be evoked by the 151 ping. In the no-overlap condition, we predict that the withdrawal of attention will result in the disappearance of evidence 152 for an active representation of the IMI during the first several seconds following the retrocue. However, because the 153 activity-silent trace of the IMI will not have been removed, the ping-evoked response will produce a conventional (i.e., not 154 flipped) IEM reconstruction of the IMI (c.f., Barbosa, Stein, et al., 2020). 155 The pattern of results summarized in Figure 2 (formalized in the statement of Preregistered hypotheses in Methods) 156 will provide neural evidence that the active removal of information from WM can be accomplished via a mechanism of 157 adaptation-hijacking. It will also provide evidence relevant for accounts of the repulsive serial bias that is sometimes Report). Importantly, our design will also allow us to assess evidence for alternative accounts of active removal, and so the no-overlap condition. This would suggest that there may not exist a mechanistic difference between what we have 166 characterized as "active" versus "passive" removal from WM, and that the two only differ in terms of the magnitude of 167 their effects. Finally, independent of the outcomes of stimulus-related analyses, we will assess evidence for the WMEM 168 model's prediction of larger mental context shifts on overlap versus no-overlap trials by assessing pattern similarity 169 between Delay 1 and the late portion of Delay 2.1, in early visual cortex and in the medial temporal lobe (MTL; Figure 1). 170 In the overlap condition, WMEM predicts a larger shift of mental context such that this discrepancy between mental 9 contexts can compensate for the elevated level of cue competition. trial, each sample item is represented during Delay 1 as elevated activity in the orientation channels corresponding to their 180 value. During Delay 1.2, the representation of A remains elevated but that of B drops to baseline. Importantly, the activity-181 silent representation of B remains (not shown). Early in Delay 2.2, although the ping nonspecifically raises the activity level 182 in every orientation channel, it also produces a reactivation of B (due to the persistence of B's activity-silent representation). 183 In the overlap trial, the cuing of A prompts a decrease in the gain of the channels corresponding to B plus a weak phasic 184 activation of these channels, and the effect during Delay 1.2 is two-fold: a "suppressed" activity-based representation of B 185 and the removal of the activity-silent representation of B (not shown). Later during Delay 2.1, the modified gain field 186 11 persists but is not evident with only baseline levels of activity. Finally, early in Delay 2.2, responses to the ping reveal the 187 modified gain field. 188 189 Methods 190

Preregistered hypotheses 191
We propose to test 7 primary hypotheses in this Registered Report: 192 Hypothesis 1a: In the overlap condition, the reconstruction of the orientation of the IMI at TR 7, with an IEM trained on 193 the retrocued item at TR 7, will have a significantly negative slope. 194 Hypothesis 1b: In the no-overlap condition, the reconstruction of the orientation of the IMI at TR 7, with an IEM trained 195 on the retrocued item at TR 7, will be unsuccessful (i.e., slope not different from 0). 196 Hypothesis 1c: The slopes from 1a and 1b will differ. item at TR 12, the reconstruction of the orientation of the IMI at TR 12 with an IEM trained on the retrocued item at TR 7 203 will be unsuccessful (i.e., slope not different from 0). 204 Hypothesis 2b: In the no-overlap condition, if an IEM can be successfully trained to reconstruct the retrocued item at TR 205 12, the reconstruction of the orientation of the IMI at TR 12, with that same IEM will be unsuccessful (i.e., slope not 206 different from 0). 207 12 Hypothesis 2b' (if needed): In the no-overlap condition, if an IEM cannot be successfully trained to reconstruct the 208 retrocued item at TR 12, the reconstruction of the orientation of the IMI at TR 12 with an IEM trained on the retrocued 209 item at TR 7 will be unsuccessful (i.e., slope not different from 0 Hypothesis 4a: In the in the early visual ROI the correlation coefficient between the high-dimensional activity pattern at 227 TR 4 and the activity pattern at TR 12 will be higher in the no-overlap condition than in the overlap condition. 228 13 Hypothesis 4b: In the in the MTL ROI the correlation coefficient between the high-dimensional activity pattern at TR 4 229 and the activity pattern at TR 12 will be higher in the no-overlap condition than in the overlap condition. This is an alternative way to assess 3b, in the event that an IEM can NOT be successfully trained to reconstruct the retrocued item at TRs 14+15; interpretation of outcomes is the same as 3b 3c (HA) Bootstrapping 3a and 3b differ = consistent with qualitative difference between active and passive removal; no statistical difference suggests no mechanistic difference between "active" and "passive" (e.g., perhaps a quantitative but not qualitative difference)

3c' (HA) Bootstrapping
This is an alternative way to assess 3c, in the event that either 3a' or 3b' were needed 30 subjects who are 18-35 years in age with normal or corrected-to-normal vision and report no history of 237 neurological disease will be recruited from the University of Wisconsin-Madison community. Informed consent will be 238 obtained. All experimental procedures for the Registered Report have been approved by the University of Wisconsin-239 Madison Health Sciences Institutional Review Board. 240 16 Power analysis. Using data from Yu, Teng, and Postle (2020), in which a negative slope of the IEM reconstruction of 241 the UMI in a DSR-of-orientation task has been observed, power analysis of the 2-tailed one sample t-test shows we will 242 need data from 30 subjects to achieve 90% power to detect a significantly negative slope for the reconstruction of 243 orientation of the UMI (Cohen's d = 0.617), and data from 26 subjects to detect a significantly positive slope for the 244 reconstruction of orientation of the PMI (Cohen's d =0.675). 245 To the best of our knowledge, there is no established way to perform power analysis for bootstrapping, which we 246 will use in the current study to test for the predicted positive and negative slopes of reconstructions. We used data from 247 Yu, Teng, and Postle (2020) to simulate the p-values obtained from t-tests versus from bootstrapping with different 248 sample sizes. Because this sample had data from 13 subjects, we generated estimates ranging from N = 8 to N =12, by 249 randomly drawing N subjects from the sample, without replacement, and conducting a t-test and a bootstrap analysis on 250 these data. For the t-tests, we collapsed over channel responses on both sides of the target channel, averaged them, and 251 calculated the slope of the averaged UMI reconstruction of each subject with linear regression. The slopes were then 252 compared to 0 with a 2-tailed one sample t-test. For bootstrapping, the method was the same as specified in the 253 Statistical Analyses subsection of fMRI Analyses section of the Methods. This process was repeated 10 times at each N to 254 get 10 (different) sets of subjects and 10 p-values for each test. For N=13, one p-value was obtained from each test. 255 Across sample sizes, the bootstrapping was generally more sensitive than the t-test (Figure 3). Based on this, we reason 256 that the sample size estimated by the power analysis for a t-test provides a conservative estimation of the sample size 257 required in the current study (because we will be using the more sensitive bootstrapping procedure). In the current 258 study we will use a sample size of 30 subjects. The stimulus presentation and response collection will be implemented with MATLAB (MathWorks, Natick, MA, 267 USA) with the Psychtoolbox-3 extensions (Brainard, 1997;Pelli, 1997). The display will be projected into the scanner and 268 onto a mirror mounted on the head coil at 60-Hz (Avotec Silent Vision 6011 projector; Avotec, Stuart, FL, USA). The 269 viewing distance will be roughly 69 cm and the screen width will be 33.02 cm. The sample stimuli will be grayscale 270 sinusoidal gratings (radius = 5; contrast = 0.6; spatial frequency = 0.5 cycles/; random phase angle) presented on gray 271 background (L= 52, a = 0, and b = 0 in CIEL*ab space). There will be six possible sample orientations: 20°, 50°, 80°, 110°, 272 140°, 170°; with a random jitter of ±0°-3° added with each presentation. These and all ensuing stimuli will appear at 273 any of six possible locations on an imaginary circle centered on fixation (radius of 8°, locations centered at each of these 274 18 polar angle:s 30°, 90°, 150°, 210°, 270°, 330°). The retrocue will be a white circle (thickness=0.08°) with the same radius 275 as the sample stimuli. Ping stimuli will be high contrast concentric circles with the same radius and spatial frequency as 276 the gratings (contrast=1). The response dial will be a black bar (thickness=0.08°) corresponding to the diameter of a black 277 circle with the same radius as the gratings (thickness=0.08°). Subjects will be instructed to adjust the orientation of the 278 bar using an MR-compatible trackball (Current Designs, Philadelphia, PA, USA) and to report their response by pressing a 279 button on the trackball when the orientation of the bar matches their memory for the probed sample. A white fixation 280 dot will be present throughout each block (i.e., also during the ITI). 281 Each trial of ABC retrocuing will start with the simultaneous presentation of two samples (A and B; 1 s) followed by 282 Delay 1 (7 s). Next the retrocue will appear for 0.75 s at the location that had been occupied by either A or B, thereby 283 designating a PMI (which might be tested at the end of the trial) and, by implication, the IMI (no longer relevant for that 284 trial). The retrocue will be followed by Delay 2.1 (15.25 s), which will be followed by the simultaneous presentation (0.25 285 s) of ping stimuli at each of the six locations, then Delay 2.2 (7.75 s), then sample item C (1 s), then Delay 3 (1 s). Finally, 286 the response dial will appear at the location that had been occupied by the retrocued item or by item C, prompting the 287 recall of the orientation of that item (4-s response window). The inter-trial interval ITI will vary randomly between 6, 8, 288 and 10 s. 289 On each trial the orientation of items A and B will be randomly selected, with replacement, from the pool of six 290 possible values. The locations of item A and B will be randomly selected from the six possible locations. To fully cross the 291 orientations of item A and B, 21 unique trials are required. 252 trials (12 repetitions per unique trial) will be used for 292 each condition. The retrocuing of A or B will be randomly determined on every trial. The orientation of item C will be 293 randomly selected from the pool of six possible values (i.e., independent of A and B), and its location will depend on 294 condition: in the overlap condition it will appear at the location that had been occupied by the uncued item; in the no-295 19 overlap condition it will appear in a location randomly selected from the four that had not been occupied by A or B. The 296 retrocued item or the item C will be probed for recall equiprobably. 297 Trials will be blocked by condition (overlap, no-overlap condition), and subjects will be explicitly informed of the 298 condition before the start of each block. Each subject will participate in 4 scanning sessions. The first scanning session 299 will consist of 6 runs, each run corresponding to a 14-trial block. The three remaining scanning sessions will each consist 300 of 10 runs (each run corresponding to a 14-trial block). There are fewer runs in the first session due to acquisition of 301 structural images. To facilitate the consistent use of active removal and passive removal, within each session the first 3 302 blocks (for the first session) or 5 blocks (for the last three sessions) will be of one condition and the remaining blocks will 303 be of the other condition. The order of conditions within a session will be counterbalanced across sessions and across 304 subjects. In the first session, each subject will first do two practice blocks (one block for each condition) outside the 305 scanner and another practice block (with the same condition as the first real block) inside the scanner. An Avotec RE-306 5700 eye-tracking system (Avotec) will be used to track eye position throughout each scanning session, and to assure 307 that subjects' eyes are open during the ping. 308 309 Behavioral Data Analysis 310