Episodic memory formation in unrestricted viewing

The brain systems of episodic memory and oculomotor control are tightly linked, suggesting a crucial role of eye movements in memory. But little is known about the neural mechanisms of memory formation across eye movements in unrestricted viewing behavior. Here, we leverage simultaneous eye tracking and EEG recording to examine episodic memory formation in free viewing. Participants memorized multi-element events while their EEG and eye movements were concurrently recorded. Each event comprised elements from three categories (face, object, place), with two exemplars from each category, in different locations on the screen. A subsequent associative memory test assessed participants’ memory for the between-category associations that specified each event. We used a deconvolution approach to overcome the problem of overlapping EEG responses to sequential saccades in free viewing. Brain activity was time-locked to the fixation onsets, and we examined EEG power in the theta and alpha frequency bands, the putative oscillatory correlates of episodic encoding mechanisms. Three modulations of fixation-related EEG predicted high subsequent memory performance: 1) theta increase at fixations after between-category gaze transitions, 2) theta and alpha increase at fixations after within-element gaze transitions, 3) alpha decrease at fixations after between-exemplar gaze transitions. Thus, event encoding with unrestricted viewing behavior was characterized by three neural mechanisms, manifested in fixation-locked theta and alpha EEG activity that rapidly turned on and off during the unfolding eye movement sequences. These three distinct neural mechanisms may be the essential building blocks that subserve the buildup of coherent episodic memories during unrestricted viewing behavior.


Introduction
To form a coherent event representation during encoding, participants critically need to engage in 126 sequential eye movements, where each event element gets visually attended to on one or multiple 127 occasions. Thus, to successfully establish the element associations specifying each event, we 128 expected the encoding process to critically depend on participants' gaze transitions between the 129 elements (cf. Kragel et al., 2021), as well as on the visual sampling of each individual element (cf. Liu 130 et al., 2017;Loftus, 1972;Olsen et al., 2016). As the ensuing associative memory task concerned 131 between-category associations, we expected gaze transitions between categories to be of particular 132 importance and, thus, more predictive of memory performance than transitions between exemplars 133 from the same category (associations that participants were not asked to retrieve). Based on 134 previous research by Kragel et al. (2021), we also expected repeated cross-category gaze transitions 135 to facilitate successful event formation. 136 137 We simultaneously recorded EEG and eye movements during event encoding to examine theta and 138 alpha activity related to such gaze behaviors during memory formation. In our analytical approach, 139 we first sought to verify that the oscillatory nature of the EEG signals coregistered with eye 140 movements in our free-viewing paradigm corresponds to the encoding-related neural signatures 141 typically found in comparable studies with fixed stimulus presentation (Hanslmayr et al., 2016). To 142 achieve the central goals of the present study, we then corrected EEG from overlapping effects of 143 eye movements using a deconvolution approach (Ehinger and Dimigen, 2019) and extracted the EEG 144 power relative to fixation onset. Since this was the first study to examine scalp-recorded EEG coregistered with eye movements during a free-viewing associative memory task, we first 146 investigated the time course of fixation-related EEG during the whole 10-s interval of event 147 encoding. Then, we examined EEG power related to three types of fixations: (1) fixations succeeding 148 a between-category saccade (i.e., a saccade moving from one element to another across categories); 149 (2) fixations succeeding a between-exemplar saccade (i.e., a saccade moving from one element to 150 another within the same category); and (3) fixations succeeding a within-element saccade (i.e., a 151 saccade moving from one point to another within the same element). In general, we expect all 152 saccades between event elements to be important for building a coherent episodic representation. 153 However, as our memory task required the retrieval of between-category associations, we reasoned 154 that saccades between categories should be vital and predictive of subsequent performance, in 155 contrast to between-exemplar saccades supporting associations that were not called for. Thus, by 156 analyzing both saccade types, we can examine the specificity of gaze transitions between event 157 elements upon subsequent memory performance as a function of task relevance. This approach 158 allowed us to examine alpha and theta activity related to functionally distinct gaze behaviors 159 contributing with complementary aspects to the episodic memory representation: eye movements 160 that serve to (1) sample visual information from the individual event elements and crucially (2)  Thirty-six healthy adults participated in the experiment in exchange for a gift card in a shopping mall 167 (approx. €10). Three participants were excluded because of system crashes during the data 168 collection and another three due to below-chance memory performance (the chance level threshold 169 was set at 53.2%, which was obtained from the permutation procedure indicating chance 170 performance in a two-alternative forced choice (2AFC) task ('left' or 'right' responses in a memory 171 test) for the 5% confidence level and for a total number of 648 test trials). Finally, two participants 172 were excluded due to ceiling performance (> 90%), i.e., too few incorrect trials for the planned 173 subsequent memory analysis. The final group included 28 participants (20 females; mean age 23.3; 174 age range 18-32). 175

176
The study was conducted in accordance with the Swedish Act concerning the Ethical Review of 177 Research involving humans (2003:460). All participants gave written informed consent, and the 178 study followed the local ethical guidelines at Lund University. Moreover, the study followed the Lund 179 9 250 Prior to the main experiment, two practice sessions with increasing difficulty (three events in the 251 first session, six events in the second session) were conducted in which participants received 252 feedback on their performance after each test trial. The gradual increase in difficulty over the two 253 practice sessions served to prepare participants for what they could expect in the main experiment 254 (where nine events per block were presented). Ten associative memory tests were delivered in the 255 practice sessions, in which between-category associations were evenly distributed over events and 256 categories. Images in the practice sessions were not repeated in the main experiment and were not 257 repeated over the two practice sessions. The overall duration of the experiment was about two 258 hours. Memory performance for each event was assessed by retrieval accuracy weighted by confidence, 298 such that correct responses were scored as 3 for "Sure" responses, as 2 for "Maybe" responses, and 299 as 1 for "Guess" responses. Each image from each event occurred either as a cue or a target in four 300 memory tests trials. Thus, each image can in total receive a score ranging from 0 (incorrect 301 responses in all four tests) to 12 (correct responses accompanied by four "sure" responses in all four 302 tests). To calculate a memory score for the whole event, we then summed the scores for the six 303 elements of each event (ranging from 0 to 72). For the EEG analysis, events were divided into "high 304 memory" and "low memory," according to the subsequent memory scores. For each participant, 305 high and low memory was calculated by a median split of that participant's memory scores for the 306 whole events. 307 308 2.5.2 Eye movement analysis 309 Fixations and saccades were detected from participants' right eye using the velocity-based algorithm 310 for saccade detection (Engbert and Mergenthaler, 2006) implemented in the EYE-EEG extension. 311 Fixations were considered located on an image if within 0.5 dva of the outer contour of the image. 312 Since participants cannot adequately perform a memory test if they do not visually process (fixate) 313 all images while event encoding, we excluded from all analyses the events where fixations were 314 detected only on two of the three categories (on average 3±4 (mean ± SD) events per participant). 315 Such events occurred because of eye-tracking issues mainly towards the end of the experiment (see 316 the Supplementary materials and Fig. S1 for details). Fixations outside the images on different parts 317 of the display were assigned to an 'other' saccade type, which was used as a reference level in the 318 deconvolution modeling (2.5.5). 319 320 Saccades were classified according to whether they occurred between categories (between-category 321 saccades), between the two exemplars of a category (between-exemplar saccades), and within an 322 exemplar (within-element saccades). In addition, we distinguished between the first visit to an image 323 and revisits to it. Altogether, we defined five different types of saccades ( Fig. 3A) We expected these saccade types to differentially contribute to forming a coherent event 331 representation. Specifically, in our associative memory task, we tested episodic memory for 332 between-category associations (i.e., associations between exemplars from different categories) 333 within each event. The formation of these associations supposedly depends on the corresponding 334 between-category saccades during encoding. In contrast, associations established by between-335 exemplar saccades (i.e., gaze transitions between associations incidental to the task) were not 336 tested and did not expect to predict subsequent memory performance. Thus, between-category and 337 between-exemplar saccades support associations between event elements that were or were not 338 critical to the success of the subsequent memory test, respectively. The goal of our study required fixation-related analysis of EEG in short epochs time-locked to fixation 364 onsets in the encoding interval. This analysis used deconvolution modeling to correct for the 365 overlapping effects of saccades on EEG. Deconvolution modeling involves time regression, which 366 requires continuous EEG; thus, preprocessing was performed on continuous EEG. In addition, we 367 analyzed EEG time-locked to the event onset during the entire 10-s encoding interval. The 368 preprocessing for this analysis was the same as for the fixation-related analysis. We used the 369 following preprocessing and deconvolution pipelines developed in our previous research (Nikolaev 370 et al., 2022). 371

372
We down-sampled EEG signals to 256 Hz. The EEG and eye-tracking data were synchronized using 373 the function pop_importeyetracker from the EYE-EEG extension. Saccades, fixations, event onsets, 374 and bad eye-tracking intervals were inserted into the EEGLAB data structure.

376
The preprocessing pipeline included a series of cleaning functions from EEGLAB. The pop_cleanline 377 function removed power line noise using multi-tapering and a Thompson F-statistic. The 378 clean_artifacts function removed flat-line channels, low-frequency drifts, noisy channels, and short-379 time bursts. This function is based on artifact subspace reconstruction (ASR), which compares the 380 structure of the artifactual EEG to that of known artifact-free reference data (Kothe and Jung, 2016). 381 The tradeoff between artifactual and retaining brain activities depends on the ASR parameter, which 382 we set to 20 according to the recommendations by Chang and colleagues (Chang et al., 2020). 383 384 Ocular artifacts were removed with the OPTICAT function (Dimigen, 2020). First, EEG was high-pass 385 filtered at 2 Hz to suppress large deviations from baseline due to summation of the corneo-retinal 386 artifacts during sequential eye movements. Next, 30-ms segments around saccade onsets were 387 obtained and re-appended to EEG to 'overweight' the contribution of the saccadic spike activity in 388 the EEG input to independent component analysis (ICA). Then, the ICA weights obtained after ICA 389 training on these filtered data were transferred to the unfiltered version of the same data. Finally, 390 the ratio between the mean variance of independent components during saccade and fixation 391 intervals was calculated. If the ratio was higher than 1.1, the corresponding independent 392 components were considered saccade-related and removed (Plöchl et al., 2012). The fBOSC method was particularly suitable for our goals because it assumes that the 1/f fit is not 423 necessarily linear across all frequencies in log-log space, and allowed us to fit an aperiodic 'knee' in 424 the power spectrum. The presence of such a 'knee' is especially likely in data with pronounced theta 425 rhythmicity (Seymour et al., 2022), which is quite expected in our data. 426

427
To detect oscillations, fBOSC computes two thresholds: a power threshold estimated from the 428 aperiodic background spectrum, which we set at the 95 th percentile of the theoretical probability 429 (chi-squared) distribution of power, and a duration threshold, which we set at 3 oscillatory cycles. 430 Since oscillations may distort the background spectrum, fBOSC excludes oscillatory peaks from its 431 calculation. To do this, fBOSC performs an initial 1/f fit, then iteratively models oscillatory peaks 432 above this background and removes them from the spectrum. Next, fBOSC models the 1/f power 433 spectrum as nonlinear by using a variable knee parameter (see Seymour et al. (2022) for details on 434 its estimation). We applied fBOSC to single EEG trials of the 10-s encoding interval. The results of the 435 single-trail modeling were averaged across all participants for low and high memory events 436 separately. To analyze the frequency distribution of the detected oscillations and their relationship 437 to memory, we used 'abundance', which is considered the most informative metric in burst analysis 438 (Kosciessa et al., 2020). Abundance is the duration of a rhythmic episode relative to the length of the 439 analyzed segment, which can range from 0 to 1, where 0 means no burst and 1 means a single 440 continuous burst. 441 442

443
To overcome the problems associated with EEG-eye movement coregistration in unrestricted 444 viewing behavior described above, we used deconvolution modeling implemented in the Unfold toolbox (Ehinger and Dimigen, 2019). The toolbox allowed us to correct overlapping EEG responses 446 to sequential saccades in free viewing, as well as to account for several possible detrimental 447 covariates. The deconvolution modeling is based on multiple regression, as traditional mass-448 univariate ERP modeling, which has been used for post-hoc control of multiple simultaneous discrete 449 and continuous covariates (Smith and Kutas, 2015). However, mass-univariate modeling requires a 450 prespecified analysis window. Therefore, when applied to correct the overlapping effects of 451 unrestricted eye movements on EEG, it cannot account for the varying temporal overlap caused by 452 variable fixation durations between adjacent eye movements (Dimigen and Ehinger, 2021). Behavioral analysis above). We used treatment (dummy) coding with 'other' fixations as the 495 reference level (intercept) relatively to which other levels (i.e., 'low memory' and 'high memory') of 496 the 'MemoryPerformance' predictor were estimated. The inclusion of 'other' fixations allowed us to 497 account for the overlapping effects on EEG produced at the latencies of 'other' eye movements. To 498 effectively account and control for these overlapping effects, we need to input in the model all 499 fixations that occurred in the experiment, without exception. Thus, the "other" fixations, together 500 with the fixations assigned to low and high memory levels, were essential levels of the 501 'MemoryPerformance' predictor. However, because the "other" fixations cannot explain memory for 502 the event, they were never used in further comparisons. Since fixation duration, and saccade size 503 and direction have nonlinear effects on EEG (Dimigen and Ehinger, 2021; Nikolaev et al., 2016), we 504 modeled them with a basis set of five spline predictors (circular splines were used to model saccade 505 direction). Since we found that the event order in the block has a nonlinear effect on the memory 506 score (Fig. 2B), and we assumed its nonlinear effect on the EEG, we modeled it also with spline 507 predictors. Spline knots were placed on the percentiles of the covariates for each participant. For 508 events, we considered only their onsets (y ~ 1), i.e., the intercept that described the overall shape of 509 EEG evoked by the event screens. In further analyses, we modified the 'MemoryPerformance' 510 predictor in the formula by inserting predictor levels depending on the analysis goals, as specified 511 below in each section of Results. All other predictors were constant in all analyses. 512

513
To recover isolated EEG responses (betas) to each fixation and to each event screen best explaining 514 the continuous EEG, we created a design matrix (Fig. S3A) and time-expanded it in a window between −200 and +400 ms around fixation and event onsets. This length of the time window was 516 motivated by the need to keep the number of iterations for fitting the deconvolution model before 517 model convergence reasonably small (e.g., <400) to avoid overfitting. The number of iterations 518 depends on the number (and quality: spline or not) of predictors and the number of sampling points. 519 At the pilot stage of the project, our experimentation with the window length using the selected 520 predictors showed that the length of -200+400 ms is optimal. Time expansion involved duplicating 521 and shifting the predictors of the mass univariate linear regression design matrix for each time lag. 522 The time-expanded design matrix (Fig. S3B)  averaged power waveforms across three electrodes since regional averaging of electrodes provides 583 a more reliable estimate of activity in a region than a single measurement (Dien and Santuzzi, 2005). 584 Eight ROIs were symmetrically about the sagittal axis and systematically distributed over the scalp.
Thus, these analyses allowed us to evaluate memory-related changes of theta and alpha power over 586 the entire encoding interval. 587 588 Second, we evaluated the general characteristics of fixation-related power extracted from the 589 deconvolved EEG. We estimated the theta and alpha power at the first peak of their cycle about 100 590 ms after the fixation onset (see 4.3 for justification of such selection). We averaged the power values 591 40 ms before and after the peak, that is, between 60 and 140 ms after the fixation onset. To 592 evaluate the time course of fixation-related EEG power during the 10-s encoding interval, we divided 593 EEG power related to successive eye movements into three bins according to fixation rank. We 594 compared memory-related changes of theta and alpha power across rank bins in eight ROIs. Thus, 595 this analysis was analogous at the fixation level to the first analysis at the event level. 596

597
The third analysis was devoted to the main goal of our study -to compare fixation-related EEG after 598 the five types of saccades described in section 2.5.2. We performed three separate analyses on each 599 saccade type because not all conditions were matched for all saccade subtypes. Specifically, we 600 could distinguish between First visits and Revisits for Between-category and Between-exemplar 601 saccades but not for Within-element saccades. All analyses compared the fixation-related theta and 602 alpha power between low and high memory for the same time window as in the second analysis and 603 eight ROIs. This analysis provided a detailed picture of associations during the formation of coherent 604 episodic memory at the fixation level. To emphasize findings directly relevant to the goals of our 605 study, we report effects and interactions in the Results section only when they involved the factors 606 of memory performance (High vs. Low) and gaze transition status (First visits vs. Revisit). Complete 607 statistical results, including effects and interactions between ROIs and hemispheres, are reported in 608

612
Average memory performance (regardless of confidence) across 28 participants was 68.6 ± 8.33 % 613 (mean ± SD). The memory score weighted by confidence for the low memory condition was 27.8 +/-614 6.8 and for the high memory condition 48.8 +/-9 (mean +/-SD). Memory performance changed in 615 the time course of a block and experiment (Fig. 2). We evaluated these changes with a repeated 616 measures ANOVA on the memory scores weighted by confidence with the factors of Event (nine 617 levels) and Block (six levels). There were significant main effects of Event (F(8, 216) = 3.7, p = .004, ε 618 = .62) and Block (F(5, 135) = 9.8, p < .001, ε = .95)), but no interaction between them. To characterize 619 the effects of Event and Block, we used planned comparisons with the linear and quadratic 620 contrasts. The contrast analysis revealed significant effects for the linear contrast for Block (F(1, 27) 621 = 31.2, p < .001) and for the quadratic contrast for Event (F(1, 27) = 16.8, p < .001), but not for the 622 linear contrast for Event (F(1, 27) = 0.7, p = .4). These results indicate a prominent learning effect 623 during the experiment, as well as possible primacy and recency effects in the time course of a block. 624 To evaluate the contribution of confidence to memory scores, we repeated the above analyses on 625 memory scores that were not weighted by confidence. The results were qualitatively the same 626

651
The dynamics of eye movement behavior during the 10-s encoding interval is illustrated in Fig. 3B by 652 the fixation rank probability density for each type of preceding saccade. At first, participants tended 653 to visit all categories quickly with between-category saccades. Then, between-exemplar saccades 654 became predominant. The number of within-element saccades was relatively stable throughout the 655 whole encoding interval. Towards the latter part of the interval, between-category revisits became 656 more frequent. 657

658
To determine if, and to what degree, the saccade categories predicted subsequent memory 659 performance, we analyzed the relationship between the cumulative number of saccades (in each 660 category) within an event and subsequent memory performance for that event. The between-661 category saccades for first visits were not analyzed since there were always 3 per event. We 662 computed the number of the remaining four saccade types for each event and pooled these 663 numbers with the corresponding event memory scores over 28 participants. Since the number of 664 saccades for different types varied quite widely (Fig. 3C) In sum, the eye movement results show that between-category saccades (revisits) and within-701 element saccades predict subsequent memory performance, whereas between-exemplar saccades 702 do not. The associative memory task determined these dependencies, which required strong 703 associations between the categories, whereas associations between the exemplars of a category 704 were less important to the task. Within-element saccades were necessary for acquiring sufficient 705 visual information about each image, a prerequisite for establishing a strong representation of each 706 element, and for recognizing them during retrieval. To assess how theta and alpha power at encoding predicts subsequent memory at the time scale of 713 the entire encoding interval, we segmented the EEG power (after removing all artifacts but before 714 deconvolution) from -0.2 to 10 s relative to the event onset and baseline corrected it to the interval 715 from -0.2 to 0 s before the event onset (Fig. 4A, B). We smoothed the individual power waveforms 716 with a Savitzky-Golay filter, using filter order 3 and filter length 201 sampling points (R package 717 'signal'). Events were split between low and high memory performance (the number of low memory 718 events was 28.5 (1.35); the number of high memory events was 25.1 (1.74) (mean (SD) per 719 participant). To investigate the modulation across a 10-s encoding interval, we created five time-bins 720 of 2 s each and averaged the power values in these bins (Fig. 4C, D). We applied a repeated- found an interaction between Memory and Time (F(4, 108) = 4.9, p = .001, ε = .99). Post-hoc test 724 revealed that this interaction occurred due to higher power for the high than low memory events for 725 the 4 th and 5 th time bins (all p < .02), i.e., between 6 and 10 s at the end of the encoding interval (Fig.  726   4C). 727 728 For alpha power, there was a clear trend towards decreased power in high-memory versus low-729 memory events, most noticeably over the occipital areas (Fig. 4B, D), which, however, did not reach 730 significance. The change of alpha power over time appears to be nonlinear. To test the properties of 731 this change statistically, we applied linear and quadratic contrasts to the time bins as planned 732 comparisons. For this analysis, we used only the data from the occipital ROIs, where the memory 733 effect on alpha power was most prominent (Fig. 4B). We found that the quadratic fit more 734 adequately describes the memory effect on the alpha activity over time, revealing a significant 735 difference between high and low memory events (F(1, 27) = 5.0, p = .03), than the linear fit, which 736 did not reveal a significant effect between high and low memory events (F(1, 27) = 0.5, p = .47). For 737 completeness, we also applied the linear and quadratic contrasts to the time bins for theta power 738 (all ROIs). Here the results were the opposite: there was a significant difference between low and 739 high memory events for the linear fit (F(1, 27) = 17.5, p < .001) but not for the quadratic fit (F(1, 27)  The finding that subsequent memory performance is associated with theta and alpha activity raises 767 the question of whether memory-related changes in spectral pattern result from neural oscillations 768 pertinent to encoding (Hanslmayr et al., 2016) or instead reflect increases in broadband activity or 769 shifts in spectral tilt. We detected theta and alpha oscillations using the fBOSC method, which 770 considered the possible nonlinearity of the 1/f power spectrum (Seymour et al., 2022). Fig. 4E shows 771 the averaged power spectrum, the aperiodic 1/f component, and the FOOOF model fit for the left 772 parietal area for high memory events (the results for all ROIs and both events are shown in Fig. S5A). 773 The grand-averaged results were about the same for low-and high-memory events. The power 774 spectrum had a clear peak in the alpha range and a deviation from the aperiodic fit at low 775 frequencies. 776 777 Although the grand-averaged power did not reach the power threshold, multiple oscillation episodes 778 were detected at the single-trial level. We compared the duration of these oscillations ('abundance') 779 between low and high memory. Fig. 4F shows the abundance averaged across participants for the 780 left parietal area for the frequency range 3-20 Hz (top) and for the close-up in the theta range 781 (bottom). The abundance for all ROIs is shown in Fig. S5B. The arrows point to the frequency interval of about 5-6 Hz, where the abundance was larger for the high than low memory events. The 783 abundance at the peak alpha frequency was visually larger for the low than high memory events. To 784 test these observations statistically, we averaged the abundance within the theta and alpha bands 785 Before analyzing the data for the different saccade categories, we wanted to assess if the EEG 807 findings at the event level could be reproduced at the fixation level. Thus, we tested the subsequent 808 memory effect on fixation-related theta and alpha power by contrasting the fixation-related EEG 809 overall eye movements within an event. We divided the memory scores into three bins according to 810 low, medium, and high memory events. We divided the fixations into three bins according to fixation 811 rank. In the formula for deconvolution modeling, we used ten levels of categorical predictors: all 812 possible combinations of three memory and three rank bins and 'other'. The number of fixations 813 used in the deconvolution modeling of each memory and rank bin condition is presented in Table S1. 814

815
We applied a repeated-measures ANOVA on the EEG power averaged in the window from 60 to 140 816 ms after the fixation onset with the factors of Memory (low vs. high, we excluded the medium memory bin to increase the memory contrast), Rank (3 bins), ROI (frontal, central, parietal, occipital) 818 and Hemisphere (left vs. right). For theta power, we found interactions between Memory and Rank 819 (F(2, 54) = 3.6, p = .038, ε = .96) (Fig. 5B) and between Memory and ROI (F(3, 81) = 3.6, p = .023, ε = 820 .86) (Fig. 5C). Post-hoc tests revealed that the Memory-Rank interaction occurred due to higher 821 power for the high than low memory in rank bin 3 (p = .02). Fig. 5A shows power waveforms for this 822 rank bin. The Memory-ROI interaction occurred due to higher power for the high than low memory 823 over the frontal (p = .006) and central (p = .001) areas. The difference maps indicate that this effect 824 was largest in rank bin 3 (Fig. 5D). 825 826 For alpha power, we found an interaction between Memory, Rank, and Hemisphere (F(2, 54) = 5.0, p 827 = .01, ε = .98) (Fig. 5E-G). Post-hoc tests revealed that the interaction occurred due to a power 828 decrease for the high than low memory in rank bin 2 (Fig. 5F) over the left hemisphere (p = .002); the 829 similar trend in rank bin 3 did not reach post-hoc significance (p = .14). There was also an interaction 830 between Memory, Rank, ROI, and Hemisphere (F(6, 162) = 2.4, p = .04, ε = .84), but the post-hoc 831 tests showed no significant differences.

841
To summarize, the fixation-related EEG results corroborate and extend what was found at the event 842 level, i.e., theta increase and alpha decrease during encoding covary with subsequent memory 843 performance. These effects were also more prominent toward the end of the encoding interval. 844 Next, we analyzed EEG power at fixations following the five saccade types of interest. 845 846 3.4.2 Subsequent memory effects across saccade types 847 848 We focused on EEG power in the fixation intervals succeeding the five saccade types (Fig. 3A). For 849 this purpose, we specified ten levels of categorical predictors in the deconvolution model formula: 850 the five saccade types x memory performance (low, high). The number of fixations that were used in 851 the deconvolution modeling of each saccade and memory condition is presented in Here, we examined EEG power in the fixation intervals succeeding the between-category saccades 859 that we predicted to support tested associations between categories (Fig. 6A). We applied a 860 repeated measures ANOVA on fixation-related theta and alpha power with the factors of Gaze 861 transition status (First visits vs. Revisit), Memory (Low vs. High), ROI (Frontal, Central, Parietal, 862 Occipital), and Hemisphere (Left vs. Right). For theta power, we found a significant main effect of 863 Gaze: theta power was higher during first visits than revisits (F(1, 27) = 5.0, p = .03). We also found 864 an interaction between Gaze and ROI (F(3, 81) = 4.2, p = .01, ε = .94). More importantly, we found an 865 interaction between Gaze, Memory, and Hemisphere (F(1, 27) = 11.8, p = .002) (Fig. 6B-E). Post-hoc 866 tests revealed that this interaction occurred due to higher power for the high than low memory for 867 revisits over the left hemisphere (p < .001) (Fig. 6E). For alpha power, no main effects of Memory 868 (F(1, 27) = 3.1, p = .09) or Gaze (F(1,27) = 1.7, p = .2) emerged (nor an interaction between them, 869 F(1, 27) = 0.5, p = .5). 870 871 Thus, theta power increase over the left hemisphere after between-category revisits predicts 872 subsequent event memory performance. This theta memory effect is consistent with the eye-873 movement analyses, where subsequent memory performance improved with the cumulative 874 number of between-category revisits during encoding (Fig. 3C). Next, we examined EEG power in the fixation intervals succeeding between-exemplar saccades, i.e., 887 gaze transitions establishing element associations that were not tested in the subsequent memory 888 test (Fig. 7A). As in the previous analysis, we used a repeated measures ANOVA with a 2x2 design: As expected, theta power after between-exemplar saccades did not predict subsequent event 896 memory performance for the tested between-category associations. This is consistent with the eye-897 movement result, where the cumulative number of between-exemplar saccades did not influence 898 subsequent memory performance (Fig. 3C). The effect of lower alpha power for high memory was 899 unexpected, but may reflect other mechanisms, apart from binding per se, that are important for 900 optimal memory formation. Finally, we analyzed theta and alpha power in the fixation intervals after within-element saccades 912 (Fig. 8A). For this saccade type, we did not distinguish between first visits and revisits, so the ANOVA 913 design included three factors: Memory (Low vs. High), ROI (Frontal, Central, Parietal, Occipital), and 914 Hemisphere (Left vs. Right). For theta power, we found an interaction between Memory and ROI 915 (F(3, 81) = 3.4, p = .025, ε = .95) (Fig. 8B, C). Post-hoc tests revealed that this interaction occurred 916 due to higher theta power for high than low memory over the frontal (p = .002) and central (p = .03) 917 areas (Fig. 8D). For alpha power, we found an interaction between Memory and Hemisphere (F(1, 918 27) = 4.9, p = .035) (Fig. 8E-F). Post-hoc tests revealed that this interaction occurred due to higher 919 alpha power for high than low memory over the right hemisphere (p = .01) (Fig. 8G). Thus, the theta power related to within-element saccades predicts subsequent memory 932 performance similarly as for the between-category revisits. Moreover, for both saccade types, 933 subsequent memory increased as a function of the cumulative number of saccades during encoding 934 (Fig. 3C). Importantly, however, the topographies of the theta memory effects for the two saccade 935 types were distinct: a left central maximum for between-category revisits (Fig. 6C) and a frontal 936 maximum for within-element saccades (Fig. 8C), suggesting dissociable neural mechanisms. 937

938
To test the observed topographical dissociation of the two theta effects, we extracted the 'within-939 element effect' as the difference between theta power in high and low memory performance for 940 fixations after within-element saccades. We extracted the 'between-category effect' as the 941 difference between theta power in high and low memory performance for fixations after between-942 category revisits. A repeated-measures ANOVA on the theta power difference with factors of Theta 943 effect ('within-element' vs. 'between-category'), ROI (Frontal, Central, Parietal, Occipital), and 944 Hemisphere (Left vs. Right) revealed an interaction between Theta effect and Hemisphere (F(1, 27) = 945 9.0, p = .006). The post-hoc test revealed higher power for the between-category effect over the left 946 hemisphere (p = .03) (Fig. 9). This finding indicates topographical differences for the two theta 947 effects and suggests that they are associated with distinct neural mechanisms operating in the same 948 frequency band. 949 950 In sum, the correspondence of theta and eye movement results for between-category saccades and 951 within-element saccades, which may be based on distinct mechanisms, indicates a diversity of 952 relationships between theta activity, eye movements, and episodic memory formation, as will be 953 discussed below. The present study set out to advance our current understanding of the relationship between eye 965 movements and episodic memory encoding. In particular, we targeted the neural mechanisms that 966 mediate the formation of coherent episodic memories during consecutive saccades to event 967 elements during unrestricted viewing behavior. To this end, we applied a state-of-the-art analytical 968 approach to EEG coregistered with eye movements during a free-viewing episodic memory task. Our 969 approach provided a unique possibility to capture neural encoding mechanisms across eye 970 movements at the level of gaze fixations while overcoming the confounding effects of sequential 971 saccades on brain activity. We identified neural signatures that are associated with the buildup of 972 episodic memories as a function of visual sampling behavior. The encoding of associations between 973 event elements was accompanied by simultaneous modulation of eye movements and theta activity 974 that were predictive of subsequent memory. This modulation likely reflects the binding of event 975 elements into a coherent representation across eye movements. In addition, a subsequent memory 976 effect was also observed for theta and alpha activity related to small scrutinizing saccades within 977 elements. Although the association between exemplars within categories was incidental to the task, 978