Analyzing combined eye-tracking/EEG experiments with (non)linear deconvolution models

ABSTRACT

Fixation-related potentials (FRPs), neural responses aligned to saccade offsets, are a promising tool to study the dynamics of attention and cognition under natural viewing conditions. In the past, four methodological problems have complicated the analysis of combined eye-tracking and EEG experiments: (i) the synchronization of data streams, (ii) the removal of ocular artifacts, (iii) the condition-specific temporal overlap between the brain responses evoked by consecutive fixations, (iv) and the fact that numerous low-level stimulus and saccade properties also influence the post-saccadic neural responses. While effective solutions exist for the first two problems, the latter ones are only beginning to be addressed. In the current paper, we present and review a unified framework to deconvolve overlapping potentials and control for linear and nonlinear confounds on the FRPs. An open software implementation is provided for all suggested procedures. We then demonstrate the advantages of this analysis approach for three commonly studied free viewing paradigms: face perception, scene viewing, and natural sentence reading. First, for a traditional ERP face recognition experiment, we show how deconvolution can separate stimulus-ERPs from the overlapping muscle and brain potentials produced by small (micro)saccades on the face. Second, in natural scene viewing, we isolate multiple non-linear influences of saccade parameters on the FRP. Finally, for a reading experiment using the classic boundary paradigm, we show how it is possible to study the neural correlates of parafoveal preview after removing the spurious overlap effects caused by the difference in average fixation time. Our results suggest a principal way of measuring reliable fixation-related potentials during natural vision.