A parallel framework for simultaneous EEG/fMRI analysis: Methodology and simulation

Abstract

Concurrent EEG/fMRI recordings represent multiple, simultaneously active, regionally overlapping neuronal mass responses. To address the problems caused by the overlapping nature of these responses, we propose a parallel framework for Spatial–Temporal EEG/fMRI Fusion (STEFF). This technique adopts Independent Component Analysis (ICA) to recover the time-course and spatial mapping components from EEG and fMRI separately. These components are then linked concurrently in the spatial and temporal domain using an Empirical Bayesian (EB) model. This approach enables information one modality to be utilized as priors for the other and hence improves the spatial (for EEG) or temporal (for fMRI) resolution of the other modality. Consequently, STEFF achieves flexible and sparse matching among EEG and fMRI components with common neuronal substrates. Simulations under realistic noise conditions indicated that STEFF is a feasible and physiologically reasonable hybrid approach for spatiotemporal mapping of cognitive processing in the human brain.

Introduction

Functional magnetic resonance imaging (fMRI) noninvasively measures local hemodynamic changes associated with neuronal activity. An unresolved issue in fMRI research is the limited temporal resolution of the blood oxygenation level dependent (BOLD) response. In contrast to the poor temporal resolution of fMRI, electroencephalogram (EEG) measurements can instantaneously record the electrophysiological signal produced by synaptic activity. However, due to the ‘inverse problem’ inherent in EEG recording (Helmholtz, 1853), the spatial location of the neural generators of observed activity cannot be conclusively determined (Baillet et al., 2001, Lei et al., 2009a). The fusion of these two complementary noninvasive methods would allow for the combined high-resolution spatial and temporal mapping of mental processes. Such a technique could provide a more comprehensive understanding of the neural correlates of perception and cognition (Ives et al., 1993, Dale and Halgren, 2001, Debener et al., 2006, Ritter and Villringer, 2006).

Physiologically, simultaneous EEG/fMRI recordings constitute volume-conducted and hemodynamically convolved signals from neural events that are spatially and temporally ‘mixed’ across the brain. That is, the observed data in both modalities represents responses from multiple, simultaneously active, regionally overlapping neuronal populations (Baudena et al., 1995). Scalp EEG recordings sample a spatially degraded map of neural activity, and a temporal mixture of independent time-courses from large-scale synchronous field potentials (Makeig et al., 2004). fMRI involves several equivalent constraints, providing temporally degraded and spatially mixed signals by measuring the neurovascular transformation of neural activity (Calhoun and Adali, 2006). Independent component analysis (ICA) is potentially an ideal approach to address this mixing problem and has been applied successfully to a variety of problems in EEG (Makeig et al., 2004) and fMRI recording (McKeown et al., 1998, Calhoun et al., 2001, Chen and Yao, 2004, Beckmann and Smith, 2004).

EEG/fMRI fusion is typically based on the assumption that the hemodynamic response is linearly related to local changes in neuronal activity and, in particular, to local field potentials (Logothetis et al., 2001). Large-scale synchronous field potentials underlie the electrophysiological signal recorded by scalp EEG (Nunez, 1995). Consequently, a method for integrating the two approaches may be developed by determining the relation between the BOLD signals measured by fMRI and the electrophysiological measures provided by EEG, either in the spatial or temporal domain.

There are currently three broad potential approaches to EEG/fMRI integration: (i) ‘symmetric fusion’, where a common generation model is constructed to explain both the EEG and fMRI data (Daunizeau et al., 2007, Deco et al., 2008, Valdes-Sosa et al., 2009); (ii) ‘spatial constraint’, where spatial information from fMRI recordings is used for source reconstruction of the EEG data (Liu et al., 1998; Dale et al., 2000, Trujillo-Barreto et al., 2001, Lei and Yao, 2009 and (iii) ‘temporal prediction’, where the fMRI signal is modeled with data from certain EEG measures, convolved with a hemodynamic response function (HRF; Martinez-Montes et al., 2004, Debener et al., 2006, Eichele et al., 2008a, Moosmann et al., 2008).

In temporal prediction, various features of the EEG signal can be used as measures of interest, such as alpha power (Goldman et al., 2002) and P300 amplitude (Eichele et al., 2005, Warbrick et al., 2009) among others. EEG data can be convolved with a canonical HRF, the result of which can be used as a hemodynamic predictor in a general linear model (GLM). This approach has been adopted in the study of spontaneous brain rhythms (Goldman et al., 2002), epileptic discharges (Laufs et al., 2008), and the inducing amplitude variation in a cognitive task (Debener et al., 2006). For spatial constraint, in order to extend the fMRI-constrained EEG inversion (Liu et al., 1998, Dale et al., 2000), we recently proposed an EEG source reconstruction method based on fMRI connectivity patterns. This technique, termed ‘network-based EEG source imaging’ (NESOI), uses multiple spatially independent maps (or networks) derived from fMRI as covariance priors for EEG source reconstruction (Lei and Yao, 2009).

The spatial constraint and temporal prediction approaches described above do not consider EEG and fMRI data sets equivalently or analyze them jointly. The goal of these methods is typically either spatial localization or temporal dynamic reconstruction, in which one modality is given privileged status as a prior for the other modality (Valdes-Sosa et al., 2009). Thus, they are examples of asymmetrical EEG/fMRI integrations. In contrast, symmetrical fusion does not assign an a priori inferential preference to any given modality (Trujillo-Barreto et al., 2001). The existing symmetrical fusion based on a cascade of generation models may provide a deeper understanding of the neural mechanisms underlying mental processes of interest (Daunizeau et al., 2007, Deco et al., 2008, Valdes-Sosa et al., 2009). However, current generative model-driven symmetrical methods employ highly detailed large-scale computational modeling and require the explicit definition of the common neuronal substrates that elicit both EEG and fMRI measurements. However, the lack of precise knowledge about neural mechanisms has led to a reduced scope for the application of these techniques.

In contrast to the generative model-driven fusion which exploits models of the chain of events leading to observed measurements, data-driven fusion is based on measuring mutual dependence between the two modalities (Valdes-Sosa et al., 2009). In this approach, original EEG/fMRI data are typically first decomposed into components, then matched to each other (Calhoun et al., 2009). Blind source separation has been used to address the mixing problems of EEG/fMRI using parallel ICA (Eichele et al., 2008a) and joint ICA (Moosmann et al., 2008). Eichele et al. (2008a) performed integration with simple pair-wise matching of across-trial modulation. In a study by Moosmann et al. (2008), the components linking both modalities were estimated using decomposition of the joint data space. The key strength of data-driven fusion is its ability to remove noise from the data, generate priors and provide group inferences that can serve as constraints for model-driven methods. Hence data-driven techniques are helpful in localizing the generators of EEG phenomena and informing models of interaction among levels of cortical hierarchies (Garrido et al., 2007). Furthermore, data-driven approaches may provide a solution to the problems faced by neurovascular transformation function estimation. This may clarify the relationship between the electrophysiology of neuronal systems and their slower hemodynamics in terms of their individual forward models (Deco et al., 2009).

Methodological and conceptual developments in the field of multimodal integration are ongoing, and the need for a more flexible model is one of the outstanding challenges for EEG/fMRI integration (Eichele et al., 2009). In the current study, we adopted a hybrid approach, where data-driven blind source separation (group ICA) was cascaded with an EEG forward model and a neurovascular transformation convolution model for neural source estimation and hemodynamic response function reconstruction. We term this approach Spatial-Temporal EEG/fMRI Fusion (STEFF). STEFF is not based on a common generative model, but employs constraints and predictions in an unmixed space. In STEFF, two asymmetric fusion methods, spatial constraint and temporal prediction, are implemented in parallel. Information from one modality is used to generate priors for the other modality and the matching between components is estimated using variational Bayesian inference. Here we present the details of the STEFF approach and the results from simulated data under realistic noise conditions.