Elsevier

NeuroImage

Volume 59, Issue 4, 15 February 2012, Pages 3641-3651
NeuroImage

Pattern analysis of EEG responses to speech and voice: Influence of feature grouping

https://doi.org/10.1016/j.neuroimage.2011.11.056Get rights and content

Abstract

Pattern recognition algorithms are becoming increasingly used in functional neuroimaging. These algorithms exploit information contained in temporal, spatial, or spatio-temporal patterns of independent variables (features) to detect subtle but reliable differences between brain responses to external stimuli or internal brain states. When applied to the analysis of electroencephalography (EEG) or magnetoencephalography (MEG) data, a choice needs to be made on how the input features to the algorithm are obtained from the signal amplitudes measured at the various channels. In this article, we consider six types of pattern analyses deriving from the combination of three types of feature selection in the temporal domain (predefined windows, shifting window, whole trial) with two approaches to handle the channel dimension (channel wise, multi-channel). We combined these different types of analyses with a Gaussian Naïve Bayes classifier and analyzed a multi-subject EEG data set from a study aimed at understanding the task dependence of the cortical mechanisms for encoding speaker's identity and speech content (vowels) from short speech utterances (Bonte, Valente, & Formisano, 2009). Outcomes of the analyses showed that different grouping of available features helps highlighting complementary (i.e. temporal, topographic) aspects of information content in the data. A shifting window/multi-channel approach proved especially valuable in tracing both the early build up of neural information reflecting speaker or vowel identity and the late and task-dependent maintenance of relevant information reflecting the performance of a working memory task. Because it exploits the high temporal resolution of EEG (and MEG), such a shifting window approach with sequential multi-channel classifications seems the most appropriate choice for tracing the temporal profile of neural information processing.

Introduction

Electroencephalography (EEG) and magnetoencephalography (MEG) are commonly used to study the time course of neural information processing in the human brain with high temporal resolution. In most cases, EEG/MEG studies rely on the comparison of averaged responses to repeated presentations of experimental conditions either in the temporal domain (event-related potentials [ERPs] or fields [ERFs], respectively) and/or in the frequency domain (event-related desynchronization and synchronization) (Pfurtscheller and Lopes Da Silva, 1999). Often, the statistical analyses (and related inferences on neural processing) are limited to a-priori specified (spectro-) temporal windows of interest – at channel or estimated source level – and therefore only a small subset of the measured signal is actually utilized.

This article illustrates several approaches to EEG data analysis based on pattern recognition (e.g. Bishop, 2007, Duda et al., 2001). In contrast to the conventional approach where a single dependent variable is examined (univariate statistics), these techniques exploit the information content in patterns of dependent variables (features), which are extracted from the measured signals. Pattern recognition allows analyzing EEG data in a more exploratory and data-driven manner and – similar to the recent developments in fMRI (e.g. Haynes and Rees, 2006) – promises to complement conventional approaches for EEG/MEG analysis.

A typical application of pattern recognition methods includes three steps, (1) extracting and selecting features (i.e. dependent variables), (2) learning a model with a machine-learning algorithm, and (3) determining the generalization ability of the learnt model using an independent evaluation dataset. In EEG/MEG, various types of features can be considered, ranging from signal amplitude in the temporal domain (e.g. Rieger et al., 2008) to power or phase information in the frequency domain (Kerlin et al., 2010, Luo and Poeppel, 2007, Rieger et al., 2008). Specific transformations, such as wavelet coefficients (Åberg and Wessberg, 2007, Rieger et al., 2008), and coherence measures (Besserve et al., 2007) can also be used. Furthermore, features can be differently grouped in the (spectral-) temporal and spatial domain. For example, limiting the information to pre-defined temporal windows of interest is essential to many realizations of EEG-based brain-computer interface (BCI) systems (e.g. Birbaumer, 2006, Blankertz et al., 2011, Wolpaw et al., 2002). Alternatively, the information contained in a sliding time interval of EEG data can be used, e.g. to detect the occurrence of seizures in epileptic subjects (Schad et al., 2008). Concerning the spatial (channel) domain, many BCI systems employed spatial filters (i.e. linear combinations of channels; see Blankertz et al., 2011) to enhance performances. For the same reason sophisticated feature selection or reduction methods were applied in BCI systems (see Bashashati et al., 2007).

Several machine-learning algorithms have been used to learn the relation between selected features of the EEG/MEG data and experimental labels. These algorithms include simple correlation (e.g. Luo and Poeppel, 2007), support vector machines (SVMs) (Vapnik, 1995), linear discriminant analysis (LDA) (e.g. Duda et al., 2001), and neural networks or Bayesian approaches (Bishop, 2007). Most frequently, learning algorithms are based upon linear models (e.g. Lotte et al., 2007, Rieger et al., 2008, van Gerven et al., 2009) due to their fast computation, robustness and simplicity of results interpretation.

To determine the generalization ability of the computed model, an independent set of test data is required. This can be done at single-subject level, splitting the measured data into training and testing sets (e.g. Luo and Poeppel, 2007) or across subjects, using a subset of subjects for training and the other for evaluating the generalization performance (e.g. Kerlin et al., 2010).

In this study, we consider and evaluate the effects of differently combining and grouping the features in the temporal (predefined windows, shifting window, whole trial) and channel domain (single channel, multichannel) in the context of a neuro-cognitive EEG paradigm. Using Gaussian Naïve Bayes (GNB; Mitchell, 1997) classification, we analyze data from an auditory EEG study aimed at understanding the task dependence of the cortical mechanisms underlying the processing of voice and speech identification (Bonte et al., 2009) and illustrate the results of each possible feature combination in the temporal and channel domain.

Section snippets

Materials and methods

Machine-learning approaches for the analysis of neuroimaging data require single trials to be described by an n-dimensional vector of features. In our approach, basic features are defined as EEG voltages and include time (samples) and measurement channels (electrodes). In particular, we consider six types of classification analyses derived from combining three types of features grouping in the temporal domain (predefined windows, shifting windows, whole trial) with two approaches to handle the

Predefined windows

We first considered the classifications of speakers and vowels in five predefined temporal intervals (N1, P2, N270, P340, LateP). Fig. 2.a shows – for the single channel case – group classification results for speaker and vowel grouping during the speaker (top panels) and vowel task (lower panels), respectively. To estimate reproducibility across subjects, we created topographic maps depicting, at each channel, the number of subjects with a significant classification performance. For each

Pattern recognition and EEG data

We have illustrated different strategies for analyzing EEG data using a pattern recognition algorithm. We have shown that it is feasible to distinguish experimental conditions above chance level, at the fine-grained level of speaker and vowel identity. Although low, our single-trial classification accuracies were significant even at a single subject level, which indicate that – although noisy – EEG single trial responses carry information on the neural processing of individual speech sounds.

Conclusions

We have illustrated different ways of analyzing EEG data by means of a pattern classification algorithm. Outcomes of the analyses show that grouping or separating available features (channels, time windows) helps highlighting different aspects of information content in the data. Because of the high temporal resolution of EEG (and MEG) a shifting window approach with sequential multi-channel classifications proved to be the most valuable as it allows tracing the temporal evolution of stimulus

Acknowledgments

Financial support by the Netherlands Organization for Scientific Research, Innovative Research Incentives Scheme VENI Grant 451-07-002 (MB) and VIDI Grant 452-04-330 (EF) is gratefully acknowledged. We thank Giancarlo Valente for comments and discussions.

References (35)

  • J.R. Wolpaw et al.

    Brain–computer interfaces for communication and control

    Clin. Neurophysiol.

    (2002)
  • M.C. Åberg et al.

    Evolutionary optimization of classifiers and features for single-trial EEG Discrimination

    Biomed. Eng. Online

    (2007)
  • A. Bashashati et al.

    A survey of signal processing algorithms in brain–computer interfaces based on electrical brain signals

    J. Neural Eng.

    (2007)
  • P. Belin et al.

    Adaptation to speaker's voice in right anterior temporal lobe

    Neuroreport

    (2003)
  • Y. Benjamini et al.

    Controlling the false discovery rate: a practical and powerful approach to multiple testing

    J. R. Stat. Soc. Series B Stat. Methodol.

    (1995)
  • M. Besserve et al.

    Classification methods for ongoing EEG and MEG signals

    Biol. Res.

    (2007)
  • N. Birbaumer

    Breaking the silence: brain–computer interfaces (BCI) for communication and motor control

    Psychophysiology

    (2006)
  • Cited by (0)

    View full text