Multimodal Study of the Effects of Varying Task Load Utilizing EEG, GSR and Eye-Tracking

Objective The effect of task load on performance is investigated by simultaneously collecting multi-modal physiological data and participant response data. Periodic response to a questionnaire is also obtained. The goal is to determine combinations of modalities that best serve as predictors of task performance.

Approach A group of participants performed a computer-based visual search task mimicking postal code sorting. A five-digit number had to be assigned to one of six different non-overlapping numeric ranges. Trials were presented in blocks of progressively increasing task difficulty. The participants’ responses were collected simultaneously with 32 channels of electroencephalography (EEG) data, eye-tracking data, and Galvanic Skin Response (GSR) data. The NASA Task-Load-Index self-reporting instrument was administered at discrete time points in the experiment.

Main results Low beta frequency EEG waves (12.5-18 Hz) were more prominent as cognitive task load increased, with most activity in frontal and parietal regions. These were accompanied by more frequent eye blinks and increased pupillary dilation. Blink duration correlated strongly with task performance. Phasic components of the GSR signal were related to cognitive workload, whereas tonic components indicated a more general state of arousal. Subjective data (NASA TLX) as reported by the participants showed an increase in frustration and mental workload. Based on one-way ANOVA, EEG and GSR provided the most reliable correlation to perceived workload level and were the most informative measures (taken together) for performance prediction.

Significance Numerous modalities come into play during task-related activity. Many of these modalities can provide information on task performance when appropriately grouped. This study suggests that while EEG is a good predictor of task performance, additional modalities such as GSR increase the likelihood of more accurate predictions. Further, in controlled laboratory conditions, the most informative or minimum number of modalities can be isolated for monitoring in real work environments.