Evaluation of commercial brain–computer interfaces in real and virtual world environment: A pilot study

Highlights

• Identify the user’s adaptation on brain-controlled systems.

• Ability to control brain-generated events.

• Control a robot in both real and virtual worlds via brainwaves.

Abstract

This paper identifies the user’s adaptation on brain-controlled systems and the ability to control brain-generated events in a closed neuro-feedback loop. The user experience is quantified for the further understanding of brain–computer interfacing. A working system has been developed based on off-the-shelf components for controlling a robot in both the real and virtual world. Using commercial brain–computer interfaces (BCIs) the overall cost, set up time and complexity can be reduced. The system is divided in two prototypes based on the headset type used. The first prototype is based on the Neurosky headset and it has been tested with 54 participants in a field study. The second prototype is based on the Emotiv headset including more sensors and accuracy, tested with 31 participants in a lab environment. Evaluation results indicate that robot navigation through commercial BCIs can be effective and natural both in the real and the virtual environment.

Introduction

Brain–computer interfaces (BCIs) are communication devices which enable users to send commands to a computing device using brain activity only [1]. This technology is a rapidly growing field of research and an interdisciplinary endeavour. Research into BCIs involves knowledge of disciplines such as neuroscience, computer science, engineering and clinical rehabilitation. Brain-controlled robots and serious games can be used for a wide range of applications from modern computer games [2], prosthetics and control systems [3] through to medical diagnostics [4]. Although research on the field started during the 1970s only the last few years it became possible to introduce BCIs mostly through commercial headsets for computer games and other simulations.

BCI’s can be categorised based on the electroencephalographic (EEG) method of recording. There are mainly three categories including: invasive, partially-invasive and non-invasive. With invasive BCIs the signals are recorded from electrodes implanted surgically over the brain cortex, into the grey matter of the brain during neurosurgery. Partially-invasive BCIs are placed inside the skull but not within the grey matter. Non-invasive BCIs operate by recording the brain activity from the scalp with EEG sensors attached to the head on an electrode cap or headset without being surgically implanted and it is most widely used.

EEG is the most studied non-invasive interface, mainly due to its portability, ease of use and low set-up cost [5]. The raw EEG is usually described in terms of frequency ranges: Gamma (γ) greater than 30 Hz, Beta (β) 13–30 Hz, Alpha (α) 8–12 Hz, Theta (θ) 4–8 Hz, and Delta (δ) less than 4 Hz [6]. Delta (δ) waves with the lowest frequencies of all the brainwaves are most apparent in deep sleep states, where conscious brain activity is minimal. Theta (θ) waves appear in a relaxed state and during light sleep and meditation. Alpha (α) waves are typically associated with meditation and relaxation, more so than any other waves. Beta (β) waves are connected to alertness and focus. Gamma (γ) waves can be stimulated by meditating while focusing on a specific object.

Different BCI systems can be distinguished based on their application and classified into different areas of use. Recent reviews [7], [8] categorised different BCI systems in: communication, motor restoration, environmental control, locomotion and entertainment. BCI for communication is recognised as one of the most important applications of brain control due to the essential need for communication, use of language and expression of feelings. A number of approaches have been explored though the last decades of BCI research for using brain activity as a control signal for communication like slow cortical potential (SCP)-based spelling BCI for selecting letters of the alphabet through a spelling device [9], detection of eye blinks [10] or classification of mental tasks [11] for keyboard control, P300 event-related brain potentials for letter spelling [12] and finally Graz-BCI, a letter speller through mental hand and leg motor imagery [13]. Another important use of BCI’s is the motor restoration for stroke rehabilitation, spinal cord injury (SCI), traumatic brain injury (TBI) or other neurological diseases. The important function of a BCI system in neurological diseases can be attributed to the ability of closing the loop between the brain and the affected limbs through neuroprostheses [14], robotic orthosis and electrical stimulation [15] for compensating the ability of the lost function and by creating a pathway from the nervous system to the limb.

People with motor disabilities are confined within a home environment, unable to use domestic devices like TV, radio, lights and various other appliances. For this reason environmental control BCI systems have been created the last few years [16], [17] with promising results. In addition, robotic wheelchairs controlled by BCI-based systems [18] are used for indoor and outdoor navigation with the help of assistive sensors for obstacle avoidance and path finding due to the small information rate of the BCI and the slow reaction time of the patient. BCI systems for entertainment and video game interaction though brain activity are known for many years [19], [20]. Within the last few years due to the launch of many commercial BCI headsets as possible gaming controller’s [21], [22] brain controlled games have started to gain ground, exposing the brain interaction and adaptation mechanisms within games and virtual environments. This recent advancement in the field of EEG technology and BCI, is offering an acceptable quality-to-cost ratio and easy-to-use, out-of-the-box equipment with commercial BCI headsets for a plethora of new applications [23], [24] that could lead to new levels of understanding towards the study of the brain, its mechanisms and brain–computer interaction.

This research focuses on how a robot operated through brainwaves can overcome the kinetic constraints of the user. It investigates ways in extracting valuable information from user’s brain activity by interacting with both real world objects and virtual world environments. The experimental prototype uses the basic movement operations of a Lego Mindstroms NXT Robot. There are two versions of this prototype, taking readings from the users’ brain electrical activity in real-time performance. The first version uses a single dry sensor headset from Neurosky using the attention levels of the user. The second version is using a 14 wet sensor headset from Emotiv taking readings not only from EEG signals but also from facial expressions, eye movement and head tilt, enabling users to fully control the robot subject to training. Evaluation results indicate that robot navigation through commercial BCIs can be effective and natural. Overall, the experience with the virtual environment was reported as quite engaging and interesting regardless certain minor issues. It was reported that previous experience in computer games can speed up the learning time for using the brain interface. Finally, many reporting’s concerned the tiredness that the system was triggering after certain minutes of exposure.

The rest of the paper is structured as follows. Section 2 provides a brief overview of similar systems and Section 3 presents an overview of our system. Sections 4 First prototype (Neurosky), 5 Second prototype (Emotiv) present the two experimental prototypes that were implemented including evaluation results. Finally, Section 7 presents conclusions and future work.