Abstract
Brain computer interfaces (BCIs) for severe stroke motor rehabilitation aim to ‘close the loop’ between attempted motor commands and sensory feedback by providing supplemental sensory information when individuals successfully establish specific brain patterns. However, previous stroke BCIs have typically employed feedback techniques with minimal biological relevance, making them difficult and unintuitive to control. To address this, we created a novel BCI that provides biologically-relevant neurofeedback in virtual reality using a head-mounted display (HMD-VR). The purpose of this experiment was to examine whether neurofeedback in HMD-VR improves BCI performance compared to the same neurofeedback presented on a normal computer screen. Twelve healthy adults were asked to control a virtual arm by imagining right hand movements, which was measured via electroencephalography (EEG) as desynchronized sensorimotor rhythms (8-30 Hz) in the left motor cortex. Participants performed two blocks of 30 trials, one for each condition (Screen, HMD-VR), counterbalanced across participants. The neurofeedback consisted of a virtual arm that moved towards or away from different targets based on the real-time EEG activity (e.g., sensorimotor desynchronization moved the arm towards the target). After completing each block, participants were asked questions relating to their sense of presence and embodiment in each environment. We found that, while participants showed similar performance on the BCI when performing the task in either environment, there was a positive correlation between performance and reported levels of embodiment, only in HMD-VR. Specifically, participants had more control over the virtual arm in HMD-VR when they reported higher levels of spatial embodiment. Furthermore, participants reported higher levels of spatial embodiment in HMD-VR compared to the computer screen. These results suggest that HMD-VR is capable of increasing levels of embodiment compared to a normal screen environment, and that increased levels of embodiment may improve performance uniquely in the HMD-VR environment. Future work will examine the effects of HMD-VR BCI on motor rehabilitation in a stroke population.
1 Introduction
Stroke is a leading cause of adult long-term disability, and despite intense physiotherapy, up to two-thirds of stroke survivors never fully recover (Langhorne et al., 2009; Mozaffarian et al., 2016). Individuals with severe motor impairments following stroke show the poorest outcomes as they are unable to actively participate in many aspects of motor rehabilitation (Kwakkel et al., 2003). At a neural level, this may result in a lack of reinforcement for potentially beneficial motor commands due to the lack of positive feedback for motor-related brain activity (Ramos-Murguialday et al., 2013).
Previous research has examined ways to actively engage the damaged motor cortex in the absence of volitional movement. One primary way to do this is through the action observation network (AON). The AON consists of motor-related regions in the brain that are active during both the performance of an action and simply during the observation of an action. This network is a feasible way to stimulate cortical motor regions in the absence of volitional movement (Garrison et al., 2010, 2013). The AON is active when stroke patients observe a limb that corresponds to their own affected limb (Garrison et al., 2013). Related, action observation therapy, in which patients observe actions that correspond to their paretic limb, has been shown to improve motor rehabilitation in individuals with severe motor impairments (Franceschini et al., 2012).
Another way to engage activity in the damaged motor cortex in individuals with severe motor impairments is through neurofeedback with brain computer interfaces (BCIs). BCI-based neurofeedback uses sensory feedback from biological activity in the brain (e.g., as measured with electroencephalography (EEG)) to control a robotic or computerized device (e.g., movement of an object on a computer screen). BCIs designed for severe stroke rehabilitation attempt to ‘close the loop’ between motor commands and sensory feedback by providing supplemental sensory information when individuals successfully establish specific brain patterns. However, these devices traditionally employ feedback techniques with minimal biological relevance, such as using an individual’s brain activity to modulate a thermometer or move a ball (Liew et al., 2016; Wang et al., 2018). In doing so, this may create a dual-task paradigm for the participant, in which they need to modulate sensorimotor brain activity, typically accomplished via motor imagery, but also need to look at a visual feedback that interferes with the motor imagery. This creates an unintuitive situation in which participants may sometimes close their eyes in order to conduct the motor imagery, and then open them every so often to see the change in neurofeedback.
To address this, we created a brain computer interface for severe stroke called REINVENT (Rehabilitation Environment using the Integration of Neuromuscular-based Virtual Enhancements for Neural Training) that can take brain (EEG) and/or muscle (EMG) signals indicating an attempt to move and provide neurofeedback of an individual’s virtual arm moving in head-mounted virtual reality (HMD-VR). In this way, elements of action observation combine with neurofeedback, effectively removing the dual task element. Since the feedback is integrated with action observation, participants can simply think about making their own arm move and watch feedback of the virtual arm move.
Furthermore, the addition of head-mounted virtual reality (HMD-VR) is thought to provide greater immersion and embodiment compared to previous screen-based BCIs. Studies have shown that embodiment of a virtual body can occur in virtual reality and that the observation of a virtual body in the first person perspective is enough to induce a strong feeling of embodiment of the virtual body’s actions (Banakou et al., 2013; Kilteni et al., 2012, 2013; Osimo et al., 2015; Yee and Bailenson, 2007). The behavior of individuals has been shown to conform to that of a digital self-representation, such as overestimating object sizes after an adult has been given a virtual child body (Banakou et al., 2013) or exhibiting a reduction in implicit racial bias when given a body of a different race (Banakou et al., 2016). Initially coined the Proteus Effect (Yee and Bailenson, 2007), this sense of embodiment that arises from viewing a virtual limb has the potential to alter one’s own neurophysiology and behavior. Related, observing the actions of virtual limbs in virtual reality have been shown to increase sensorimotor activity (Leeb et al., 2007; Pavone et al., 2016; Prochnow et al., 2013). By replacing the affected limbs of individuals with severe motor impairments with a healthy (virtual) arm controlled by their own brain activity, individuals may be able to improve control of their virtual limb while simultaneously seeing changes in their own physical behavior.
We designed REINVENT as a BCI for individuals with severe motor impairments after stroke. However, before exploring the effectiveness of this device with a stroke population, we first examined whether providing neurofeedback in HMD-VR improves BCI performance compared to receiving the same neurofeedback on a computer screen in healthy adults. We further examined whether the level of embodiment induced by HMD-VR or the computer screen relate to each individual’s performance on the BCI. As embodiment plays an important role in increasing sensorimotor activity and HMD-VR induces high levels of embodiment, we predicted that participants would show better BCI performance in an HMD-VR environment compared to a computer screen, and that improved performance would be related to increased embodiment.
2 Materials and Methods
2.1 Participants
Twelve healthy participants were recruited for this experiment (7 females/ 5 males; age: M = 24.4 years, SD = 2.7 years). Eligibility criteria included healthy, right handed individuals and informed consent was obtained from all participants. Eight participants reported being naïve to head mounted virtual reality; the four participants with previous use of head mounted virtual reality reported using the device no more than four times. The experimental protocol was approved by the University of Southern California Health Sciences Campus Institutional Review Board and performed in accordance with the 1964 Declaration of Helsinki.
2.2 REINVENT hardware, software and data integration
The REINVENT system is described in more detail in Spicer et al., 2017. Briefly, REINVENT (Figure 1A) is a brain computer interface (BCI) that is composed of four main components: electroencephalography (EEG), electromyography (EMG), an inertial measurement unit (IMU), and a head-mounted virtual reality (HMD-VR) system. Custom software is used to control the BCI and provide users with real-time feedback of a virtual arm. EEG signals were recorded from electrodes of interest over the left motor cortex (i.e., C1, C3, and CP1, based on the international 10-20 system) with the both ear lobes used as the reference electrodes, and sent to the REINVENT software. Data processing occurred online as a virtual arm moves in response to sensorimotor desynchronization, measured as a decrease in amplitude of the combined electrodes computed between the frequency ranges of 8-30 Hz.
2.2.1 Electroencephalography (EEG) and electromyography (EMG)
The EEG/EMG component of REINVENT is composed of hardware from OpenBCI (www.openbci.com), a low-cost solution for measuring brain and muscle activity. The EEG component consists of reusable dry EEG electrodes and the EMG component consists of snap electrode cables connected to mini disposable gel electrodes (Davis Medical Electronics, Inc.). Both EEG and EMG wires were connected to a 16-channel, 32-bit v3 processor (Cyton + Daisy Biosensing OpenBCI Board) and sampled at 125 Hz.
Twelve EEG locations based on the international 10-20 system and concentrated over the prefrontal and motor cortex was used to record brain activity (F3, F4, C1, C2, C3, C4, CP1, CP2, CP5, CP6, P3, and P4). Ground and reference electrodes were located at the right and left earlobes, respectively. For the neurofeedback, the sum desynchronization from C1, C3 and CP1, representing the left motor network, was used to drive the movement of a virtual right arm towards a target arm. EMG was recorded from four electrodes placed on the wrist flexors and extensors on the muscle bellies of the right forearm, with a reference electrode on the bony prominence of the elbow. In the current experiment, muscle activity from EMG was collected but not analyzed or reported.
2.2.2 Arm movement
To foster a sense of embodiment between the participant and the virtual arm, the participant’s own arm movements were recorded using two Nine Degrees of Freedom (9DOF) IMUs, with one placed on the hand and the other placed on the wrist of the right arm (Spicer et al., 2017). Before beginning the experiment, the participant’s arm was passively moved by the experimenter and the virtual representation of the arm was shown on the computer screen and in HMD-VR. In this way, a sensorimotor contingency was developed between the participant’s own arm and the virtual arm they were subsequently asked to control.
2.3 Displays
For the HMD-VR environment, we used the Oculus CV1 which includes positional and rotational tracking to display the stimuli. For the Screen environment, we used a 24.1 inch, 1920 × 1200 pixel resolution computer monitor (Hewlett-Packard) to display the stimuli. In both displays, participants observed a scene that included two virtual arms: (1) one virtual arm that represented the participant’s own arm and (2) a second virtual arm, colored in orange, that provided different target arm positions that participants were asked to move their own arm towards (Figure 1B).
2.4 Experimental design
Prior to the experiment, a resting EEG baseline of three minutes with the HMD-VR removed was recorded for each participant. Participants were instructed to keep their eyes open and fixed on a location at the center of the computer screen. For the duration of the recording, participants were asked to think about a stationary object and to stay as still as possible. The recording was used to provide the baseline EEG values for the experiment. Participants then completed three blocks of 30 trials (90 trials in total) where each block was a separate condition. The conditions were (1) controlling the virtual arm with brain activity on the computer screen (Screen), (2) controlling the virtual arm with brain activity in head-mounted virtual reality (HMD-VR), and (3) controlling the virtual arm with actual arm movements in head-mounted virtual reality (IMU). Participants completed the conditions in the following block order: Block 1 (Screen), Block 2 (HMD-VR), Block 3 (IMU); with the first two blocks being counterbalanced. In this experiment, the IMU condition was strictly to get a baseline performance during real movement; this data is briefly reported but not focused on in this paper. Before starting the experimental conditions, participants were given instructions on how to control their virtual arm (i.e., “You will see two right arms. One is orange and that is the target arm that moves to different positions. The other is your arm. We want you to move it to match the target arm’s position. You can move your arm in two ways. First, you will complete 60 trials of moving the virtual arm with just your thoughts by thinking about moving; 30 of the trials will be on the computer screen, without the head-mounted virtual reality, and 30 trials will be with the head-mounted virtual reality. Then you will complete 30 trials of moving the virtual arm using your actual arm movements.”). Instructions were repeated at the start of each block. After the completion of each EEG block (Screen, HMD-VR), a resting-EEG acquisition of three minutes was recorded while the HMD-VR was removed; participants were again instructed to keep their eyes open and fixed on the center of the screen for the duration of the recording. Figure 2 shows a detailed timeline of the experimental design.
2.4.1 Individual trials
At the start of each trial, a target arm animated a wrist extension pose in one of three target positions. Once the target arm stopped moving, participants were instructed to move their virtual arm to match the position of the target arm given the current condition (i.e., in the case of the EEG conditions (Blocks 1, 2), they were asked to think about moving; in the case of the IMU condition (Block 3), they were asked to actually move their arm to the target location). Participants had 15 seconds to reach the target arm; if the target arm was reached within this time constraint, a successful auditory tone was played, however, if the target arm was not reached, then an unsuccessful auditory tone was played. At the completion of each trial, the target and virtual arms returned to a resting baseline position.
2.5 Experimental design
Prior to the experiment, participants were given a series of standard questions about their baseline sickness level (Simulator Sickness Questionnaire; adapted from Kennedy et al., (1993) and revised by the UQO Cyberpsychology Lab, 2013). After participants completed each EEG block (Screen, HMD-VR), they were given the same simulator sickness questionnaire to examine changes following each block. Responses were reported on a 0 to 3-point scale and questions were collapsed along two main themes: Nausea and Oculo-Motor. In addition, after completing each EEG block (Screen, HMD-VR), participants were also asked questions pertaining to their overall sense of presence and embodiment in each environment. The Presence Questionnaire was adapted from Witmer and Singer (1998) and revised by the UQO Cyberpsychology Lab (2004) and asked participants a series of questions to gauge their sense of presence in each environment. Responses were reported on a 1 to 7-point scale and questions were collapsed along five main themes: Realism, Possibility to Act, Quality of Interface, Possibility to Examine, and Self-Evaluation of Performance. The Embodiment Questionnaire was adapted from Bailey et al. (2013) and Banakou et al. (2013) and asked participants a series of questions to gauge their sense of embodiment. Responses were reported on a 1 to 10-point scale and questions relating to either Self Embodiment or Spatial Embodiment were averaged to generate the two embodiment themes. Table 1 includes individual questions asked on the Embodiment Questionnaire.
2.6 Analyses
2.6.1 Resting EEG pre-processing
The signals from 3 channels, C1, C3 and CP1, were used to record resting EEG in the left motor network during a 3-minute eyes open session acquired at baseline and after each EEG block (Screen, HMD-VR). Resting EEG data was analyzed using BrainVision Analyzer 2.1 (Brain Products, Germany). Given that the neurofeedback was measured in the 8-30 Hz frequency window, we also constricted the resting EEG analyses to this frequency band. Any data with artifacts, including movement, eye blinks and high frequency noise, in any of the 3 channels were excluded using semi-automatic artifact rejection and visual inspection. Subsequently, the data were segmented into epochs of 1 second and artifact-free epochs were extracted through a Hanning window. Power spectra was then calculated via Fast Fourier Transform (FFT) and expressed as the absolute power (µV2) of the 8-30 Hz band for each participant at baseline, post-Screen, and post-HMD-VR.
2.6.2 Statistical Analysis
Statistical analysis for BCI performance, subjective experience from questionnaires, and resting EEG was analyzed using the statistical package R (3.2.2, The R Foundation for Statistical Computing, Vienna, Austria). To assess statistical differences in performance and subjective experience between the two EEG conditions (Screen, HMD-VR), a two-sample paired t-test was performed on each measure between conditions and across participants. Means (M) and standard deviations (SD) are reported for each measure. Furthermore, we correlated BCI performance with resting EEG and also correlated BCI performance with the Presence Questionnaire and the Embodiment Questionnaire using a spearman’s rank correlation. For the Presence Questionnaire, p-values of p < 0.01 were considered statistically significant (corrected for 5 comparisons as correlations were run across the 5 themes) and for the Embodiment Questionnaire, p-values of p < 0.025 were considered statistically significant (corrected for 2 comparisons as correlations were run across the 2 themes). Lastly, to assess statistical differences in resting EEG, a paired t-test was performed between post-Screen and post-HMD-VR on absolute power of the 8-30 Hz band in the left motor network after correction (i.e., subtracting the baseline absolute power). All participants completed the IMU condition with 100% accuracy and therefore this condition is not included in this analysis.
3 Results
3.1 Differences in subjective experience between Screen and HMD-VR
There were no significant differences between reports of simulator sickness for the Screen (Nausea: M = 0.33, SD = 0.98; Oculo-Motor: M = 0.83, SD = 1.19) and the HMD-VR (Nausea: M = 0.17, SD = 0.83; Oculo-Motor: M = 0.83, SD = 0.94) conditions (Nausea: t(11) = 1.48, p = 0.166; Oculo-Motor: t(11) = 0, p = 1). These results suggest that using an HMD-VR BCI does not cause additional adverse effects beyond using a computer screen in healthy individuals. In addition, there were no significant differences between reports of presence in the two conditions (Realism: t(11) = −1.95, p = 0.078, Screen: M = 30.0, SD = 6.35, HMD-VR: M = 33.0, SD = 6.40; Possibility to Act: t(11) = −1.37, p = 0.199, Screen: M = 18.17, SD = 3.7, HMD-VR: M = 19.92, SD = 4.19; Quality of Interface: t(11) = −0.62, p = 0.548, Screen: M = 12.83, SD = 3.07, HMD-VR: M = 13.42, SD = 2.97; Possibility to Examine: t(11) = −2.01, p = 0.070, Screen: M = 13.17, SD = 2.59, HMD-VR: M = 14.92, SD = 2.27; Self-Evaluation of Performance: t(11) = −1.24, p = 0.241, Screen: M = 10.0, SD = 1.95, HMD-VR: M = 11.00, SD = 2.13). There was also no significant difference between reports of Self Embodiment in the two conditions (t(11) = −0.10, p = 0.922, Screen: M = 5.39, SD = 1.17, HMD-VR: M = 5.43, SD = 1.76). However, we did find a significant difference in report of Spatial Embodiment between the Screen and HMD-VR conditions (t(11) = −3.77, p = 0.003, Screen: M = 3.60, SD = 2.04, HMD-VR: M = 5.35, SD = 2.00) where individuals in the HMD-VR condition reported higher levels of spatial embodiment.
3.2 Differences in BCI performance and time to complete trials between Screen and HMD-VR
The proportion of correct trials completed was similar between the two conditions (Figure 3; t(11) = −0.46, p = 0.656, Screen: M = 80.95%, SD = 9.1%, and HMD-VR: M = 83.33%, SD = 14.9%). These results suggest that participants seem to have an equal amount of control of their sensorimotor activity when submersed in either the HMD-VR BCI environment or viewing on a computer screen BCI.
The time to complete each of the successful trials was also similar between the two conditions (Figure 4; t(11) = 0.54, p = 0.597, Screen: M = 4.347 s, SD = 1.17 s; HMD-VR: M = 3.996 s, SD = 2.41 s). These results suggest that when participants were able to control the virtual arm with their brain activity, the efficiency of control was similar whether viewing the arm in the HMD-VR environment or on a computer screen.
3.3 Correlations between BCI performance and subjective experience in Screen and HMD-VR
To determine if participants’ level of presence or embodiment had an influence on their performance in either the computer screen or HMD-VR environments, we correlated each participant’s performance on the conditions with their respective responses on the Presence and Embodiment Questionnaires. For the HMD-VR environment, we found a positive correlation for Realism where participants who reported higher levels of realism had a higher level of performance in HMD-VR; however, this did not survive multiple comparisons across the 5 themes (Figure 5; rs = 0.58, p = 0.046). Separately, we found a significant positive correlation for Spatial Embodiment, where participants who reported higher levels of spatial embodiment had a higher level of performance in HMD-VR; this survived multiple comparisons across the 2 themes (Figure 6; rs = 0.66, p = 0.020). In contrast, for the Screen condition, we found no significant correlations across the five themes on the Presence Questionnaire or across the two themes on the Embodiment Questionnaire. As seen in Figure 7, individuals who had higher embodiment in HMD-VR (shown in yellow, pink) showed greater BCI performance in HMD-VR compared to the Screen condition than those with less embodiment in HMD-VR (shown in purple). These results suggest that the higher the sense of realism or spatial embodiment individuals have in HMD-VR, the more likely they are to have a higher BCI performance. Table 2 lists the correlations for each of the themes on the Presence and Embodiment Questionnaires.
3.4 Exploratory analysis of correlations between BCI performance and individual embodiment questions
As an exploratory analysis of embodiment, we then performed correlations on each question from the Embodiment Questionnaire. Six questions relate to Self Embodiment and four questions relate to Spatial Embodiment; thus, we considered p-values of p < 0.008 statistically significant for Self Embodiment questions (corrected for 6 comparisons) and considered p-values of p < 0.0125 statistically significant for Spatial Embodiment questions (corrected for 4 comparisons). For Self Embodiment, we found a significant positive correlation for Amount of Control where participants who reported higher levels of control had a higher level of performance in HMD-VR (Supplemental Figure 1; rs = 0.77, p = 0.003). For Spatial Embodiment, we found a positive trend for Location where participants who reported a higher rating of actually being located in the virtual environment had a higher level of performance in HMD-VR (Supplemental Figure 2A; rs = 0.65, p = 0.021). We also found a positive trend for Real World where participants who reported a higher rating of the virtual environment seeming similar to the real world had a higher level of performance in HMD-VR (Supplemental Figure 2B; rs = 0.69, p = 0.013). In contrast, for the Screen condition, we found no significant correlations or trends across the 6 Self Embodiment questions or across the 4 Spatial Embodiment questions. Supplemental Table 1 lists the correlations for each of the questions on the Embodiment Questionnaire.
3.5 Resting EEG between post-Screen and post-HMD-VR
There was a non-significant difference in absolute power (8-30 Hz band) in the left motor network between the post-Screen and the post-HMD-VR conditions normalized to baseline (t(7) = −2.09, p = 0.075; post-Screen changes from baseline: −0.20 ± 20.20 µV2; post-HMD-VR changes from baseline: +21.43 ± 45.61 µV2). This suggests a trend towards greater sensorimotor desynchronization at rest following HMD-VR-based neurofeedback sessions, although this was not significant.
3.6 Correlations between BCI performance and resting EEG
We ran correlations to determine whether absolute power of the resting motor network at baseline predicted performance on either the computer screen or the HMD-VR environment. The absolute power of the left motor network resting EEG at baseline did not predict how participants performed in either the Screen (rs = 0.085, p = 0.240) or HMD-VR (rs = 0.073, p = 0.863) conditions. Furthermore, there were no significant correlations between the level of performance and the absolute power of the left motor network resting-EEG for either the Screen (rs = −0.400, p = 0.326) or HMD-VR (rs = 0.220, p = 0.601) conditions. This suggests that baseline resting motor activity does not predict performance in either HMD-VR-or screen-based neurofeedback sessions.
4 Discussion
The current pilot study examined whether neurofeedback from a motor-related brain computer interface provided in HMD-VR could lead to better BCI control compared to the same neurofeedback provided on a standard computer screen. We examined whether healthy individuals showed similar BCI performance on a computer screen versus in head-mounted virtual reality and whether the resulting level of presence and embodiment in each environment had any effect on participants’ BCI performance. Overall, we found that, while participants showed similar performance on the BCI at the group level when performing the task in either environment, there was a positive correlation between performance and reported levels of embodiment only in the HMD-VR environment.
4.1 Similar BCI performance between a computer screen and HMD-VR
Regardless of environment (Screen, HMD-VR), we found that on average, individuals were able to accurately modulate their brain activity to successfully control a virtual arm on over 80 percent of trials. These results suggest that neurofeedback based on action observation, using biologically-relevant stimuli, can occur either on a computer screen or in head-mounted virtual reality. This is in line with previous literature showing that passive action observation in either environment increases sensorimotor brain activity (Leeb et al., 2007; Pavone et al., 2016; Prochnow et al., 2013), and extends these findings to show that such evoked activity can be actively controlled in a BCI. Given that previous literature has also shown similar activation of ipsilesional sensorimotor regions during action observation in individuals after stroke, future work might examine whether this type of neurofeedback, on either a computer screen or in HMD-VR, can be similarly controlled by individuals after stroke.
4.2 Higher embodiment in HMD-VR compared to a computer screen
After performing the task in each condition (Screen, HMD-VR), participants reported having higher levels of spatial embodiment in HMD-VR compared to the computer screen. This agrees with previous research showing that HMD-VR is effective for inducing embodiment (Osimo et al., 2015; Slater and Sanchez-Vives, 2016). However, while it has been intuitively suggested that viewing a virtual body in HMD-VR should induce greater embodiment than viewing the same virtual body on a computer screen, there has been little empirical evidence to demonstrate this. Here, we address this gap by providing evidence that HMD-VR does in fact increase embodiment compared to a computer screen.
4.3 Higher embodiment leads to better BCI performance uniquely in HMD-VR
In line with our hypothesis, we show that increased embodiment in HMD-VR is positively correlated with better BCI performance. Importantly, this finding is only seen in the HMD-VR condition, and not in the Screen condition, and occurred even though there was a range of scores in both environments and even though the same individuals completed both conditions. Specifically, the embodiment level of an individual in an HMD-VR environment seemed to affect BCI performance while the embodiment for the same individual on a computer screen does not seem to have similar affects. This is consistent with previous research where embodiment has been shown to lead to neurophysiological and behavioral changes based on the virtual body’s characteristics, such as overestimating object distances after given an elongated virtual arm in HMD-VR (Kilteni et al., 2012). These findings are important because they suggest that embodiment in HMD-VR has the potential to improve an individuals’ BCI control, beyond their normal capabilities on a computer screen. Indeed, we found that individuals with greater embodiment in HMD-VR also performed better in HMD-VR than in the Screen condition. This suggests that if individuals were to hit a ceiling effect controlling the BCI on a computer screen, they might be able to show greater improvements, beyond this ceiling, in HMD-VR.
4.4 Clinical implications
This work also has implications for clinical populations, such as individuals with stroke. Specifically, these findings suggest that the use of HMD-VR with biologically-relevant neurofeedback may improve patients’ BCI control and potentially their recovery, beyond what might be seen with traditional screen-based BCIs. As previous brain computer interfaces have been shown to have a positive change on muscle and sensorimotor brain activity in post-stroke individuals, even when using screen-based environments (Ono et al., 2014), we anticipate that embodiment in HMD-VR may lead to even greater improvements. Future work might explore whether additional measures of embodiment, administered prior to HMD-VR BCI use, could predict embodiment and related performance, during HMD-VR BCI use. If so, these “pre-assessments” of embodiment potential could be used to predict and personalize BCI therapy. Importantly, this measure of embodiment may be more predictive of performance than a neural measure, such as baseline resting EEG. However, as this data is preliminary, more data is needed to explore this hypothesis.
4.5 Limitations
Our study has two main limitations. First was the limited sample size of 12 individuals. However, as this study was a pilot study aimed to assess whether HMD-VR provided any advantages for BCI control over a normal computer screen, we believe that these novel preliminary results will contribute to the development of future large-scale BCI studies, which could examine these effects with greater robustness. In addition, despite the small sample, we found relatively consistent effects for spatial embodiment across all individuals, suggesting a true effect.
A second limitation is that here, we studied healthy individuals who used the BCI only briefly (30 trials per condition). This is notable as the effects observed may be smaller than those of a clinical population, who may have more room to improve, or in healthy individuals who use the BCI for a longer period of time. Specifically, the healthy individuals in our study showed, on average, 80% accuracy with the BCI within a short time frame, which may reflect their intact sensorimotor control. However, individuals with stroke may start with lower scores and have greater room for improvement due to damage to these same networks. Future work may examine extended training with the HMD-VR environment to see if it is possible for individuals to improve beyond their current levels with greater time in the environment, as well as the effects of embodiment on BCI performance in individuals with stroke, which may provide a greater range of abilities and thus greater potential effects with immersive virtual reality.
4.6 Conclusions
This preliminary work suggests that individuals have higher levels of spatial embodiment when given immersive virtual reality-based neurofeedback compared to the neurofeedback displayed on a computer screen. Furthermore, this increased sense of embodiment in immersive virtual reality neurofeedback has the potential to improve BCI performance in healthy individuals over their performance on a screen. HMD-VR may provide a unique medium for improving BCI performance, especially in clinical settings related to motor recovery. Future work will explore ways to increases presence and embodiment in immersive head-mounted virtual reality and examine these effects on motor rehabilitation in a clinical stroke population.
6 Conflict of Interest
The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.
5 Acknowledgments
We thank David Saldana for assistance with recruitment and Catherine Finnegan for assistance with recruitment and initial pilot data collection.