Cortical correlate of spatial presence in 2D and 3D interactive virtual reality: An EEG study

Abstract

The present study is the first that examined neuronal underpinnings of spatial presence using multi-channel EEG in an interactive virtual reality (VR). We compared two VR-systems: a highly immersive Single-Wall-VR-system (three-dimensional view, large screen) and a less immersive Desktop-VR-system (two-dimensional view, small screen). Twenty-nine participants performed a spatial navigation task in a virtual maze and had to state their sensation of “being there” on a 5-point rating scale. Task-related power decrease/increase (TRPD/TRPI) in the Alpha band (8–12 Hz) and coherence analyses in different frequency bands were used to analyze the EEG data. The Single-Wall-VR-system caused a more intense presence experience than the Desktop-VR-system. This increased feeling of presence in the Single-Wall-VR-condition was accompanied by an increased parietal TRPD in the Alpha band, which is associated with cortical activation. The lower presence experience in the Desktop-VR-group was accompanied by a stronger functional connectivity between frontal and parietal brain regions indicating that the communication between these two brain areas is crucial for the presence experience. Hence, we found a positive relationship between presence and parietal brain activation and a negative relationship between presence and frontal brain activation in an interactive VR-paradigm, supporting the results of passive non-interactive VR-studies.

Introduction

The concept of spatial presence that is defined as “the sense of being in an environment” (Steuer, 1995) has received great attention in connection with virtual reality (VR) applications. Virtual environments are used in education, entertainment, training situations, psychotherapy, and medicine, including surgical training, assessment and rehabilitation of behavioral and neurological disorders (Lewis and Griffin, 1997). For all these applications virtual reality has become an important tool for simulating real world events and scenarios. The success of virtual reality applications is thought to be associated with the subjective presence experience. The greater the degree of presence the greater the chance that participants will behave in a virtual environment in the same or in a similar manner than in an adequate real world setting. Additionally, a higher sense of presence can foster the transfer of knowledge acquired in the virtual environment to corresponding real world behavior (Slater et al., 1996). Because of the rapid rising of VR application fields, it is important to understand brain activation patterns underlying the spatial presence experience, which are to date scarcely investigated.

Various studies examined different factors that enhance the subjective feeling of presence. Slater and Usoh (1993) differentiated between internal and external factors that may contribute to a users feeling of presence within a virtual environment. The internal factors deal with how the individual experiences in virtual environments are processed internally. Most literature has placed an emphasis on the external and more objective factors that include technology related aspects of virtual realities. There is evidence that a larger screen size, a three-dimensional (3D) or stereoscopic presentation, image motion or a realistic and detailed design enhance the sense of being in a virtual environment (Slater et al., 1995, Freeman et al., 2000, Ijsselsteijn et al., 2001, Lee and Kim, 2008, Slater et al., 2009).

In the present study, these well established technology related factors were used to generate two different immersive virtual reality systems, a highly immersive Single-Wall-VR-system (3D, large screen size) and a less immersive Desktop-VR-system (2D, smaller screen size). According to the literature, these two VR-setups should lead to different presence experiences. Ijsselsteijn et al. (2001) found a significant effect of screen size on subjective presence ratings. Viewing rally car scenarios on a large projection display lead to an increased sense of presence compared to viewing these scenarios on a 20-inch screen. They argued that the screen size enhances the psychological impact of motion stimuli, because a larger portion of peripheral vision is being simulated. Freeman et al. (1999) found that the presence experience is enhanced during stereoscopic stimulus presentation (3D). They argue that in a stereoscopic view more sensory information, like information about the depth structure, is available compared to a monoscopic view (2D).

There is a growing interest in neuronal underpinnings of presence although there are to date only a few studies investigating the neurophysiological correlates of spatial presence. Using electroencephalogram (EEG) recordings, Baumgartner et al. (2006) were the first to examine cortical activity during different spatial presence experiences in virtual reality. In their study, the participants sat in front of a conventional computer screen and watched passively different virtual roller coaster scenarios. In a low presence condition, the subjects viewed a roller coaster cab driving through a horizontal roundabout track, whereas in a high presence condition, the subjects viewed a roller coaster ride with spectacular ups, downs, and loops. After the virtual roller coaster rides the participants had to rate their feeling of presence on a rating scale. An increased spatial presence experience was accompanied by an increased event-related desynchronization (ERD) in the Alpha band (8–13 Hz) reflecting increased cortical activation (Pfurtscheller, 1989) in parietal brain areas known to be involved in spatial navigation. A subsequent fMRI study with the same virtual roller coaster scenarios showed that presence is associated with activation of a distributed network (Baumgartner et al., 2008). This network includes the dorsal and ventral visual stream, the parietal cortex, the premotor cortex, mesial temporal areas, the brainstem, the thalamus and the dorsolateral prefrontal cortex (DLPFC). Particularly, the DLPFC seems to be a key node of this network. Hemodynamic responses in the DLPFC showed a strong negative correlation with the subjective feeling of presence in the virtual roller coaster scenarios. Additionally, they found a negative connectivity between activity in the right-sided DLPFC and parietal brain areas (Baumgartner et al., 2008).

Summing up, these prior neurophysiological studies revealed two brain areas which might be crucial for the presence experience in VR: The parietal lobe and the dorsolateral prefrontal cortex (DLPFC). Additionally, there is evidence that the connectivity between these two brain areas might play an important role in the presence experience.

Increased parietal brain activation is associated with an increased presence experience. Generally, the parietal lobe is associated with spatial processing and navigation, it provides an egocentric (body-centered) representation of space and is involved in visuo-motor control (Maguire et al., 1998, Maguire et al., 1999). In spatial tasks, the Alpha (8–12 Hz) and Theta (4–8 Hz) frequency bands are the most extensively studied EEG oscillations. The use of an egocentric spatial reference frame (self-centered, body-centered) is associated with a relative Alpha desynchronization in parietal brain areas, whereas the use of an allocentric spatial reference frame (environment-centered, world-centered) displays increased Alpha within cortical areas along the ventral pathway, including the temporal lobe (Plank et al., 2010). Theta power increases during spatial navigation, especially during processing of spatial cues or landmarks (Kahana et al., 1999, Kober and Neuper, 2011). Presence is also defined as egocentric spatial experience of virtual environments. Therefore, it is likely that parietal brain regions play an important role in this experience because these brain structures are involved in generating an egocentric view by integrating information from different sensory modalities to form a coherent representation of space coded in a body centered reference frame. Hence, such an egocentric view might be essential for the spatial presence experience in VR (Baumgartner et al., 2008, Jäncke et al., 2009). Baumgartner et al. (2006) maintained that the strength of activation in these parietal areas is important in determining spatial presence experience.

Increased activation in the DLPFC is associated with a decreased feeling of presence. The main function of the prefrontal cortex (PFC) is one of ‘executive control’, like regulatory and inhibitory control over emotions and behavior (Aron et al., 2004, Ridderinkhof et al., 2004, Garavan et al., 1999). Additionally, the DLPFC is highly anatomically connected to parietal brain regions (Croxson et al., 2005). Hence, Jäncke et al. (2009) postulated that the DLPFC might modulate or down-regulate the activity of parietal brain areas and consequently the associated experience of presence. Supporting this hypothesis, Baumgartner et al. (2008) found a negative connectivity between activity in the right-sided DLPFC and parietal brain areas, indicating that the DLPFC exercises control over parietal regions. Given the specific role played by parietal brain areas in egocentric processing of the visual environment, it is conceivable that the DLPFC is recruited as part of a strategy for regulating presence experience by constraining the egocentric processing of the VR stimulus display (Baumgartner et al., 2008). It can also be assumed that by increasing the activation in the dorsal visual stream during strong presence experience with simultaneous diminished activation in the DLPFC, the brain prepares actions in the virtual world as if the brain actually responds to real-life situations. There is evidence that the dorsal visual stream and the connected parieto-frontal areas are strongly involved in action and movement control. Hence, the stronger the VR users are involved in the virtual world, the stronger they plan to act within the virtual environment (Jäncke et al., 2009). Furthermore, Baumgartner et al. (2006) showed that children reported higher spatial presence experiences and demonstrated a different frontal activation pattern, compared to adults. In contrast to adults, children showed a decreased frontal brain activity. The authors argued that the increased spatial presence experience in children may result from the not fully developed control functions of the frontal cortex. Hence, in children the DLPFC could not down-regulate the activity of parietal brain areas and consequently children experienced a higher sense of presence compared to adults (Baumgartner et al., 2006). These findings give rise to the assumption that there might be a fronto-parietal network, which is associated with the subjective feeling of presence in virtual environments. The DLPFC seems to control the activity of this fronto-parietal network and consequently the resulting feeling of presence.

It remains unclear if the brain activation patterns found in non-interactive passive viewing VR-conditions during increased spatial presence (Baumgartner et al., 2006, Baumgartner et al., 2008) can be found in interactive virtual environments, too. There is evidence that the experience of presence becomes more convincing as media becomes more interactive (Ijsselsteijn and Riva, 2003). Additionally, brain activation patterns may change if one can freely move around or has to solve cognitive problems, which would for example lead to activation in the motor and premotor cortex or to activation in the short and long-term memory system respectively. Hence, the question if the content of an interactive VR evokes similar frontal and parietal brain activation than passive viewing conditions is not answered yet (Baumgartner et al., 2006). The present study pursued this question. We examined the relationship between spatial presence and cortical activation using multi-channel EEG in an interactive virtual environment, where the subjects could move freely through VR. The feeling of presence and its neuronal underpinnings were examined in two different immersive VR-systems. Manipulation of technology related factors (Single-Wall-VR-system: 3D, large screen size; Desktop-VR-system: 2D, smaller screen size) was used to guarantee that the two VR-conditions lead to different presence experiences. According to the literature, a 3D view combined with a larger screen size should enhance the feeling of presence more than a 2D view combined with a smaller screen size (Freeman et al., 1999, Ijsselsteijn et al., 2001).

Concerning the neuronal underpinnings of presence in an interactive VR, we formulated the following hypotheses: If an enhanced spatial presence is accompanied by an increased parietal activation, the parietal activity should be higher in the more immersive Single-Wall-VR-condition than in the less immersive Desktop-VR-condition. Hence, in the Single-Wall-VR-condition there should be an increased task-related power decrease (TRPD) in the Alpha band (8–12 Hz) at parietal electrode sites, which is associated with cortical activation (Pfurtscheller, 1989), compared to the Desktop-VR-condition. Additionally, there should be a positive correlation between TRPD in the Alpha at parietal sites and the subjective presence rating, especially in the more immersive Single-Wall-VR-condition. On the other hand, if an increased activation in the DLPFC is associated with a decreased feeling of presence, there should be a negative correlation between TRPD in the Alpha at frontal sites and the subjective presence rating, especially in the less immersive Desktop-VR-condition. For testing the fronto-parietal network hypothesis, EEG coherence analyses were applied to assess functional brain connectivity between frontal and parietal brain areas. Especially, fronto-parietal coherence in the Theta (4–8 Hz) and Alpha (8–12 Hz) band is linked to frontal executive functions, which might play a role in presence experience. An increase of fronto-parietal coherence in these two frequency bands should mirror the activation of a fronto-parietal network (Sauseng et al., 2005). If the DLPFC controls parietal brain activity and consequently the presence experience, a decreased presence experience should be accompanied by an increased functional connectivity between frontal and parietal brain regions. Hence, EEG coherence should be higher in the Desktop-VR-condition than in the Single-Wall-VR-condition, indicating a stronger fronto-parietal connectivity in the less immersive Desktop-VR-condition.