Summary
The retrosplenial complex (RSC) plays a crucial role in spatial orientation by computing heading direction and translating between distinct spatial reference frames. While invasive studies allow investigating heading computation in moving animals, established non-invasive analyses of human brain dynamics are restricted to stationary setups. To investigate the role of the RSC in heading computation of actively moving humans, we used a Mobile Brain/Body Imaging approach synchronizing electroencephalography with motion capture and virtual reality. Data from physically rotating participants were contrasted with rotations based only on visual flow. Varying rotation velocities were accompanied by pronounced beta synchronization during physical rotation. In addition, heading computation based only on visual flow replicated alpha desynchronization in the RSC, which was absent during physical rotation. These results suggest an involvement of the human RSC in heading computation based on vestibular input and implicate revisiting traditional findings of alpha desynchronization during spatial orientation in movement-restricted participants.
Heading computation is fundamental for spatial orientation in the human and other species. The registration of moment-to-moment changes in orientation with respect to an allocentric reference direction provides information about an animal’s current heading relative to the environment. This is accomplished by the integration of vestibular, proprioceptive, and visual signals providing information about linear and angular velocity signals of the head, the relative position of the head with respect to the trunk, and information about stable aspects of the environment, respectively.1 Single cell recordings in freely behaving animals identified several brain structures involved in heading computation, including the retrosplenial cortex (RSC).2, 3 The RSC receives input from the visual system and from head direction cells in the thalamic nuclei.4 It also hosts subpopulations of heading-sensitive cells that are sentient to local features of the environment, while other cells exhibit mixed activity patterns related to both local and global heading computation.5 These findings suggest that neural activity in the RSC subserves the integration of information about the local and global environment, integrating egocentrically coded landmark cues based on sensory fusion (vision and proprioception)6 with allocentric heading information originating from the Papez circuit.7 This allows the compensation of the rotational offset between egocentric and allocentric spatial representations, routed from the parietal and medial temporal cortices, providing the necessary information for translating between both egocentric and allocentric spatial representational frames in the RSC.8
The central role of the RSC for spatial orientation in general and for heading computation specifically is supported by human imaging studies.6, 9, 10 Due to the restricted anatomical differentiation of the retrosplenial cortex (BA 29 and 30) and the adjacent posterior cingulate (BA 23 and 31), the abbreviation RSC is used here to refer to the retrosplenial complex.9 Haemodynamic changes in the RSC were shown to be associated with landmark learning11, 12 and with both global and local heading estimation.13 While functional magnetic resonance imaging (fMRI) studies provide valuable insights into the function of the RSC regarding spatial cognition, they do not allow movements of the participant in the scanner.14 This is due to the fact that fMRI studies use sensors that are too heavy to follow movements of the signal-generating source.15 Electroencephalography (EEG) studies, though utilizing lighter sensors, are considerably affected by movement-related artefacts and thus traditionally rely on stationary setups as well. However, heading computation depends on input from the vestibular organ1 indicating movement of the head and body that can be related to, among other features, the location and orientation of external information like landmarks encoded through other senses.16 Therefore, established imaging studies do not allow a recording of the very signal that is essential for heading computation, fostering cognitive processes that might not resemble the computation and use of directional heading in more natural environments.17
In the present study, we overcome previous restrictions of traditional imaging studies by investigating neural dynamics in the human RSC during heading computation in actively rotating humans. To this end we used a Mobile Brain/Body Imaging approach (MoBI)14, 18 synchronizing high-density EEG to motion capture and head-mounted virtual reality (VR). Data-driven analyses, based on blind source separation and subsequent source reconstruction, were used in order to investigate neural dynamics and their neuroanatomical origins accompanying heading computation during physical rotations in movement-unrestricted participants. The results demonstrate significant spectral modulations in a wide frequency range in the RSC during active physical rotations compared to a stationary setup that provided only visual flow.
Results
We analyzed data from 19 participants performing a spatial orientation task in two rotation conditions (see Figure 1A–B), involving i) physical rotations of the whole body (“physR”), and ii) standing in front of a desktop monitor controlling the visual flow by manually operating a joystick (“joyR”). The latter condition mimicked traditional stationary setups investigating neural dynamics underlying spatial orienting. The participants rotated on the spot in a sparse virtual environment that provided only an initial local landmark (pole). With a button press, the landmark was replaced by a sphere, which moved either to the left or to the right around the participant at a constant distance. In this outward rotation phase, participants had to follow the movement of the sphere by rotating on the spot using the respective control (physR or joyR). The sphere followed two different cosine velocity profiles along its path and stopped at varying eccentricities with respect to the participants’ initial facing direction. Upon stopping, the sphere changed its colour, and the participants were tasked with rotating back and indicating their initial heading. EEG data was decomposed into independent components (ICs) using adaptive mixture independent component analysis (AMICA).19 The approximate locations of the resultant ICs were reconstructed using equivalent dipole models, and the ICs were clustered using a repetitive k-means algorithm optimized to the RSC as a region of interest (see Methods for details).
Heading estimation is more accurate for physical rotation
Replicating previous results, the performance data showed that physR resulted in higher accuracy for heading reproduction than joyR.20, 21 A repeated measures analysis of variance (rANOVA) for the mean absolute heading error, averaged over categories of pseudo-continuous eccentricities of 3° steps centred around 45° (30–60°), 90° (75–105°), and 135° (120–150°), revealed a significant main effect of “rotation condition” (F1,18 = 33.78; p < 0.001; partial η2 = 0.65) and of “eccentricity” (F1.28,23.11 = 26.59; p < 0.001; partial η2 = 0.6), as well as a significant interaction between both factors (F1.78,31.97 = 5.75; p = 0.009; partial η2 = 0.24). Post hoc analysis of the interaction effect using the Wilcoxon signed-rank test revealed that the absolute heading error was significantly smaller for the physR than for the joyR condition in all three eccentricity categories (45°: 6.76 ± 2.77 vs. 14.4 ± 8.9; 90°: 7.79 ± 2.21 vs. 19.73 ± 10.16; 135°: 12.38 ± 4.68 vs. 23.83 ± 7.1; p < 0.001 for all comparisons). The post hoc tests also revealed an increase of absolute heading errors with increasing eccentricity in the joyR condition (p < 0.01 for all comparisons), but significant differences in the physR condition only for the most eccentric positions of 135° as compared to both 45° and 90°.
Neural activity in RSC accompanying head rotation velocity
In order to test whether the RSC processes angular movement information from the vestibular system, oscillatory amplitude differences (Hilbert-transform) in the theta (4–7 Hz), alpha (8–13 Hz), low beta (14–18 Hz), and high beta (22–30 Hz) frequency bands were compared between both physR and joyR rotation conditions for varying movement velocity (yaw orientation) during the outward rotation following the visual stimulus. For this purpose, a velocity binning procedure was applied (see Methods/Supplements for details), as previously established for single cell recordings of heading-sensitive cells in rodents22 and in the context of analyzing EEG oscillations.23 For each trial across both rotation conditions, continuous time segments were extracted from head rotation velocity onset to offset, and the data were appended to form a continuous data stream. Then, all velocity samples were sorted in ascending order, and amplitudes were averaged within 10-percentile velocity bins separately for each frequency band and IC, obtaining a single value per participant IC and velocity bin. Statistical comparison between both rotation conditions revealed only minor amplitude differences in the theta and high beta frequency band in RSC (Figure 2). In contrast, larger amplitudes during physR compared to joyR were significant for two velocity bins in the alpha band (for rather high velocities), whereas in 14–18 Hz beta oscillations the amplitude increases during physR were significant throughout the majority of velocity bins.
Event-related spectral dynamics in RSC differ between physical and visual flow only rotation
The primary interest of the present study was to investigate modulations of oscillatory neural activity during heading computation based on active body rotations or based on visual flow only. For this purpose, single-trial spectrograms were computed for all outward rotation trials. In order to account for variable eccentricities associated with different trial durations, the spectrograms were linearly time-warped to the onset of the visual stimulus and to the movement onset and movement offset at the trial end (participant’s head or joystick movement), resulting in time-warped event-related spectral perturbations (ERSP; see Figure 3). The spectral baseline was defined as the 200ms period before stimulus onset, excluding movement-contaminated trials. The power spectrum for the joyR and physR baseline time period revealed reduced 8–12 Hz alpha as well as increased 28 to 48 Hz gamma power in the RSC in the physR condition (permutation-based statistics with a p-level of 0.05; cf. Fig. 3A).
During the outward rotation, the statistical analyses revealed distinct modulations in spectral power between the baseline and task performance for each rotation condition in the time-frequency domain. Furthermore, pronounced power differences between physR and joyR were present in a wide frequency range from 3–60 Hz. Largest differences for the RSC in terms of pronounced power increases were observed in the 4–7 Hz theta range, as well as in the alpha and beta frequency ranges (8–30 Hz; cf. Fig. 3C).
Discussion
We demonstrated the modulation of neural dynamics in the human RSC during heading computation in a spatial orientation task. Heading reproduction was significantly more accurate when participants rotated with their whole body compared to the stationary setup using only a joystick, reflecting the synergistic use of idiothetic information during active movement.20 While an increase in heading error was observed with an increasing eccentricity of the outward rotation, the physical rotation of the whole body led to relatively low heading errors that increased only for outward rotations ending beyond 90 degrees relative to the initial heading. This points to general differences in the accuracy of spatial judgments based on the principal body axes, in this case in the front as compared to the back of the body.24 Notably, the multisensory fusion of information from the visual, vestibular, and proprioceptive systems allowed for accurate heading computation even in the absence of prominent landmarks.
Importantly, the observed behavioral results were accompanied by marked rotation velocity-related differences in brain dynamics. We observed significantly increased 14-18 Hz beta amplitudes during physR compared to joyR across a wide range of small to large velocities, pointing to frequency-specific velocity-based heading computation in the human RSC depending on the presence of congruent visual-vestibular input. While several recent MoBI studies demonstrated a prominent role of power modulations in beta and gamma frequency ranges during active movement in several brain regions,25, 26 so far no study has demonstrated velocity-related modulations of neural oscillations in the RSC. One possible explanation for the differential involvement of distinct frequency bands might be that amplitude modulations of beta oscillations in RSC are specifically associated with heading computation. Beta synchronization in non-motor brain regions was hypothesized to relate to stimulus-driven or endogenous (re-)activation of cortical representations, reflecting memory formation and changes in cognitive-motor sets contingent upon task demands.27 Such functions would also be relevant in the context of heading estimations being continuously updated during the rotation task. In contrast, modulations of alpha oscillations might reflect ongoing translations of spatial information from egocentric into allocentric reference frames and vice versa.28 Future studies might investigate the potential distinct roles of alpha and (lower) beta oscillations in RSC for spatial orientation in moving participants, as implied in previous studies of visual flow in stationary setups.29, 30
Analyses of the power spectrum in the RSC before onset of the rotation revealed significant differences in the alpha, high beta, and gamma frequency bands. Such a baseline difference, despite similar sensory input, lends strong support to the notion of state-dependent sensory processing in humans. In more general terms, ongoing neuronal dynamics in the brain can be modulated not only by the current behavioral state31, 32 but also by planned future behavior.33 The present state-dependent baseline differences became even more pronounced during the subsequent rotation task: Notably, ERSPs over the complete time course of the outward rotation in the joyR condition displayed pronounced the desynchronization of alpha oscillations. Significant differences in alpha oscillations during the outward rotation are in line with similar findings of traditional desktop-based EEG studies in a variety of tasks,34in which attention-related alpha band desynchronizations can be observed in topographically diverse areas while tasks requiring sensory-semantic information processing demonstrated alpha desynchronization with an occipito-parietal topography.35 Specifically, alpha desynchronization during heading computation was frequently documented in movement-restricted participants,28-30, 36, 37 a finding that we replicated here for the desktop-based rotation condition using a joystick and providing only visual flow information. This alpha desynchronization, however, was completely absent during active physical rotations, whereas pronounced broad-band increases of spectral power were present primarily in theta and beta oscillations. This unexpected modulation in (alpha) oscillatory activity might be explained by congruent efferent and reafferent sensory information in the active rotation condition, as suggested by results from single cell recordings during active and passive rotations in non-human primates. Here, a suppression of neural responses in the vestibular nuclei can be observed when monkeys actively rotated their head, as compared to being passively rotated on a chair.38 This suppression could be due to the fact that the vestibular nuclei receive vestibular afferents as well as efferent projections from diverse sensory systems.39 Consistent with the reafference principle,40 during active head movements vestibular afferent information is suppressed by the motor-proprioceptive information generated by the active movement.1 Here, the predicted sensory outcome of the head rotation (based on the efference copy of the motor command) is compared with the sensory feedback derived from proprioception and the vestibular nerves (reafference). Resulting residuals (exafference) thus differentiate between active and passive movements, providing information about the external stimulus only. Consequently, and in line with the predictive processing approach41, the frequently documented alpha desynchronization in sensory (e.g., occipital) and multimodal (e.g., parietal) cortical areas in traditional stationary EEG setups that addressed spatial orientation in movement-restricted participants might actually not reflect heading computation per se, but rather the processing of significant exafferences stemming from a discrepancy of predicted and perceived sensory input.42
Using a MoBI approach that allows unrestricted physical movement in a spatial orientation task, we demonstrated pronounced differences in spectral modulation for heading computation based on visual flow, as compared to self-generated movements that allow utilizing idiothetic information from the visual, vestibular, and proprioceptive senses for heading computation. The results revealed the modulation of 14– 18 Hz beta oscillations in the human RSC across wide ranges of velocities, implying the relevance of velocity information from the vestibular system for heading computation. Finally, alpha desynchronization during spatial orientation tasks in traditional desktop-based setups that provide only visual flow information might point to sensory mismatch processing in the RSC rather than heading computation or translation of spatial information into egocentric and allocentric reference frames itself.
Methods summary
Methods, along with any additional extended data display items, are available in the online version of the paper; references that are cited in this section appear only in the online version.
Participants
Data were collected from 20 healthy adults (11 females) with a mean age of 30.25 years (SD = 7.68, ranging from ages 20 to 46) who received 10€/h or course credit for compensation. All participants reported normal or corrected to normal vision and no history of neurological disease. Eighteen participants reported being right-handed (two left-handed). To control for the effects of different reference frame proclivities on neural dynamics, the online version of the spatial reference frame proclivity test (RFPT43, 44) was administered prior to the experiment. Participants had to consistently use an ego-or allocentric reference frame in at least 80% of their responses. Of the 20 participants, nine preferentially used an egocentric reference frame, nine used an allocentric reference frame, and two used a mixed strategy. One participant (egocentric reference frame) dropped out of the experiment after the first block due to motion sickness and was removed from further data analyses. The reported results are based on the remaining 19 participants. The experimental procedures were approved by the local ethics committee (Technische Universität Berlin, Germany) and all participants signed a written informed consent in accordance with the Declaration of Helsinki.
Experimental Design and Task
Participants performed a spatial orientation task in a sparse virtual environment (WorldViz Vizard, Santa Barbara, USA) consisting of an infinite floor granulated in green and black (see figure 1B and complementary video 1). The experiment was self-paced and participants advanced the experiment by starting and ending each trial with a button press using the index finger of the dominant hand. A trial started with the onset of a red pole, which participants had to face and align with. Once the button was pressed the pole disappeared and was immediately replaced by a red sphere floating at eye level. The sphere automatically started to move around the participant along a circular trajectory at a fixed distance (30m) with one of two different velocity profiles (see Supplement for a description of the cosine functions). Participants were asked to rotate on the spot and to follow the sphere, keeping it in the center of their visual field (outward rotation). The sphere stopped unpredictably at varying eccentricity between 30° and 150° and turned blue, which indicated that participants had to rotate back to the initial heading (backward rotation). When participants had reproduced their estimated initial heading, they confirmed their heading with a button press and the red pole reappeared for reorientation. To ensure that the floor could not be used as an external landmark during the trials, it was faded out, turned randomly, and faded back in after each outward and backward rotation.
The participants completed the experimental task twice, using i) a traditional desktop 2D setup (visual flow controlled through joystick movement; “joyR”), and ii) equipped with a MoBI setup (visual flow controlled through active physical rotation with the whole body; “physR”). The condition order was balanced across participants. To ensure the comparability of both rotation conditions, participants carried the full motion capture system at all times. In the joyR condition participants stood in the dimly lit experimental hall in front of a standard TV monitor (1.5m viewing distance, HD resolution, 60Hz refresh rate, 40″ diagonal size) and were instructed to move as little as possible. They followed the sphere by tilting the joystick and were thus only able to use visual flow information to complete the task. In the physical rotation condition participants were situated in a 3D virtual reality environment using a head mounted display (HTC Vive; 2×1080×1200 resolution, 90 Hz refresh rate, 110° field of view). Participants’ movements were unconstrained, i.e., in order to follow the sphere they physically rotated on the spot, thus enabling them to use motor and kinesthetic information (i.e., vestibular input and proprioception) in addition to the visual flow for completing the task. If participants diverged from the center position as determined through motion capture of the head position, the task automatically halted and participants were asked to regain center position, indicated by a yellow floating sphere, before continuing with the task. Each movement condition was preceded by recording a three-minute baseline, during which the participants were instructed to stand still and to look straight ahead.
The starting condition (visual flow only or physical rotation) was also counterbalanced for participants with different reference frame proclivities, such that five egocentric, four allocentric, and two mixed-strategy participants started with the joyR condition, and four egocentric, five allocentric participants started with the physR condition. In each rotation condition, participants practiced the experiment in three learning trials with instructions presented on screen. Subsequently, the main experiment started, including 140 experimental trials per rotation condition. The experimental trials in each condition were randomized and split into five blocks of 28 trials each. The breaks were self-paced and the next block was initiated with the push of a button. The sphere moved either clockwise or anticlockwise around the participant; this movement was either slow or fast (randomized), depending on two different velocity profiles. The eccentricities of the sphere’s end positions were clustered from −15° to +15° around the mean eccentric end positions of 45°, 90°, and 135° in steps of 3° (e.g., the cluster 45° eccentricity ranged from 30° and 60° with 11 trials covering all eccentricities). In addition, eccentricities of 67° and 112° (2 x 8 trials) were used to achieve a near continuous distribution of end positions for the outward rotation in both rotation directions.
Mobile Brain/Body Imaging (MoBI) setup
To allow for a meaningful interpretation of the data modalities and to preserve their temporal context, the EEG data, motion capture data from different sources, and experiment event marker data were time-stamped, streamed, recorded, and synchronized using the Lab Streaming Layer.45
Data Recordings: EEG
EEG data was recorded from 157 active electrodes with a sampling rate of 1000 Hz and band-pass filtered from 0.016 Hz to 500 Hz (BrainAmp Move System, Brain Products, Gilching, Germany). Using an elastic cap with an equidistant design (EASYCAP, Herrsching, Germany), 129 electrodes were placed on the scalp, and 28 electrodes were placed around the neck using a custom neckband (EASYCAP, Herrsching, Germany) in order to record neck muscle activity. Data were referenced to an electrode located closest to the standard position FCz. Impedances were kept below 10kΩ for standard locations on the scalp, and below 50kΩ for the neckband. Electrode locations were digitized using an optical tracking system (Polaris Vicra, NDI, Waterloo, ON, Canada).
Data Recordings: Motion Capture
Two different motion capture data sources were used: 19 red active light-emitting diodes (LEDs) were captured using 31 cameras of the Impulse X2 System (PhaseSpace Inc., San Leandro, CA, USA) with a sampling rate of 90 Hz. They were placed on the feet (2 x 4 LEDs), around the hips (5 LEDs), on the shoulders (4 LEDs), and on the HTC Vive (2 LEDs; to account for an offset in yaw angle between the PhaseSpace and the HTC Vive tracking). Except for the two LEDs on the HTC Vive, they were subsequently grouped together to form rigid body parts of feet, hip, and shoulders, enabling tracking with six degrees of freedom (x, y, and z position and roll, yaw, and pitch orientation) per body part. Head motion capture data (position and orientation) was acquired using the HTC Lighthouse tracking system with 90Hz sampling rate, since it was also used for the positional tracking of the virtual reality view. Because the main focus of the study concerned the head movement-related modulation of neural dynamics in RSC, only data streams from the head motion capture data were used for the analysis.
Data Analysis
Data analysis was done in MATLAB (R2016b version 9.1; The MathWorks Inc., Natick, Massachusetts, USA), using custom scripts based on the EEGLAB46 and MoBILAB47 toolboxes.
Motion Capture Data Analysis: Automatic Detection of Movement Markers
Motion capture data was preprocessed using MoBILAB47 with adapted functions. The rigid body data was recorded in x, y, z, as well as quaternion orientation values, a 6 Hz zero-lag lowpass FIR filter was applied to the data, the orientation was then transformed into Euler angles and three time derivatives were subsequently calculated. For movement detection the absolute velocity of head orientation (yaw) was used (“physR” condition: visual flow controlled by head MoCap; “joyR”: visual flow controlled by the joystick). For convenience, in the remainder the term “head movement” refers to both rotation conditions. Head movement onset and offset events were extracted based on the velocity: A movement onset was initially defined as having a greater velocity than the 65% quantile of the complete data set (estimating movement to happen 35% of the time during the experiment). Once this coarse threshold was reached, the movement onset and offset event markers were created based on a finer threshold of 5% of the maximum velocity in a window of 2s around the detected movement. New movements could only be detected after the offset of the previous movement. A minimal movement duration of 285ms was defined to exclude movement artefacts created by jitter in the MoCap recording. The final data was exported as EEGLAB data set, synchronized, and different streams (EEG, MoCAP) were split into separate sets to allow for EEG-specific analysis based on movement markers.
Behavioral Data Analysis – Heading Error
Absolute heading error
For each epoch the absolute heading error was defined by taking the absolute difference between the participant’s initial start orientation (yaw; Euler angles) and the participant’s orientation after completing the backward rotation (completion indicated by the button press). The absolute heading error gives a robust overall indication of deviations from the starting orientation, without considering direction-specific over-or underestimation. For each participant and condition occasional outlier epochs with errors larger than three standard deviations were excluded. Furthermore, filler trials were also excluded. Finally, the absolute heading errors from the remaining valid epochs were averaged for three eccentricity categories within a range of ± 15° (45°: 30–60°; 90°: 75–105°; 135°: 120–150°). The term “heading error” refers to the average within each eccentricity category.
Statistics
The group-level statistics were performed with SPSS (version 25; IBM SPSS Statistics for Windows, Armonk, NY: IBM Corp.). For the absolute heading error a 2 x 2 x 3 factorial repeated measures analysis of variance (rANOVA) was performed with the within-participant factors “rotation condition” (physR, joyR), “direction” (clockwise, anti-clockwise), and “eccentricity” (15°, 30°, 45°); for the signed heading error cf. Supplements. In case the assumption of sphericity was violated, Greenhouse-Geisser corrected values are reported. If required, post hoc analysis was performed with the paired t-test. If the majority of data sets (≥ 50%) were not normally distributed (Kolmogorov-Smirnov test), post hoc testing was performed with the paired Wilcoxon signed-rank test. Bonferroni correction for post hoc testing was used in the case of multiple comparisons. Raw p-values are reported and indicated as significant if they were lower than the Bonferroni adjustment significance threshold. In general, while being aware that non-parametric post hoc testing was performed on ranks, for convenience the average values are presented as mean ± standard deviation (SD).
Independent Component Analysis (ICA)
Artefactual channels were manually identified, removed, and interpolated using spherical interpolation (on average, 17.6 channels were interpolated, SD=9.5). Subsequently, the data was re-referenced to the average of all channels and a zero-phase Hamming windowed high-pass FIR filter (order 827, pass-band edge 1 Hz) was applied to the data. Then artefactual time segments in the data were manually rejected; eye movements were not considered as artefacts. Data of both rotation conditions and their respective baselines were appended, and subsequently, the data was parsed into maximally independent components (IC), using an adaptive mixture independent component analysis (AMICA) algorithm19 with a principal component analysis (PCA) reduction to the remaining rank of the data set. For each IC an equivalent dipole model was computed as implemented by DIPFIT routines.48 For this purpose the individually measured electrode locations were rotated and rescaled to fit a boundary element head model (BEM) based on the MNI brain (Montreal Neurological Institute, MNI, Montreal, QC, Canada). We refer to the approximated spatial origin of an IC as “in or near” the specified location.
Event-Related Spectral Perturbations (ERSP)
A new copy of the pre-processed EEG data set was created, comprising interpolated channels and the complete continuous time courses, and the computed IC spatial filters and their respective source localization estimates were copied to this data set. Epochs of 13s length were created around the sphere onset markers, including a 1s pre-stimulus interval, resulting in 140 epochs per participant and condition. A spectrogram of all single trials was computed for all IC activation time courses using the newtimef() function of EEGLAB (3 to 100 Hz in logarithmic scale, using a wavelet transformation with 3 cycles for the lowest frequency and a linear increase with frequency of 0.5 cycles). To compute the final ERSP for the RSC region, artefactual epochs were first automatically rejected based on MoCap data and the IC activation time courses present in the respective cluster by removing epochs with baselines contaminated by head movements and epochs that contained considerable artefacts in the IC activation time course during task performance (as evaluated by epoch mean, standard deviation, and Mahalanobis distance49 see Supplements for more details). Artefact cleaning resulted in a sum of 1566 ERSP epochs for the physical rotation condition and 1880 epochs for the joystick rotation condition.
EEG Data Group-Level Analysis
Repetitive clustering approach
To allow for a group-level comparison of EEG data at the source level (ICs), the 70 ICs of each participant explaining most of the variance of the data were selected (1330 ICs in total) and subsequently clustered based on their equivalent dipole locations (weight=6), grand-average ERSPs (weight=3), mean log spectra (weight=1), and scalp topography (weight=1), using a region of interest (ROI) driven repetitive k-means clustering approach. The weighted IC measures were summed and compressed using PCA, resulting in a 10-dimensional feature vector for clustering. ICs were clustered by applying the k-means algorithm with n=50 cluster centroids to the resulting vectors and their respective distance between each other in vector space. We chose to use fewer clusters than ICs per participant because of our assumption that, although statistically independent per time point, there may be more than one IC per participant that is similar in function and location. ICs with a distance of more than three standard deviations from any final centroid mean were considered outliers.
Crucially, to ensure replicability of the clustering, we clustered 10,000 times and selected the final solution based on the following approach: i) We defined the Retrosplenial Complex (RSC) as the ROI in Talairach coordinates (x=0, y=-45, z=10) for which the clustering should be optimized; ii) for each resulting cluster in the region of interest (cluster of interest; COI), we calculated the number of participants with an IC in the cluster, the ratio of the number of ICs per participant in the cluster, the spread of the cluster (average squared distance of each IC from the cluster centroid), the mean residual variance (RV) of the fitted dipoles in the cluster, x, y, z coordinates of the cluster centroid, the distance of the cluster centroid from the ROI, and the Mahalanobis distance of this COI from the median of the distribution of the 10,000 solutions. The x, y, z coordinates were only used to determine the Mahalanobis distance and were not in the final quality measure vector; iii) the quality measures were standardized to their respective maximum, then weighted (#participants: 2, ICs/participants: −3, spread: −1, RV: −1, distance from ROI: −3, Mahalanobis distance from the median: −1). These weights optimize for a clustering solution that contains a cluster close to our ROI, which contains the ICs of many participants, but a low number of ICs per participant. Finally, the solutions were ranked according to their summed score, and the highest ranked solution was chosen as the final clustering solution. The final clustering solution contained the ICs from 15 participants, a ratio of 1.13 ICs per participant (two participants with two ICs each), a spread of 296, a mean RV of 10.3%, and a distance of 10.2 units in the Talairach space.
ERSP group-level analysis
ERSPs were computed for the RSC cluster by first averaging the time-frequency data at the IC level, then at the participant level, and finally at the group level. The time-frequency data of each trial was normalized by its mean activity50 and the average ERSP for each IC was calculated and baseline-corrected using a divisive baseline (mean activity in the interval of 200ms prior to sphere movement onset). Subsequently, the ERSPs of all ICs per participant were averaged. Finally, the ERSPs of all participants were averaged and log-transformed from the power-space into decibels (dB=10*log10(power)). Statistical analysis comparing ERSP activity either against the baseline or between conditions was performed at the group level using a permutation test with 2,000 permutations, and a multiple comparison correction using the false discovery rate (FDR; α=0.01). Final plots contain the significance thresholds as contours of significant time-frequency bins.
Supplements
Supplements – Video material
Video ‘Spot_Rotation_2018_joyR.mp4’
The video displays a participant in the joyR condition positioned in front of the screen and following the sphere during a rotation to the left via joystick control. One trial is commented according to the different phases of the experiment. The video is available at: https://osf.io/qrw9d/
Video ‘Spot_Rotation_2018_physR.mp4’
The video displays a participant in the physR condition in the MoBI setup with HMD including a rigid body for motion capture and EEG, following the sphere during a rotation to the left via physical rotation with the whole body. One trial is commented according to the different phases of the experiment. The video is available at: https://osf.io/6mfzg
Supplements – Methods I: Experimental design and task
Velocity Profiles
Upon button press, a red sphere started to travel at 5 m/s along a circle with a 30 m radius and accelerated either to a maximum of 35 m/s (fast condition) or to a maximum of 30 m/s (slow) condition. The acceleration and deceleration profiles were stretched or compressed to the eccentricity of the current trial. Eccentricity indicates the angle away from the participant when facing the initial local landmark (pole).
Supplements – Methods II: Behavioural data
Relative heading error
When considering relative differences, i.e., signed errors, for instance for clockwise rotations a negative error refers to an “undershoot” (i.e., the participant did not fully rotate back to the initial start orientation), whereas a positive error refers to an “overshoot” (i.e., the participant rotated further than the initial start orientation). Notably, for anti-clockwise rotations the sign for over-and undershoot is reversed. However, in order to have matching signs for both rotation directions, the relative error values were inverted for the anti-clockwise rotation epochs before averaging across valid epochs. Previously identified occasional outlier and filler epochs were excluded (cf. absolute heading error), and the remaining valid epochs were averaged. The group level statistics were performed as the absolute heading error.
Reaction times
The reaction time of the outward rotation was determined for each available epoch, defined as the time difference between the appearance of the visual stimulus (i.e., red sphere) and the first subsequent movement onset (cf. automatic detection of movement markers, as described in the Methods). In order to exclude epochs that contained excessively strong movements, which could have impeded an accurate detection of the movement onset, epochs were defined as invalid and were rejected if at least one of the following criteria was violated: i) if the yaw range exceeded 5° in the pre-stimulus interval (−500 ms to stimulus onset); ii) if the yaw range exceeded 5° in the pre-movement interval (stimulus onset to movement onset); iii) the first detected movement marker after the visual stimulus was an offset, not an onset; iv) no detected movement onset within the outward rotation epoch (i.e., appearance until disappearance of the visual stimulus). For each participant and condition, the reaction times were averaged across the remaining valid epochs, subsequently referred to by the term “reaction time“. A 2 × 2 factorial rANOVA was performed with the within-participant factors “rotation condition” (physR, joyR) and “direction” (clockwise, anti-clockwise). For further details on the group level statistics cf. the absolute heading error section.
The Simulator Sickness Questionnaire (SSQ)
The Simulator Sickness Questionnaire51 was administered to each participant three times: prior to any experimental recordings (“baseline“) and after each of the experimental conditions (physR, joyR). For each participant the mean value across all items (Likert scale, ranging 0-4) was calculated separately for each SSQ subscore (“nausea“, “oculomotor“, “disorientation” score). Subsequently, the term “SSQ” refers to the mean across items. Separately for each “SSQ” subscore (“nausea“, “oculomotor“, “disorientation” score) a oneway rANOVA was performed with the within-participant factor “rotation condition” (physR, joyR). For further details on the group level statistics cf. the absolute heading error section.
The Igroup Presence Questionnaire (IPQ)
The Igroup Presence Questionnaire52 was administered to each participant once after performing the physR condition. For each participant the mean value across all items (Likert scale, ranging 0-6) was calculated separately for each IPQ subscore (“spatial presence“, “involvement“, “experienced realism“), whereas “general presence” consisted of one item only.
Supplements – Methods III: Automatic epoch cleaning of ERSP
For each rotation condition the ERSP epochs were automatically cleaned by three approaches: i) in order to exclude possible contamination of the ERSP baseline, epochs that contained strong head movements (i.e., orientation yaw exceeding 5°) in the pre-stimulus interval and/or pre-movement interval were excluded for baseline calculation; ii) epochs that contained large head movement before the movement onset event (exceeding 5°; indicating a miss of the automatic movement onset detection) and/or the first detected movement marker after the visual stimulus was an offset, not an onset; iii) 10 % of epochs were automatically removed ranking largest for contamination by artefacts, as described below.
Automatic epoch removal by ranking
For iii) the epochs (absolute values of IC time courses) were separately ranked with respect to their maximum value, as determined by three approaches: “standard deviation”, “Mahalanobis distance”, and “mean”. First, the time course in each epoch was averaged for each selected IC, denoted as averageICepoch. Then each epoch was evaluated separately by each of the three approaches:
A) “Standard deviation”. The standard deviation was calculated for averageIC epoch resulting in a single value per epoch. Increasing values indicate more diverse IC activity in the epoch, thus being a sensitive measure for the detection of several “outlier” IC (or channels, if performed in sensor space).
B) “Mahalanobis distance”. The Mahalanobis distance was calculated for averageICepoch, resulting in a single value per epoch49. Complementary to A), it considers also the covariance between IC/channels, and in practice the Mahalanobis distance appeared to be sensitive especially to the presence of only few yet strong outliers.
C) “Mean”. The mean across averageICepoch was calculated, obtaining a single value per epoch (i.e., averaged across time and all IC). This measure represents an indicator of contaminations occurring in many IC/channels, e.g., due to strong baseline shifts or technical artefacts.
Ranking procedure
After obtaining numeric indices for each epoch separately by the three approaches, for each approach the indices were ranked in ascending order. Furthermore, in order to take into account the possibility that different epochs were detected by the three approaches (and thus possibly leading to unintended data loss more than the initially threshold, here 10%), the indices were weighted separately (i.e., ranks multiplied by the weighting factor [3 1 1] for “Mahalanobis distance”, “standard deviation”, and “mean”, respectively). Finally, the sum across weighted ranks was calculated and sorted in ascending order. Based on this final epoch rank list the 10% “worst” epochs were removed. This approach takes all three approaches into account, thus allowing sensitivity for different kinds of artifacts, while at the same time avoiding unintended data loss.
The final set comprised 127.5 ± 15.8 (mean ± SD) epochs for the baseline in the physR condition and 131.9 ± 16.1 in the joyR condition. On average 104.4 ± 12.6 epochs (physR) and 125.0 ± 9.6 epochs (joyR) were considered for the ERSP data per rotation condition, resulting in a total sum of 1566 and 1880 epochs for the physR and joyR condition, respectively.
Supplements – Methods IV: EEG differences associated with varying head movement velocity
Data preprocessing
One interest of the present study was the modulation of oscillatory brain activity associated with velocity changes (orientation; yaw) of visual flow (joyR condition; visual flow controlled by joystick movement) as well as for actual head movements (motion capture in the physR condition: visual flow controlled by head orientation), both referred to by “head movement velocity”. Only clusters with ≥ 75% of participants were included in the subsequent analysis, resulting in five clusters (anterior cingulate cortex, RSC, left and right parietal cortices, and occipital cortex). For each component in a given cluster, amplitude envelopes (Hilbert transform53) of band-pass filtered (Butterworth filter, fourth order) continuous oscillatory activity in four major frequency bands were obtained: 4-7, 8-13, 14-18, and 22-30 Hz (excluding technical artefacts at approx. 20-21 Hz due to the head-mounted display), subsequently referred to as theta, alpha, low beta and high beta bands, respectively. The frequency bands were selected with respect to previous findings suggesting their involvement in task-relevant domains, such as spatial navigation28-30, 36, 37 and sensorimotor processing54, 55. EEG data were shifted back by 100 ms in order to match the MoCap recordings (due to technical acquisition delays of the multivariate data streams.
In each frequency band for each IC in a given cluster, paired EEG epochs were created with respect to visual stimulus onset (i.e., start of the outward rotation): “no movement” (NMEEG) referring to −200 ms to stimulus onset, and “movement” (MEEG, Mvelocity) epochs, referring to the first movement onset to offset after the stimulus, according to the automatically determined velocity onset and offset marker in the continuous MoCap data stream (head orientation; yaw) as described in the online methods section. In order to ensure sufficient data quality, MEEG and Mvelocity epochs were excluded that were too short (< 500 ms) and/or too long (> 15 s). Furthermore, NM and/or M epochs with mean values exceeding 3 SD were excluded (in both, EEG and movement velocity epochs). In order to remove potential movement artifacts from the pre-stimulus baseline, NM epochs with yaw range > 5° were excluded, along with the corresponding M epochs. Subsequently, all NMEEG and MEEG epochs were separately appended into a continuous data stream (separately for each frequency band and IC per cluster). Then baseline-corrected values were obtained: EEGdiff = MEEG_SingleSample − NMEEG mean.
Velocity binning analysis
In order to assess modulation of EEG oscillations accompanying different movement velocities, a binning procedure was applied.22, 23 For this purpose, in each rotation condition all Mvelocity epochs (non-averaged) were appended to form a continuous data stream of head movement velocity. Percentiles (10 percent steps; resulting in 10 bins) were taken of all velocity values (sorted in ascending order), and data samples were assembled corresponding to each percentile category. Notably, this “relative binning” procedure by percentiles (in contrast to the “absolute binning” of previous work, i.e., equidistant velocity binning22) has the advantage of providing equal amounts of samples per velocity percentile bin. Therefore, this approach avoids a considerable drawback of absolute velocity binning which would provide only very few data samples for higher (i.e., less frequent) velocity values. Subsequently, EEGdiff was averaged across all data samples in a given percentile bin (EEGdiffMeanPerBin; separately in each band for each IC per cluster, in each rotation condition). If occasionally multiple IC were available per participant in a given cluster, the obtained EEGdiffMeanPerBin values were averaged across the available IC for the respective participant, thus obtaining a single value per participant and velocity bin. These finally obtained EEGdiffMeanPerBin values represent changes in EEG (IC) amplitude with respect to the pre-stimulus baseline, averaged across all available samples within a certain range of movement velocity (as defined by the percentile categories).
Statistics
The EEGdiffMeanPerBin values were compared between both rotation conditions (physR vs. joyR) in each frequency band, separately for each velocity percentile bin: statistical significance of condition differences was determined by paired, non-parametric testing (Wilcoxon signed rank test; n = 40 tests per cluster). Correction for multiple comparisons was obtained with the False Discovery Rate (FDR56) at 0.05 level; raw p-values were masked if not being FDR-significant.
Supplements - Results
Heading error
The analysis of heading errors comprised 131 ± 1 (mean ± SD) trials in both, the physR and joyR condition, respectively.
Relative heading error
The rANOVA revealed a significant main effect of “direction” (F1,18 = 6.82; p = 0.018) and of “eccentricity” (F1.24,22.34 = 33.63; p < 0.001). Two significant interaction effects were present: i) between “rotation condition” and “eccentricity” (F1.58,28.42 = 13.27; p < 0.001) and ii) between “rotation condition“, “eccentricity“, and “direction” (F2,36 = 3.54; p = 0.039). Post hoc analysis (paired t-test) of the latter interaction effect ii) revealed that for both rotation directions the smallest eccentricity category (45°) was associated with a positive error (i.e., overshoot, too far rotation), while the largest eccentricity category (135°) was associated with a negative error (i.e., undershoot, too short rotation). Furthermore, direction-specific differences between both rotation conditions were obtained. Summarizing the results: i) in both rotation conditions smaller eccentricities (∼ 45°) were associated rather with error overshoots (on average 6.73°), i.e., the back rotation exceeded the initial start orientation, whereas larger eccentricities (∼ 135°) were associated rather with error undershoots (on average −6.67°), i.e., the back rotation was too short and the participants did not reach the initial start orientation; ii) the relative heading error was significantly larger for the joyR condition than for the physR condition, depending on the rotation direction and the eccentricity (anti-clockwise: 45°; clockwise: 135 °), which in actual terms suggests that for small/large eccentricities in the joyR condition the participant had a tendency to stop the back rotation not at about 0°, but rather showing a slight shift to the left stopping at about −12°. The statistical results are presented in Supplements Fig. 1. While the analysis of more differentiated eccentricity categories was beyond the scope of the present study due to a limited number of trials, visual inspection of the relative heading errors in decimal steps revealed for both conditions and rotation directions a flip from “overshoot” to “undershoot” errors at about 110° (cf. Supplements Fig. 2A-B).
Reaction times
Outward rotation
The reaction time comprised 110 ± 20 (mean ± SD) and 127 ± 20 epochs in the physR and joyR condition, respectively. The rANOVA revealed a significant main effect of “rotation condition” (F1,18 = 68.37; p < 0.001), showing significantly shorter reaction time for “physR” (0.48 ± 0.22 s) than for “joyR” (1.12 ± 0.31 s). No further significant effects were obtained. Results are presented in Supplements Fig. 3.
Simulator Sickness Questionnaire (SSQ) & Igroup Presence Questionnaire (IPQ)
IPG
Overall, the IPG indicated that the participants experienced a relatively degree of presence during the physR condition (general presence: 3.63 ± 1.71; spatial presence: 4.31 ± 1.36; involvement: 2.75 ± 1.56; experienced realism: 1.55 ± 1.06).
SSQ
In general, while overall the SSQ scores were very low (grand-averages < 1), SSQ in both rotation conditions was slightly increased compared to the baseline. However, SSQ for both rotation conditions was similar in each of the subscores (mean ± SD):
SSQ - Nausea subscore
The rANOVA revealed a significant main effect of “rotation condition” (F1.48,26.57 = 7.93; p = 0.004). Post hoc analysis (Wilcoxon signed rank test) revealed slight albeit significant differences, with “baseline” (0.06 ± 0.1) being smaller than “physR” (0.26 ± 0.2; p = 0.002) and than “joyR” (0.3 ± 0.34; p = 0.004), respectively. No further significant effects were obtained.
SSQ - Oculomotor subscore
The rANOVA revealed a significant main effect of “rotation condition” (F2,36 = 10.44; p < 0.001). Post hoc analysis (Wilcoxon signed rank test) revealed slight albeit significant differences, with “baseline” (0.23 ± 0.21) being smaller than “physR” (0.61 ± 0.49; p = 0.002) and than “joyR” (0.53 ± 0.47; p = 0.006), respectively. No further significant effects were obtained.
SSQ - Disorientation subscore
The rANOVA revealed a significant main effect of “rotation condition” (F2,36 = 7.45; p = 0.002). Post hoc analysis (Wilcoxon signed rank test) revealed slight albeit significant differences, with “baseline” (0.05 ± 0.07) being smaller than “physR” (0.27 ± 0.27; p = 0.003), but no Bonferroni-significant differences to “joyR” (0.14 ± 0.27; p = 0.03). No further significant effects were obtained.
EEG differences associated with varying head movement velocity
Velocity binning
During the “movement” interval of the outward rotation (as defined by velocity onset to offset after the visual stimulus; head orientation yaw), FDR-significant differences of the baseline-corrected EEG amplitude (EEGdiffMeanPerBin) between the physR and the joyR condition were obtained in the distinct clusters in different frequency bands. On average artefact-cleaned 126 ± 10 “movement” epochs (physR: 123 ± 8; joyR: 128 ± 11) were subjected to the velocity binning analysis. The average “movement” epoch duration was 3.8 ± 0.7 s (physR: 3.6 ± 0.7 s; joyR: 4 ± 0.7 s). The ranges for the obtained velocity bins were on average (percentile bin 1-10; ≥ start until < end): 0-8°/s (± 1); 8-11°/s (± 2), 11-14°/s (± 3), 14-17°/s (± 3), 17-20°/s (± 3), 20-24°/s (± 4), 24-29°/s (± 4), 29-35°/s (± 4), 35-42°/s (± 4), 42-81°/s (± 9). The average duration per velocity bin (i.e., appended samples per category) was 48 ± 11 s.
Retrosplenial complex (RSC) cluster
The physR condition was associated with increased oscillatory amplitude in the RSC cluster compared to joyR, primarily in the low beta band (14-18 Hz) with 80% of significant velocity bins, while showing less expressed differences in the alpha band (8-13 Hz; 20% of significant velocity bins); results are presented in Fig. 3 of the main text.
Acknowledgements
The research was supported by the German Research Foundation (DFG) grant no. GR 2627/8-1. We gratefully acknowledge the help of Jonna Jürs and Yiru Chen with the data acquisition and pre-processing.
Footnotes
klaus.gramann{at}tu-berlin.de
friederike.hohlefeld{at}tu-berlin.de
lukas.gehrke{at}tu-berlin.de
marius.klug{at}tu-berlin.de