Neural decoding of gait phase information during motor imagery and improvement of the decoding accuracy by concurrent action observation

Brain decoding of motor imagery (MI) is crucial for the control of neuroprosthesis, and it provides insights into the underlying neural mechanisms. Walking consists of stance and swing phases, which are associated with different biomechanical and neural control features. However, previous studies on the decoding of the MI of walking focused on the classification of more simple information (e.g., walk and rest). Here, we investigated the feasibility of electroencephalogram (EEG) decoding of the two gait phases during the MI of walking and whether the combined use of MI and action observation (AO) would improve decoding accuracy. We demonstrated that the stance and swing phases could be decoded from EEGs during AO or MI alone. Additionally, the combined use of MI and AO improved decoding accuracy. The decoding models indicated that the improved decoding accuracy following the combined use of MI and AO was facilitated by the additional information resulting from the concurrent cortical activations by multiple regions associated with MI and AO. This study is the first to show that decoding the stance versus swing phases during MI is feasible. The current findings provide fundamental knowledge for neuroprosthetic design and gait rehabilitation, and they expand our understanding of the neural activity underlying AO, MI, and AO+MI of walking. Significance Statement Brain decoding of detailed gait-related information during motor imagery (MI) is important for brain-computer interfaces (BCIs) for gait rehabilitation. However, previous knowledge on decoding the motor imagery of gait is limited to simple information (e.g., the classification of “walking” and “rest”). Here, we demonstrated the feasibility of EEG decoding of the two gait phases during MI. We also demonstrated that the combined use of MI and action observation (AO) improves decoding accuracy, which is facilitated by the concurrent and synergistic involvement of the cortical activations by multiple regions for MI and AO. These findings extend the current understanding of neural activity and the combined effects of AO and MI and provide a basis for developing effective techniques for walking rehabilitation.


163
The above-mentioned results suggested that different cortical regions were involved in 164 decoding during each condition. To quantitatively examine the contributions of each cortical region 165 during each condition, we divided the electrodes into 11 regions of interest (ROIs) (Fig. 7A).
Subsequently, the average contributions within each ROI were compared to the mean contribution 167 across all the electrodes as a baseline (0.217 = 1/46 electrodes) ( Fig. 7B−7D). For the AO+MI condition, the mean weights of the left anterotemporal, left occipital, and right occipital ROIs were 169 significantly greater than the baseline weight (p < 0.05, permutation test with FDR correction) (Fig.   170   7B). For the AO condition, as in AO+MI, the mean weights of the left anterior-temporal and left and 171 right occipital ROIs were significantly greater than the baseline weight (p < 0.05, permutation test 172 with FDR correction) (Fig. 7C). On the other hand, the mean weights of the left and right 173 frontocentral and left parietal ROIs were significantly lower than the baseline weight (p < 0.05, 174 permutation test with FDR correction) (Fig. 7C). For the MI condition, the left and right occipital In the field of neural engineering, brain activities during AO

211
Topographic cortical contribution to the decoding of the gait phase contributions to decoding (Fig. 7C). The anterotemporal ROI spatially overlaps with a part of the that presents with deficits in the performance of purposeful movements such as imitation, is 224 frequently observed in patients with left hemisphere damage (51). Also, it has been demonstrated 225 that limb-apraxic patients had an impairment in gesture comprehension and the severity of the

249
For AO+MI, we observed higher contributions of the left and right occipital and the left 250 anterotemporal ROIs (Fig. 7A). As with MI, the contribution of the sensorimotor regions, which was 251 low during AO, was not significantly different from the mean contribution across the electrodes 252 during AO+MI. The difference between the contribution patterns during the tasks suggests that the 253 underlying cortical processes were different for AO and MI, and these processes were concurrently 254 involved during AO+MI. This may explain the higher decoding accuracy during AO+MI than AO or recording. The duration was 1 minute for all the tasks. The participants performed the three different 325 tasks six times in a random order (six minutes for each condition).

327
Data collection EEG signals were recorded from 61 channels using an EEG cap (Waveguard original, ANT 333 Neuro b.v., Enschede, Netherlands) according to the international 10-10 system layout (Fig. 1D) 334 and an EEG amplifier (eego sports, ANT Neuro b.v., Enschede, Netherlands) at a sampling 335 frequency of 500 Hz. Ground and reference electrodes were placed on AFz and CPz. Impedances 336 of the electrodes were kept below 30 kΩ (10 kΩ in most electrodes), which was substantially lower 337 than the recommended impedance (below 50 kΩ) for the high-impedance EEG amplifier.

345
We performed analyses of the EEG signals using custom-written programs in MATLAB

403
For each subject, the decoding accuracy was calculated by 5-fold cross-validation, where the data 404 recorded during the 6-min task were divided into 6 segments (1 min each); 5 segments were used 405 as training data while the remaining segment was used for testing the decoding model.

406
To evaluate the spatial contributions of the cortical regions to the decoding of the gait (FDR) for multiple comparisons (75). Additionally, the differences between the decoding accuracies 433 of the swing and stance phases of each condition were compared using two-tailed paired t-tests.

434
Normality was confirmed by the Lilliefors test.

435
The mean weights of the model of the ROIs were compared. Because normality was not 436 observed in all the cases (tested by the Lilliefors test), the mean weights were compared using a 437 permutation test involving 1000 random permutations (76)