Validation of manifold-based direct control for a brain-to-body neural bypass

Brain-body interfaces (BBIs) are neuroprostheses that can restore the connection between brain activity and body movements. They have emerged as a radical solution for restoring voluntary hand control in people with upper-limb paralysis. The BBI module decoding motor commands to actuate the limb from brain signals should provide the user with intuitive, accurate, and stable control. Here, we present the design and demonstration in a monkey of a novel brain decoding strategy based on the direct coupling between the activity of intrinsic neural ensembles and output variables, meant to achieve ease of learning and long-term robustness. We identified once an intrinsic low-dimensional space (called manifold) capturing the co-variation patterns of the monkey’s neural activity associated to reach-to-grasp movements. We then tested the animal’s ability to directly control a computer cursor using cortical activation along the manifold axes and demonstrated rapid learning and stable high performance over 16 weeks of experiments. Finally, we showed that this brain decoding strategy can be effectively coupled to peripheral nerve stimulation to trigger hand movements. These results provide evidence that manifold-based direct control has promising characteristics for clinical applications of BBIs.

activity and body movements. They have emerged as a radical solution for restoring voluntary hand 27 control in people with upper-limb paralysis. The BBI module decoding motor commands to actuate 28 the limb from brain signals should provide the user with intuitive, accurate, and stable control. Here, 29 we present the design and demonstration in a monkey of a novel brain decoding strategy based on the 30 direct coupling between the activity of intrinsic neural ensembles and output variables, meant to 31 achieve ease of learning and long-term robustness. We identified once an intrinsic low-dimensional 32 space (called manifold) capturing the co-variation patterns of the monkey's neural activity associated 33 to reach-to-grasp movements. We then tested the animal's ability to directly control a computer cursor 34 using cortical activation along the manifold axes and demonstrated rapid learning and stable high 35 performance over 16 weeks of experiments. Finally, we showed that this brain decoding strategy can 36 be effectively coupled to peripheral nerve stimulation to trigger hand movements. These results 37 provide evidence that manifold-based direct control has promising characteristics for clinical 38 applications of BBIs.

45
Brain-body Interfaces (BBIs) are neuroprostheses that allow users to voluntarily control the 46 movement of their body through an artificial neural bypass. A survey of patients with tetraplegia due 47 to spinal cord injury [1] showed that BBIs are the preferred solution compared to the control of 48 external robotic devices characterizing classic brain-machine interfaces (BMIs) [2]. In BBIs, brain Focusing on the restoration of hand function, an ideal BBI should effectively integrate an easy-to-55 learn, accurate, and stable brain decoding paradigm with a motor restoration module allowing the 56 selective control of the hand. Recently, we demonstrated in a preclinical study in monkeys that 57 peripheral nerve stimulation (PNS) at the intrafascicular level can evoke multiple grasps and hand 58 extension movements with only two nerve implants [14], thus complying with the requirement of 59 movement selectivity. Here, we present a brain decoding module based on the direct linear coupling 60 between intrinsic neural ensemble dynamics and motion commands, which satisfies the 61 characteristics of ease of learning and temporal stability. We next validate a full BBI integrating this 62 brain decoding approach with intrafascicular PNS to trigger hand movements.

64
To design our brain decoding strategy, we built on recent studies [15]- [17] showing that neural 65 population dynamics is constrained by the brain circuitry in a low-dimensional space, i.e., the neural 66 manifold, spanned by the so-called neural modes, and that learning a new task is facilitated when the 67 underlying neural activity pattern lies within this intrinsic manifold [17]. We hypothesized that by 68 directly linking the activation of intrinsic neural modes to the controlled variables, the subject could 69 learn to modulate this activation in such a manner that reduces the need for frequent calibration. Thus, We examined the performance of the manifold-based direct control strategy in a macaque monkey. 74 Specifically, we computed once a 2D manifold capturing a significant portion of the variance of the 75 animal's neural activity while performing a behavioral grasping task. We then coupled the activation 76 of the two fixed neural modes to the 2D movement of a cursor and tested this BMI paradigm in a 77 point-to-point task with incremental variations over weeks. This BMI phase was used to evaluate the 78 intuitiveness and long-term performance of our decoding strategy. We show that the monkey could 79 succeed rapidly and robustly over time. Finally, we additionally coupled the dynamics of the two 80 neural modes to the amplitude of stimuli delivered by intrafascicular electrodes implanted in the 81 animal's arm nerves. We demonstrate that our decoding strategy can be integrated with intrafascicular 82 PNS into a BBI to grade hand movements.

87
We tested a manifold-based direct control paradigm to control two degrees of freedom Next, we tested the effectiveness and robustness of a 2D BMI with manifold-based direct control over 115 38 sessions (spanned over 113 days, Supp. Table 1). The monkey controlled a cursor on a screen 116 through its M1 activity mapped into the 2D manifold ( Figure 1B). The second and third latent 117 variables, hereafter referred to as " and # , were proportionally converted into the vertical (y) and 118 4 horizontal (x) coordinates of the cursor, respectively. We designed a delayed point-to-point cursor 119 control task: the animal had to first keep the cursor in a baseline position for 0.5 s and then reach and 120 hold a target location for 0.1 s. Trial timeout was set to 8 s and successful trials were rewarded with 121 liquid food. We employed an incremental training paradigm [19]: the number of DoFs to be controlled 122 and the reaching space were progressively changed during the protocol ( Figure 1C). For the first 10 123 sessions, only the y-coordinate of the cursor was brain-controlled with targets placed vertically with 124 respect to the baseline position (cyan in Figure 1C): during these sessions the x-coordinate was set 125 to 0. Next, and for the rest of the protocol, we allowed the monkey to control the cursor both in the x 126 and y directions and we varied the location of the target: on session 11 we only presented vertical 127 targets (blue), on sessions 12 to 15 the targets were placed diagonally to the baseline position (purple), 128 and on sessions 16 to 20, horizontally (red). Finally, between sessions 21 and 38, the targets were 129 randomly alternated (gray).

131
The monkey was able to effectively modulate its latent neural activity to perform the different tasks 132 (Figure 2A). Importantly, the control was possible without using hand muscle contractions (Supp.  Table 1). This performance plateau possibly reflects the saturation of both the animal's 159 neuromodulation ability and motivation.  Figure 2B), the monkey having reached a stable success rate and execution time. of M1 channels in the baseline condition (Supp. Figure 3A) and the corresponding latent neural 173 activity (Supp. Figure 3B) varied across sessions. We thus investigated whether, following these 174 instabilities, the animal changed its neural tuning strategy to perform the different tasks. In particular, 175 we analyzed the inter-session variability of M1 channels preferential tuning, as measured by diagonal, and horizontal targets, respectively; Figure 3A). Interestingly, the variation between 182 sessions of the single and multi-task phases was higher than the variation across sessions within the 183 same phase for all the three targets (median of 1.32, 1.30, 0.78 a.u.; Figure 3A) and to a greater extent 184 for the vertical and diagonal targets. This suggests that the circumstances of the task contributed 185 significantly to the changes in neural tuning. We next analyzed the average neural tuning strategy at 186 the single channel level in each protocol phase ( Figure 3B) and focused on the most modulated 187 channels (Supp. Figure 3C). We can see that during 1D control with only vertical targets, the monkey 188 preferentially modulated channels #3, 13, 16, 20, and 22, all of which had a positive weight on " . 189 When the horizontal DoF was introduced, channel #20 was abandoned, likely because of its similar 190 positive contribution to both neural modes. Moreover, the animal started to tune channel #29, 191 associated with a positive weight on " and a slightly negative weight on # , and, interestingly, 192 channel #27, associated with a much higher weight on # than on " , likely to counteract the strong 193 negative effect of channel #22 on # and thus keep the horizontal displacement at zero. The diagonal 194 target in the single task phase was attained by favorably tuning channels #3 and 13, which had a more 195 positive impact on " than on # , and channel #20. When introduced, the horizontal target was 196 reached by mostly modulating channels #25 and 27, which had a much higher weight on # than on 197 " , and channel #20. These three channels were maintained in the multi-task phase of the protocol the median nerve and one in the radial nerve, to trigger the opening and closing of the hand, 213 respectively. We designed the BBI experiment as follows. While the monkey performed the cursor 214 control task with vertical and/or horizontal targets, the latent variables # and " , once over a 215 threshold, linearly modulated the amplitude of the stimuli applied to the median and radial nerve, 216 respectively ( Figure 1B), either jointly or independently (Supp. Table 2). Through a short calibration 217 phase at the beginning of the experimental session, we set the saturation level and threshold for 218 stimulation of the driving latent variable/s (Supp. Figure 4A). This latter value was regulated to 219 reduce target-unspecific stimuli due to the frequent coactivation of # and " , and at the same time 220 span a large range of neuromodulation. The calibration also served to determine the functional 221 amplitude range for the selected Mk-TIME channels (Supp. Figure 4B). After setting the control 222 parameters, we tested the BBI in grading the two target motor functions, i.e., hand opening and 223 closing. The full BBI protocol is described in Figure 4A. M1 activity was processed in real-time to 224 extract spike events and compute the channels firing rate. Stimulation-induced artifacts were then 225 removed by subtracting the firing rate of a channel that responded only when stimuli were applied. 226 Noise-free spike rates were projected into the 2D manifold to derive the activation of the two latent

255
We assessed the performance of a 2-DoF brain control strategy confined within a fixed intrinsic motor  The monkey was trained to perform a center-out reach-and-grasp task, which is detailed in [18]. The experiment was performed for 6 sessions, in which we enabled one or both types of stimulation 478 (i.e., median, or radial) and presented one or both types of target (i.e., vertical, or horizontal), as 479 specified in Supp. At the beginning of each session of the brain PNS control experiment, a calibration procedure was 484 performed to tune the parameters of the linear relationship between latent variable activation and 485 stimulation amplitude (Supp. Figure 4). In the first step, the animal performed the brain cursor 486 control task for approximately 10 minutes, alternating between vertical and horizontal targets (Supp. 487 Figure 4A). This phase served to determine the range of latent variable modulation that the animal 488 exhibited for the two target types on that day. Based on these recordings, we determined FL# and 489 *)K . FL# of a given latent variable was set to be just above the maximum of its activation averaged 490 across the successful trials with the target type for which it was leading (horizontal target for # and 491 vertical target for " ). Conversely, *)K was set to be just above the maximum of the latent variable In the last two sessions of brain PNS control experiment, we measured the grip force using a custom-529 made sensor [18] or the wrist extension force using a commercial dual-range force sensor (Vernier, 530 EducaTEC AG,CH) when median or radial nerve stimulation was enabled, respectively. These 531 signals were recorded at 1 kHz using the RZ2 processor. To show that the animal performed the brain cursor control task without exploiting hand movements, 555 we recorded the corresponding muscle activity in two sessions after the implantation of the EMG 556 electrodes. In these two sessions the monkey also performed the behavioral reach-and-grasp task. We 557 compared the EMG activity of the implanted muscles acquired during the brain cursor control task 558 with the activity measured during the behavioral task (Supp. Figure 1). To evaluate the changes in neural recordings across sessions, we computed the mean firing rate of 572 M1 channels during the baseline phase of the cursor control task (i.e., when the cursor was in the 573 baseline box) and averaged across all the trials of each session (Supp. Figure 3A). Similarly, for each 574 trial we computed the mean activity of latent variables # and " during the baseline phase (Supp. 575 Figure 3B).

577
To evaluate the neural tuning strategy used by the monkey to reach the different targets, we measured We evaluated the monkey's ability to successfully perform the brain cursor control task even when 614 PNS was enabled, by comparing the success rate obtained during the brain PNS control task with that 615 obtained during the calibration phase on the same session.