## Abstract

The healthy human nervous system accurately and robustly controls movements despite nonlinear dynamics, noise, and delays. After a stroke, motor ability frequently becomes impaired. To provide insight into the relative impact of specific sensorimotor deficits on motor performance, we modeled neural control of reaching with the human upper limb as a near-optimally feedback-controlled two-degree-of-freedom system with biologically based parameters. We added three sensorimotor impairments commonly associated with post-stroke hemiparesis—abnormal joint coupling, increased noise on internally modeled dynamics, and muscular weakness— and examined the impact on reaching performance. We found that abnormal joint coupling unknown to the system’s internal model caused systematic perturbations to trajectories, longer reach durations, and target overshoot. Increasing internal model noise and muscular weakness had little impact on motor performance unless model noise was increased by several orders of magnitude. Many reaches performed by our perturbed models replicate features commonly observed in reaches by hemiparetic stroke survivors. The sensitivity to unmodeled abnormal joint coupling agrees with experimental findings that abnormal coupling (possibly related to internal model errors) is the main cause of post-stroke motor impairment.

## I. Introduction

The healthy human nervous system quickly, accurately, and robustly controls movement in the face of myriad difficulties including nonlinearities, noise, and delays [1]. Despite this adeptness, roughly 5 million individuals become permanently disabled by stroke each year [2]. Researchers and clinicians note that disabled stroke survivors’ nervous systems must contend with additional sensorimotor deficits such as abnormally coupled joints [3]–[5], increased noise in sensing and estimation [6], muscular weakness [7], and spasticity [8]. These difficulties generally correlate with motor impairment, working in concert to degrade motor performance.

The complexity of the neuromuscular system makes it challenging to determine causal relationships between the sensorimotor deficits facing stroke survivors and their resultant motor impairments [9]–[11]. Many techniques have been developed to study how the nervous system controls movements. Imaging techniques (e.g., fMRI), stimulation techniques (e.g., transcranial magnetic stimulation (TMS), deep-brain stimulation, optogenetics), and movement studies (e.g., center-out reaching tasks, perturbed locomotion studies [12], simple games [13]) have all helped us understand the functioning of healthy and impaired nervous systems. Studies using these techniques correlate inputs (e.g., displayed images in fMRI studies, electromagnetic pulses in TMS, mechanical perturbations in movement studies) with neuro-physiological outputs (e.g., blood-oxygen contrast, electrical signals in the muscles, bodily kinematics/dynamics). However, motor impairments can stem from sensorimotor deficits at any stage of the control process—sensory encoding, motor planning, or motor execution. These details are difficult to identify using the input-output approach.

Computational models offer the ability to ethically and noninvasively alter the system being studied in a controlled manner (e.g., weakening muscles [14], shifting tendon-attachment points [15]). Such models are useful for testing scientific theories (e.g., central pattern generators [16]), planning clinical interventions (e.g., tendon-transfer surgery [15]), and even designing engineered systems (e.g., McMahon’s tuned track [17]). Some studies have used optimal control—thought to be a good generative model of the healthy human nervous system [1], [18]–[20]—to reproduce human behaviors [18], [21]–[23]. While such models have been used extensively to study the healthy nervous system, they have rarely been applied to understand motor impairment, particularly that of neural origin.

To examine how sensorimotor deficits may impact motor performance, we model the healthy human neuromuscular system as a planar two-degree-of-freedom (DOF) arm controlled by a near-optimal feedback controller with biologically based parameters (e.g., mass, inertia, sensory noise, actuator limits). To explore the behaviors captured by this model, we add three sensorimotor impairments commonly associated with post-stroke hemiparesis: abnormally coupled joints, increased noise on internally modeled dynamics, and muscular weakness. We demonstrate the sensitivity of our model to each of these deficits and show general consistency with experimental assessments of post-stroke hemiparesis.

## II. Methods

We model the healthy neuromusculoskeletal system as a near-optimally feedback-controlled planar arm, summarized in Fig. 1 with variables and parameters listed (including references for specific values) after Section III. We then perturb the model in ways consistent with sensorimotor deficits associated with post-stroke hemiparesis.

### A. Model

#### 1) Dynamics

We model the arm using the dynamics of a revolute-revolute (RR) robot constrained to move in the plane of the shoulder [18], [20], [25], [26]. The nonlinear differential equation of motion is as follows: where state contains the arm’s realized joint angles, velocities, and torques and input contains commanded joint torques:

Joint torques are separated based on contribution to flexion (positive) and extension (negative), thereby allowing for cocontractions. In Eqn. 1, *M*, *C*, *V*, and *B* represent the arm’s mass matrix, dynamic coupling matrix, centrifugal and Coriolis effects, and viscous damping (parameters taken from [27] and [28]). *M* and *V* vary nonlinearly with arm state, while *C* and *B* remain constant. To model first-order muscle activation dynamics, we place a low-pass filter with time constant *τ* = 60 ms between the commanded and realized joint torques [28]. Panel B of Fig. 1 defines joint origins and sign conventions for the model; conventions for commanded joint torques (not illustrated) mirror those for realized joint torques.

We numerically integrate the equation of motion over each simulation time step (10 ms) using a variable-time-step medium-order Runge-Kutta method. This gives the following discrete-time state-update equation:
where captures white Gaussian process noise with covariance
where *σ _{u}* = 0.02 N-m is the standard deviation of motor noise [29].

Time delays (estimated to be 60 ms for transcortical pathways) result from nerve conduction dynamics and neural computation. To capture this in our model, we augment the state vector with previous time steps up to the first one observable by the controller [28], [29]. For a delay of *T _{d}* time steps, the augmented state at time step

*k*is and the corresponding state-update equation is where with covariance

The system senses a noisy and biased version of the delayed arm state at time step *k*. We model uncertainty in the sensory feedback as additive white Gaussian noise and sensory bias as a linear function of the current arm configuration [30]. This gives the following feedback equation:
where constants *c*_{0} and *c*_{1} and with covariance are fit to data from Cusmano *et al*. [6]. Because bias is a function of current state, we can condense the feedback equation as follows:

#### 2) Estimation

For control, the neuromuscular system needs reliable information about the arm’s current state. However, at any given time, the system has only delayed and noisy sensory information coming in from the periphery. Just as the (healthy) neuromuscular system appears to optimally control movements, it also appears to optimally estimate the state of the limb [31]. We approximate this optimal estimation using a Kalman filter, merging sensory feedback with state predictions generated by the brain’s internal model:
where and with covariance
*σ _{p}*, the standard deviation of noise on internally modeled dynamics (i.e., internal process noise), is all that differentiates the internal mental model of the arm from the physical plant. We will refer to

*σ*as estimation noise.

_{p}By accounting for time delay via state augmentation, the filter estimates all states within the interval of the time delay. Given the arm’s nonlinear dynamics, we use an unscented Kalman filter (UKF) [32]. As depicted in panel C of Fig. 1, the UKF uses sampling points to capture the mean and covariance of the predicted state. Unlike the extended Kalman filter, which relies on a linear approximation of the dynamics, the UKF’s sampling points can be exactly propagated through the nonlinear equation of motion. To the best of our knowledge, use of the UKF in computational biomechanics has not been seen in the literature.

#### 3) Control

We use a model predictive controller (MPC) to determine near-optimal joint torques for the arm [33]. This MPC uses the brain’s internal model of the arm to compute a dynamically consistent, optimal torque trajectory over a finite time horizon while accounting for limits on joint positions and torques (from Chackwick *et al*. [34] and van Dijk *et al*. [35]). Previous work on human motor control suggests that movements generally balance error minimization with effort minimization [36]. This is reflected in our finite-time-horizon optimization:

In the cost function, *H* is the number of time steps in the finite horizon, is the internal model’s hand position, is the target location specified in Cartesian coordinates (constant during the reaches performed here), *W _{p}* and

*W*are weighting matrices, and is the resultant optimal torque trajectory. We solve the MPC optimization problem using the Multi-Parametric Toolbox (MPT3 [37]), which requires linear or affine equality constraints. Therefore, the first constraint equation is a linearization (in this case, affinization) of the internal model’s dynamics and the output equation is a linearization of the internal model’s forward kinematics. Both linearizations are first-order Taylor-series approximations about the current estimated arm state and known commanded joint torques, which is only appropriate for

_{u}*t*“close to”

*k*; we assume that this holds over the finite horizon. By linearizing the forward kinematics, we track a reference in Cartesian instead of joint space. Finally, the parameter

*α*in the inequality constraint for control effort is a scalar between 0 and 1; it provides a rough approximation of arm strength. We re-plan joint torques every 30 ms.

### B. Model Perturbations

We perturbed the healthy model in ways that align with sensorimotor deficits commonly observed following stroke.

#### 1) Abnormal joint coupling

Chronic stroke survivors frequently exhibit abnormal coupling between joints [3], [11], [38]. To examine the effects of this coupling on motor performance, we changed the coupling matrix *C* in the arm’s equations of motion (Eqn. 1) from an identity matrix to one with off-diagonal elements extracted from data published by Dewald *et al*. [3]. As alluded to in Sections II-A.2 and II-A.3, our simulations use two arm models: a “physical” arm as the plant and the brain’s internal arm model for estimation and control. Though stroke primarily affects the brain, previous studies have shown substantial improvements in upper-extremity motor performance by adding mechanical assistance to the arm [11]. This improvement suggests that cortical planning and estimation remains intact while the issues stem from sub-cortical signal transmission and the execution of motor commands. Because coupling the joints in both the plant and internal model does not result in impaired reaches (tested, but not shown), we explored the effect of joint coupling in the plant only. We simulated center-out reaches to 8 different targets with both a “healthy” (unperturbed) plant and one with abnormal joint coupling.

#### 2) Increased estimation noise

In addition to motor deficits, stroke survivors have difficulty estimating the state of their limbs [6]. Although Cusmano *et al*. [6] demonstrated a stroke-induced increase in proprioceptive bias and noise, simulations with this deficit alone do not display any impairment (tested, but not shown). Poor estimation is more likely caused by increases in the internal model’s prediction error. This is achieved by increasing estimation noise (i.e., process noise within the Kalman filter). For healthy reaches, we assume *σ _{p}* = 0.02 N-m [29]. To examine the effect of poor estimation, we simulated reaches to targets with

*σ*= 0.2, 2, and 8 N-m.

_{p}#### 3) Muscular weakness

Hemiparetic subjects are commonly noted to have muscular weakness. We modeled the effect of this weakness by reducing the maximum torque constraint in the optimization problem to 80% (*α* = 0.8), 40% (*α* = 0.4), and 10% (*α* = 0.1) of the healthy values. We simulated reaches to targets for each weakness level.

## III. Results and Discussion

Planar reaches simulated with our model exhibited characteristics of healthy human motor control: straight reaching trajectories with bell-shaped velocity profiles (Figs. 2 and 3, blue lines). When we systematically perturbed the model in ways associated with post-stroke sensorimotor deficits, each of the perturbations produced changes in motor performance subjectively characteristic of hemiparetic reaches. Although we could have perturbed the model in a subtractive (removing deficits from a fully deficient model) as opposed to additive (introducing deficits to the healthy model), it is unclear how to define “fully deficient.” Moreover, at this stage of model development—prior to rigorous validation— the additive model is at least as instructive as the subtractive one. Subtractive modeling will be important to consider when validating the model, as well as when using the model to assist with the design of rehabilitative devices.

Our simulations indicate that abnormally coupled joints cause systematic direction-dependent perturbations to straight reaching trajectories, target overshoot, and longer settling times. These trajectories are subjectively similar to reaches performed by hemiparetic subjects [39], [40]. Moreover, the sensitivity of the model to the coupling agrees with experimental studies suggesting that abnormal joint or muscle coupling causes the bulk of motor impairments after stroke [10], [11], [40]. The simulation results presented in Fig. 2 show the effects of coupling efferent motor commands without allowing the brain to internally account for this coupling (i.e., capture this coupling in its internal model). For this reason, such a perturbation might be most representative of an acute stroke patient, for whom the mapping between neural commands and muscles has been altered, but the nervous system has not yet had time to learn the new mapping. This type of perturbation might also be representative of stroke patients who were unable to learn an altered mapping between neural commands and muscles during the critical recovery period following the stroke event [41].

While simulations are robust to increases in estimation noise (*σ _{p}* = 0.02 to

*σ*= 2, Fig. 3, panel A), once the noise gets large enough (

_{p}*σ*= 8, Fig. 3), it displays a significant effect on movement trajectories, primarily at the end of a reach as the arm attempts to settle on the target. The resultant “wandering” behavior can be attributed to error compounded during successive estimations over the course of the reach. The resultant trajectories are qualitatively similar to reaches seen in experiments with hemiparetic patients [11]. The trajectories also resemble data collected from patients suffering from deafferentation, a condition that leaves them with almost non-existent sensory input [42], [43]. As seen by comparing panels A.1 and A.2 of Fig. 3, this difficulty settling on a target might vary with reach direction. The severity likely correlates with the relative contributions of each joint to the desired movement, as quantified by the arm’s effective mass/inertia over the course of the reach. The arm is most sensitive to noise in directions of lower effective mass [42]. Regardless, given that a small change in estimation noise can make reaches look healthy again, future research should consider designing rehabilitation protocols that help patients develop robustness to this noise.

_{p}The significance of muscular weakness for post-stroke motor impairment has been debated in the literature (e.g., [9], [44] versus [5], [10]). Our simulations display no motor impairment due to muscle weakness in a planar, gravitysupported reaching task. These findings agree with experimental studies in which stroke survivors who had undergone strength training therapy saw little improvement in motor capability [5], [10], [11]. However, muscular strength is likely to play a much larger role in motor performance in the presence of gravity or external loads (e.g., lifting objects).

Overall, our results indicate that reaching performance is highly robust to prediction error resulting from increased estimation noise, but is greatly impaired by unexpected joint coupling. More generally, the neuromuscular control system seems relatively insensitive to noise but very sensitive to unmodeled dynamics. This is likely thanks to the combination of Bayesian sensory integration [24] and optimization-based control that regularly re-plans motor commands [33]. Therefore, while many control models have focused on stochastic optimal control [45]–[47], a more difficult control problem appears to be the robust one: achieving adept motor performance in the face of poor internal models.

Though the framework presented here incorporates many biologically based parameters including mass/length/inertia properties, nerve conduction delays, and a near-optimal feedback controller thought to best approximate healthy neuromuscular control, some approximations were necessary. First, because nonlinear dynamics make the optimal control problem nonconvex [48], most simulations of upper-limb movements rely on linearized models for which optimal control is well-defined [49], [50]. Our model uses locally linear models that we assume to remain valid over the short (30-ms) time horizon used for MPC optimization. However, the human nervous system is likely able to plan movements over longer time horizons and account for dynamic nonlinearities. Second, while the unscented Kalman filter (UKF) [32] estimates arm state using the fully nonlinear dynamics, it requires the manual selection of several parameters. Third, the applied model perturbations are only approximations of hemiparetic sensorimotor impairments. For example, the abnormal coupling matrix was based on data from subjects performing maximum effort and may not scale linearly to smaller efforts. Despite such approximations, the unperturbed model behaves as expected, providing a tool to understand the map between sensorimotor deficits and motor performance. Model improvements and validation, both driven by data collected from stroke patients, will be the subject of future work.

While many of the sensorimotor deficits encountered by disabled stroke survivors are treatable, the benefits of applying a treatment cannot be foreseen due to the unknown causal relationship between sensorimotor deficits and motor impairments. A computational, parameterized simulation such as the one presented here allows for the selection of subject-specific parameters and sensorimotor deficits. In this way, it has the potential to simulate the effects of specific medical interventions so that only the most effective treatments are implemented on a patient. Additionally, the framework could allow for the optimal design of subject-specific assistive devices. While this has been done in healthy subjects [22], [23], technical limitations have prevented this type of simulated human-in-the-loop design of assistive devices for sensorimotor-impaired populations.

## List of Variables & Parameters

## Footnotes

This work was supported by the Stanford Neurosciences Institute’s Stroke Collaborative Action Network, a Pfeiffer Research Foundation fellowship (a Stanford Interdisciplinary Graduate Fellowship affiliated with the Stanford Neurosciences Institute) to SMS, and a National Science Foundation graduate research fellowship to CSS.