Sensory uncertainty punctuates motor learning independently of movement error when both feedforward and feedback control processes are engaged

Integrating sensory information during movement and adapting motor plans over successive movements are both essential for accurate, flexible motor behavior. When an ongoing movement is off target, feedback control mechanisms update the descending motor commands to counter the sensed error. Over longer timescales, errors induce adaptation in feedforward planning so that future movements become more accurate and require less online adjustment from feedback control processes. Both the degree to which sensory feedback is integrated into an ongoing movement and the degree to which movement errors drive adaptive changes in feedforward motor plans have been shown to scale inversely with sensory uncertainty. However, since they have only been studied in isolation of each other, little is know about how they respond to sensory uncertainty in real-world movement contexts where they co-occur. Here, we show that sensory uncertainty impacts feedforward adaptation of reaching movements differently when feedback integration is present versus when it is absent. In particular, participants gradually adjust their movements from trial-to-trial in a manner that is well characterised by a slow and consistent envelope of error reduction. Riding on top of this slow envelope, participants display large and abrupt changes in their initial movement vectors that clearly correlate with the degree of sensory uncertainty present on the previous trial. However, these abrupt changes are insensitive to the magnitude and direction of the sensed movement error. These results prompt important questions for current models of sensorimotor learning under uncertainty and open up exciting new avenues for future exploration. Author Summary A large body of literature shows that sensory uncertainty inversely scales the degree of error-driven corrections made to motor plans from one trial to the next. However, by limiting sensory feedback to the endpoint of movements, these studies prevent corrections from taking place during the movement. Here, we show that when such corrections are promoted, sensory uncertainty punctuates between-trial movement corrections with abrupt changes that closely track the degree of sensory uncertainty but are insensitive to the magnitude and direction of movement error. This result marks a significant departure from existing findings and opens up new paths for future exploration.

impacts feedforward adaptation of reaching movements differently when feedback integration is 23 present versus when it is absent. In particular, participants gradually adjust their movements 24 from trial-to-trial in a manner that is well characterised by a slow and consistent envelope of error  All one-state models fit poorly to the behavioural data as seen in experiment 1 feedforward ν parameters is also present in the error-scaling and bias-scaling models.

228
In summary, the state-scaling and the bias-scaling models provide a significantly better fit 229 to the behavioral data than the error-scaling model. The addition of matched endpoint feedback 230 did not change the general finding. The presence of feedback integration fundamentally alters how 231 feedforward adaptation is affected by sensory uncertainty and rules out the possibility that feedforward adaptation is influenced by sensory uncertainty differently depending on the temporal 233 and / or spatial proximity of the sensory feedback signal to movement offset.   Rather models that assume that sensory uncertainty acts on non-error terms 248 (e.g., bias-scaling and state-scaling) appear to offer a better account of our data.

249
In Experiment 3, we sought to further probe how sensory uncertainty influences feedback 250 and feedforward error-correction systems. In particular, Experiment 3 disassociates the sensory 251 uncertainty experienced at midpoint from that experienced at endpoint (see Fig. 2C). This allows 252 us to investigate if sensory uncertainty at midpoint dominates sensory uncertainty at endpoint (as  uncertainty. Furthermore, similar to Experiments 1 and 2, we see that in the transition from lower endpoint uncertainty trials to higher endpoint uncertainty trials (e.g., σ 0 , σ 0 → σ 0 , σ H ; 261 σ H , σ 0 → σ H , σ H ) the change in movement vector is anti-adaptive. 262 Figure 8 shows a comparison of one-state and two-state model variants. Here, the 263 behavioral data is overlaid with model predictions and best-fitting feedforward and feedback 264 parameters. Panals A-C show initial movement vectors overlaid with one-state model predictions.

291
As in Experiments 1 and 2, the standard inverse scaling model of feedback integration 292 provided a very good fit to the data (R 2 = 0.98). The best fitting low uncertainty parameter (η 0 ) 293 is significantly larger than the high uncertainty parameter (η H ) (t(38) = 13.1, p < .001, d = 2.9) 294 which indicate that feedback integration scales with uncertainty at midpoint ( Figure 8Q).

296
The current study is the first to examine how sensory uncertainty influences feedforward 297 adaptation and feedback integration when they co-occur. We find that the response of feedback 298 integration to sensory uncertainty is unaltered by the presence of feedforward adaptation (i.e., the 299 extent to which sensory feedback is integrated into an ongoing reach is inversely scaled by its level  show that endpoint feedback, and in turn perhaps explicit aiming strategies keyed to task 364 performance errors, are the driving force behind our data. 365 We should, however, be careful not to endorse this possibility too quickly. First, the best 366 fitting models in our experiments (i.e., the state-scaling and the bias-scaling models) provided 367 good fits to our data without appealing to aiming strategies or otherwise distinguishing between 368 sensory prediction errors and task performance errors. Second, it is also possible that sensory  command is computed at movement midpoint that is an attempt to correct the ongoing 508 movement for any error experienced at midpoint.

509
• Feedforward motor plans are adapted on a trial-by-trial basis using both the error 510 experienced at midpoint and the error experienced at endpoint as a learning signal.

511
• The gain applied to feedback corrections is similarly adjusted on a trial-by-trial basis but is 512 sensitive only to the error experienced at endpoint.

513
• The sensory uncertainty experienced at midpoint and / or endpoint modulates the 514 between-trial feedforward update, and the within-trial feedback correction, but -for 515 simplicity -not the between-trial feedback gain update.

516
The feedforward adaptation component of all three models is based on simple 517 discrete-time linear dynamical systems -so-called state-space models [30]. The simplest version of 518 these models assumes that an internal state variable x maps desired motor goals to motor plans y, 519 and that x is updated on a trial-by-trial basis in response to sensory feedback about movement 520 error. The update to x has (1) an error term that determines how the internal state is updated 521 after a movement error is detected, (2) a bias term that determines the baseline mapping that will 522 be returned to in the absence of sensory input and (3) retention term that determines how quickly 523 the internal state returns to baseline after sensory feedback about error is removed. This 524 arrangement is encapsulated in the following equations: where n is the current trial, δ(n) is the error (i.e., the angular distance between the reach 529 endpoint and the target location), y * (n) is the desired output (e.g., the angular position of the 530 reach target), y(n) is the motor output and corresponds to the angle of the movement that will be 531 generated when trying to reach to the target (i.e., it is a readout of the sensorimotor state), x(n) 532 is the state of the system (i.e., the sensorimotor transformation), β is a retention rate that describes how much is retained from the value of the state at the previous trial, α is a learning 534 rate that describes how quickly states are updated in response to errors, λ is a constant bias, and 535 r(n) is the imposed rotation.

536
These models are sometimes equipped with a second internal state variable [36, 45] as 537 follows: 542 where x f is a fast state variable, x s is a slow state variable, β f < β s , and α f > α s . That is, 543 feedforward adaptation is assumed to arise from the combination of a slow-but-stable system and 544 a fast-but-labile system. Existing studies of have not strongly delineated the appropriateness of 545 one-state versus two-state models in mapping out how sensory uncertainty influences feedforward 546 adaptation, and as such, we explore both one-state and two-state model variants in this paper.

547
The simple state-space framework just described assumes motor output reflects the 548 execution of feedforward motor commands, whereas the behavior observed in our experiments is 549 also likely influenced by feedback motor commands. We therefore augment the simple models 550 presented above as follows.

551
The total motor output of the model is defined at three discrete time points within each 552 trial. We denote the time of reach initiation as t 0 , the time of midpoint crossing as t MP , and the 553 time of endpoint crossing as t EP . The total motor output on trial n at any time t denoted y(n, t) 554 is a combination of feedforward y ff (n) and feedback y fb (n, t) motor commands as follows: 555 y(n, t) = y ff (n) + y fb (n, t) (8) 556 Note that feedforward motor output is not a function of time within a trial because we assume 557 that the feedforward motor output is computed at t 0 and remains fixed throughout the rest of each trial. This is equivalent to assuming that the execution of the movement occurs too rapidly 559 for new feedforward motor planning to influence the ongoing movement.

560
In the single-state models, the feedforward motor command y ff (n) is determined by a 561 single internal state variables denoted by x ff (n) that maps the current movement goal to motor 562 commands as follows: In the two-state models, the feedforward motor command y ff (n) is determined by two internal 565 state variables denoted by x ff f (n) and x ffs (n) that map the current movement goal to motor 566 commands as follows: At reach initiation, sensory feedback has not yet been provided so the feedback motor 569 command is zero: If sensory feedback is provided at midpoint, then the following sensory prediction error is 572 experienced: Here, δ(n, t MP ) is the sensory prediction error, and r(n) is the visuomotor rotation applied on 575 trial n. Notice that the motor command issued at time t 0 is responsible for generating the sensory 576 prediction error at time t MP . In response to this sensory prediction error, the following 577 compensatory feedback motor command is triggered: 578 y fb (n, t MP ) = −x fb (n)δ(n, t MP )ηI(n) Here, x fb (n) is an internal state variable that represents the gain of the feedback controller, (2) the best-fitting parameters were such that ν 0 > ν M > ν H > ν ∞ .

636
State-scaling models 637 The uncertainty of sensory feedback influences the update to x ff in the single-state Bias-scaling models 648 The uncertainty of sensory feedback influences the update to x ff in the single-state 649 state-scaling model by acting as a gain on the bias term λ. The update is given by: All parameters and nomenclature are identical to that described above for the error-scaling

668
For each model, we computed the Bayesian Information Criterion (BIC) as follows:

Figure 1
Kinarm Endpoint Robot. Green start target, red end target, and white cursor feedback are represented.

Figure 4
Panels A-C show initial movement vectors averaged across participants in Experiment

Figure 6
Panels

Figure 8
Panels