Abstract
An important window into sensorimotor function is how we catch moving objects. Studies that examined catching of free-falling objects report that the timing of the motor response is independent of the momentum of the projectile, whereas the motor response amplitude scales with projectile momentum. However, this pattern may not be a general strategy of catching since objects accelerate under gravity in a characteristic manner (unlike object motion in the horizontal plane) and the human visual motion-processing system is not adept at encoding acceleration. Accordingly, we developed a new experimental paradigm using a robotic manipulandum and augmented reality where participants stabilized against the impact of a virtual object moving at constant velocity in the horizontal plane. Participants needed to apply an impulse that mirrored the object momentum to bring it to rest and received explicit feedback on their performance. In different blocks, object momentum was varied by an increase in its speed or mass. In contrast to previous reports on free falling objects, we observed that increasing object speed caused earlier onset of arm muscle activity and limb force relative to the impending time to contact. Also, arm force increased as a function of target momentum linked to changes in speed or mass. Our results demonstrate velocity-dependent timing to catch objects and a complex pattern of scaling to momentum.
Introduction
When humans catch a moving projectile such as a baseball, they produce large impulsive forces calibrated to the impact along with patterned feedback responses (Lacquaniti and Maioli 1989a; Lacquaniti et al. 1993a). Understanding how visual information sculpts these actions is central to understanding the neural processes underlying catching. In the late 80s, Lacquaniti and colleagues performed a series of seminal catching studies, which showed that projectile kinematics directly influenced activation of muscles both before and during impact (Lacquaniti and Maioli 1989b; Lacquaniti and Maioli 1989a; Lacquaniti et al. 1991; Lacquaniti et al. 1992; Lacquaniti et al. 1993b; Lacquaniti et al. 1993a). Specifically, they showed two interesting results. The anticipatory muscle responses were tuned parametrically to the estimated momentum (mass x speed) of the projectile (reviewed in Fig. 3, Zago and Lacquaniti 2005) such that the consequent increase in limb impedance stabilized the arm during the large and transient transfer of mechanical impulse from the projectile to the arm. In addition, ball speed appeared to have little to no effect on the timing of anticipatory muscle activation prior to collision (reviewed in Fig. 2, Zago and Lacquaniti 2005).
Parametric tuning to estimated momentum provides evidence of a forward model simulating the effect of gravity on the falling object which aligns with many other studies pointing to models of gravity for vestibular-ocular function (Merfeld et al. 1999; Angelaki et al. 2004) and limb motor function (Miall and Wolpert 1996; McIntyre et al. 2001). However, the result of invariant timing is somewhat surprising as it runs counter to the predictions from the well-supported Tau hypothesis (Gibson 1966; Regan and Vincent 1995; Lee 1998; Tresilian 1999). The Tau hypothesis predicts that during interception movements, motor responses are initiated when the time-to-collision (Tau) between the projectile and the body reaches a threshold. Note that this is actually a modification of the original hypothesis which dealt with objects heading to the head and requires an additional transform for the head to hand distance (Tresilian 1999). Thus, for the ball-drop paradigm used by Lacquaniti and colleagues (Lacquaniti et al. 1993a), the Tau hypothesis predicts that balls dropped from greater heights would evoke earlier motor responses relative to the actual time-of-collision. Yet Lacquaniti and colleagues reported that this is not the case for free falling objects.
Humans appear to use model-based priors of gravity to guide limb movements for free falling objects (McIntyre et al. 2001; La Scaleia et al. 2015) rather than direct measures of acceleration, likely since our visual motion-processing system is quite poor at acceleration (McKee 1981; Werkhoven et al. 1992; Trewhella et al. 2003), a fact which motivated the conclusions of Lacquaniti et al. However, it is possible that free fall reflects an important and ecologically valid special case and not a general strategy of catching. Objects accelerate under gravity in a characteristic manner unlike the wide range of motion patterns in the horizontal plane. So, a fuller understanding of catching requires a paradigm involving catching in the horizontal plane.
To replicate the physics of mechanical interactions with horizontally moving projectiles, we use an augmented virtual environment which can present visual objects and simulate the physical equivalence between momentum and applied impulse (area under force-time curve) defined by Newton’s Second Law. To intercept a projectile and bring it to rest, an impulse equivalent to the momentum of the projectile right before collision has to be applied. Imagine we are playing catch with a friend who either throws a baseball (∼150 gm) or a heavier softball (∼200 gm) at us. If we assume that the balls are thrown with similar speeds of 15 m/s, and that both the balls come to rest within the same time once they contact the hand (∼100 ms), then since the softball is 33% heavier, we will have to apply a 33% higher force to stabilize our arm. But what if in some trials, the friend throws the baseball at a 33% higher speed (20 m/s). Since the momenta of the two balls are now identical, the same feedforward motor response would be sufficient to stop both the balls. So, would the motor system prepare a generic feedforward response that is independent of target motion or would the visual motion-processing system alter the timing of motor responses (Ilg 2008; Zago et al. 2009)?
We hypothesized that the amplitude of the feedforward motor responses are modulated based on a forward-model (Miall and Wolpert 1996; Bastian 2006) of the mechanical interaction between the projectile and the hand. A key prediction from this hypothesis is that the amplitude of the feedforward motor response would scale with target momentum, regardless of whether the momentum increased due to speed or mass. We also hypothesized that motor responses are timed based on a model-free online motion-processing system (La Scaleia et al. 2015) that likely uses an efference copy of smooth pursuit eye movements as a feedforward input to guide interceptive hand movements (reviewed in Ilg 2008; de Brouwer et al. 2021). The naïve prediction from this hypothesis is that faster target velocities will initiate early motor responses. Our results support the first prediction; the maximum feedforward force applied by the hand prior to the collision scaled as a function of target momentum. Our second prediction was not supported. In fact, we found that when the target traveled at faster speeds, participants increased hand force much closer to the time of collision, i.e., later motor responses were neither predicted by the Tau hypothesis nor the application of a model-based control. This suggests that alternative factors might affect timing of motor responses.
Methods
Participants
Ten participants (20.8 ± 1.5 years; 5 women) completed the study. All participants were right-handed, had no history of neurological disorders, and had no current injuries or pain of the upper limbs. Each participant provided written informed consent prior to participating and were compensated for their time. All procedures were approved by the local Institutional Review Board of the University of Georgia.
Apparatus
The task was performed with a robotic manipulandum (KINARM End-Point Lab, KINARM, Kingston, ON, Canada) that participants grasped with their right hand. The robotic arm could be moved in a horizontal plane and a mobile arm support (SaeboMAS, Saebo Inc., Charlotte, NC, USA) was used to support the weight of the participant’s arm to avoid fatigue. EMG activity of the biceps brachii, triceps brachii, anterior deltoid, and posterior deltoid was recorded with bipolar surface electrodes (Bagnoli, Delsys Inc., Boston, MA, USA) during the experiment. Visual objects were generated using an online Gabor-patch generator (https://www.cogsci.nl/gabor-generator). Gabor patches are sinusoidal gratings convolved with a Gaussian envelope that are typically used in psychophysics research to elicit strong neural activation in the early visual sensory areas, including MT/MST (Olshausen and Field 1996; Takeuchi 1998).
Experimental design and procedure
Participants performed a task called “Catch the object”, where the objective was to “virtually catch” a virtual circular object moving in the negative Y direction (Fig. 1A) by applying a force pulse to bring its momentum to 0. We assigned a virtual mass to an object and then multiplied it with its speed (in the KINARM frame of reference) to obtain the momentum of the object. Since the change in momentum and applied impulse (area under force-time curve) are equivalent (Eq. 1, Newton’s Second Law), we converted the momentum of the object into an impulse template (see Fig. 1B) that the KINARM robot applied to the participant’s hand when the object came in contact with the hand (right panel, Fig. 1A). This allowed us to replicate the mechanics of collision during virtual catching.
Experimental setup. A) Once participants moved their hand into the rectangular box (black boundary) and stabilized their arm against a background load, a fixation cross appeared (not shown). 200 ms after the cross disappeared, a circular Gabor patch (object) appeared and moved towards the participant. Participants were instructed to match the force applied by the robot during the collision to intercept the object. B) The force-time curves (left and middle panels) for the perturbation applied by the robot in two speed sub-conditions (SHigh and SLow). The arrows indicate hypothetical hand force trajectories, and their intersection with the x-axis indicates the onset timings and the predictions from our hypothesis. The area under the curve, the impulse, is exactly equal to the momentum carried by the virtual object. The right panel shows the impulse applied by the robot in the Speed and Mass conditions. Note that the forces applied by the robot should be negative, but are shown as positive for visualization purposes here and in panel C.
Here, the term on the left-hand side is the change in momentum; the term on the right-hand side is the impulse, the integral of force and time. To bring an object approaching the body with a speed V and an assigned mass ‘m’ to a complete stop, the hand will have to apply a time-varying force F(t) over Δt s. We fixed Δt to 90 milliseconds (0.09 s). When the object came in contact with the hand, the robot applied the impulse on the left-hand side of equation 1 over 90 ms with a 10 ms long rise time and fall time and a 70 ms steady state force. Participants experienced that impulse as simulating a collision between the object and the hand. During the collision between the hand and the object, participants were required to match that impulse applied by the robot as closely as possible to bring the object to a stop. Exemplar force profiles applied by the robot are shown in Figure 1B (left and middle panels) for objects with different speeds and masses.
To test the predictions from our hypothesis, we created two different experimental conditions (Speed and Mass). In the Speed condition, we varied object momentum across blocks by varying the object speed (low or high), and in the Mass condition, we varied momentum by changing the object “mass” (low or high). Thus, the Mass condition simulated objects with different masses but same speeds and vice-versa. SLow (low speed) and MLow (low mass) had identical momenta (and impulses applied by the robot) and very similar object speeds (see right panel of Fig. 1B and Table 1). Similarly, SHigh and MHigh had identical momenta but very different object speeds. The momentum difference between the low and high sub-conditions was ∼44%. To differentiate the objects in the two Mass sub-conditions, we used Gabor patches with different colors in the MLow and MHigh sub-conditions. The Speed condition simulated objects that moved at different speeds but had a constant mass (same color objects) across sub-conditions, with higher momentum produced by objects with higher speeds.
Participants performed two blocks of each sub-condition (4 total) with 35 trials in each block. The order of the conditions was counterbalanced across participants, half of the participants performed blocks within the Mass condition first and the other half performed the blocks within the Speed condition first. Within each condition, the order of the sub-conditions (Low and High) was randomized for each participant and across participants.
At the beginning of each trial, participants moved the cursor that represented their hand location into a designated area (5 x 10 cm, rectangle) and maintained a static hold until the end of the trial. If their hand escaped the rectangle, the trial was aborted and repeated. When participants entered their hand in the rectangle, a background load (4 N in the -Y direction, see Fig. 1A) was applied and it stayed on for the remainder of the trial. Background loads were applied to minimize unwanted hand movements (Singh et al. 2017; Barany et al. 2020). After a fixed delay of 1,700 ms, a fixation cross appeared in the middle of the screen (20 cm away from the center of the rectangle). Participants were instructed to fixate on the cross until it disappeared (600 ms). Then a Gabor patch (1 cm radius) appeared on the same position 200 ms after the cross disappeared and immediately started moving towards the middle of the rectangle mimicking a virtual object.
When the object reached the position of the hand cursor, participants were asked to “catch the object” by matching the force impulse applied by the object. The precise instruction given to the participant was “match the force applied to the hand so that the object stops.” Participants were allowed to move their hand as long as the hand stayed within the rectangle. It is important to note that the mechanical interaction during the collision is primarily determined by feedforward commands to muscles and muscle’s intrinsic viscoelastic properties (Burdet et al. 2013) because the 90 ms collision duration is too short for voluntary feedback correction. Though fast feedback corrections are observed in muscle activity within the first 50-100 ms after mechanical perturbations (reviewed in Scott 2012; Kurtzer 2015), hand force is likely low-pass filtered by the musculoskeletal system (Burkholder 2016) and consequently may not include effects of feedback responses.
The performance of a participant in three different trials of the SLow sub-condition along with the impulse applied by the robot (FRobot as a function of time) is shown in Figure 2A. In each trial, based on their performance during the collision, participants were given two forms of feedback – one with the object and another one with a visual scale (see Fig. 2B). Object feedback had three different levels: if the impulse was matched within a ±5% margin of error (see Eq. 2), that would imply that an appropriate impulse was applied to match the “momentum” of the object and the object would stop after the collision (reflecting a successful catch). If the impulse applied by the participant was higher than the 5% margin of error, that would imply that more than necessary impulse was applied, and the object would “fly back” and move in the positive Y direction. Lastly, if the impulse applied by the participant was lower than the margin of error, that would imply that the impulse applied by the hand was inadequate to stop the object momentum, and the object would continue unabated on its original trajectory in the negative Y direction.
Performance feedback. A) Exemplar trials from a participant in three different situations when the participant applied less than 95% of the required impulse (left), an impulse within 95-105% of the desired impulse (middle), and more than 105% of the impulse (right). The red curve is the impulse applied by the robot and the cyan is the impulse applied by the participant. B) The corresponding feedback provided to participants for the three trials in A. The black bar on the horizontal line indicated how far off they were from the desired impulse (middle of green section of the horizontal line).
However, from pilot studies we learned that this form of discrete feedback was inadequate for participants to improve their performance. Thus, we also provided a more continuous measure of performance by adding a visual analog scale that had a green center region surrounded by red region. The scale was set up based on Equation 2 with the green region indicating the ±5% margin of error and the red regions indicating how low or high the applied impulse was from the impulse required to bring the object to rest. Examples of the provided feedback are shown in Figure 2B. The inter-trial delay was 2,500 ms.
Data recording and analysis
Hand kinetics and kinematics were sampled at 1 kHz and digitally low-pass filtered (second-order, dual-pass Butterworth, 50 Hz effective cutoff). For each sub-condition, we first calculated the success rate for each block and then calculated the average across blocks, Successful Trials. For each trial within a block, we calculated the percentage difference between the impulse applied by the robot and the participant (see Equation 2), Δ Impulse (%).
Participants typically increased their hand force and moved their hand slightly toward the object right before collision. Thus, we quantified when the hand force and hand speed increased above baseline values in anticipation of the collision with the object during the static hold. We first calculated the mean force applied against the background load in an interval [-500, -400 ms] prior to the collision and then subtracted it from the peak force recorded from the force sensor during collision. Then starting from the point of the collision, we went backward in time and found the first time point when the hand force dropped below 5% of the peak force value. We called this point FOnset (see Fig. 3A).
Performance variables for subsequent analysis. A) The left y-axis shows the force applied by the participant in the Y direction (grey curve) to counter the impulsive force applied by the robot (red curve) during the collision with the virtual object. The shaded area indicates the impulse applied by the robot (red) and participant (grey). The right axis shows the averaged EMG activity of the triceps (brown curve) over one block of trials. The triangles indicate onset of EMG (brown), force (grey), and hand movement (black), respectively. The circles indicate the peak values before collision. B) The left axis is the same as panel A and the right axis shows hand velocity in the Y direction (black curve).
We first calculated the object location along the Y-axis (see Fig. 1A) at FOnset and called it distance-to-collision (DTCBody). At the same time point, we also calculated the distance along the Y-axis between the hand and the object at called it distance-to-collision-hand (DTCHand). This variable was calculated because it has been suggested that the nervous system plans hand movements by computing the hand-to-object difference vectors based on hand and object locations encoded in an eye-centered frame of reference (Batista et al. 1999; Beurze et al. 2006).
In addition, we computed two time-to-collision variables (see Equation 3). First, we computed τBody, which indicated the time-to-collision between the current object location and the body (y=0, Fig. 1A) if the object continued to move uninterrupted. This variable was computed by dividing the DTCBody by object speed (Lee 1998; Tresilian 1999). Then we computed the time-to-collision between the object and the hand, TTCHand, by dividing DTCHand by object speed. These two variables accounted for the relative differences in object speed for the different conditions and sub-conditions (see Table 1). We also calculated the point of hand movement initiation using the same logic as force. However, we found the times for force and hand movement initiation were very similar (see Fig. 3B), so we only used the force data for all subsequent analyses.
We then calculated two parameters for the feedforward control of hand force. First, we calculated the hand force right before the collision and called it FPeak. We then fit a first order polynomial to the force data between the time of FOnset and FPeak and computed the slope of the fitted line. This slope provided a measure of rate of force increase (dF/dt) between TTCHand and collision.
EMG signals were also recorded at 1 kHz. Before placing the electrodes, the skin surface over each muscle was shaved, exfoliated and then cleaned with alcohol to improve signal conductance. Electrodes were attached to the skin using double-sided tape; the ground electrode was placed on the subject’s olecranon. The amplifier gain for the EMG signals was set at 104. EMG data were first band-pass filtered between 20 and 450 Hz with a 6-th order Butterworth filter with forward/backward pass to eliminate delays (Kurtzer et al. 2020) and then full-wave rectified. For each muscle and each trial, we first subtracted the baseline activity when the hand was in static hold in the rectangle and the background load was on. We used a 200 ms window when the fixation cross was on to calculate the baseline activity. The band-pass and rectified filtered signal was averaged, and this value was subtracted from the raw muscle activity. We then normalized the muscle activity using the same procedure when participants held a 1 N background load (in a different trial).
The EMG signals for each muscle were then ensemble averaged over all trials for each block and then averaged over the two blocks. To determine the timing of onset of muscle activity, EMGOnset, we used the same algorithm as for FOnset, but we chose the cutoff as 15% instead of 5% to account for the higher levels of noise in the EMG signals. We also compared the amplitude of normalized EMG activity within [-100, 0] ms window prior to the collision and [25-50] ms (short-latency) and [50-100] ms (long-latency) windows after the collision was initiated. For plotting purposes of the EMG data, we also used a moving average window of 20 ms (Cluff and Scott 2013). To quantify how the shoulder and the elbow joints were stabilized before (feedforward) and during collision (feedback), we calculated an index of co-contraction by calculating the mean of the rectified EMG signals; for the shoulder joint, we averaged the anterior and posterior deltoids and for the elbow joint, we averaged the biceps and triceps muscles.
We treated the first three trials in each block as practice and only performed analyses on the remaining 32 trials. All analyses were performed in MATLAB (Mathworks R2020b, Natick, MA).
Statistics
We conducted paired t-tests between the two sub-conditions for both the Speed and Mass conditions. This study was not a conventional 2×2 design because object speeds were quite similar in SLow and MLow and the momenta of the object were exactly identical. However, because the blocks within the Speed and Mass conditions were performed separately and because the visual features of the objects were different across the two blocks, SLow and MLow served as separate control blocks for the Speed and Mass conditions, respectively. To test the predictions from our main hypothesis, we expected large effects and significant differences for SHigh – SLow, but not for MHigh - MLow. Outliers that were 1.5 interquartile ranges above the upper quartile or below the lower quartile were removed from analysis. Effect sizes were calculated using Cohen’s d. The level of significance was set at α=0.05. All values are reported as mean ± SE. All statistical analyses were performed in R (version 4.0.4).
Results
Performance accuracy improved when the object speed increased
On average, participants showed a marginal improvement when the object speed was high than slow. Participants applied impulses within the acceptable range of 95-105% of the impulse applied by the robot in 36.2% and 37.9% of the trials in the SLow and MLow sub-conditions, respectively, compared to 40.6% and 42.1% in SHigh and MHigh sub-conditions, respectively. However, only the marginal improvement in the Speed condition was significant. Participants performed more accurately in the SHigh sub-condition compared to the SLow sub-condition (p=0.019, Cohen’s d=0.50). There were no differences in success rate between the Mass sub-conditions (see Fig. 4A).
Performance accuracy increased with object speed. A) Surprisingly, despite the results shown in panels A and C, participants performed a higher number of successful trials in the high impulse conditions. This difference was significant for the Speed condition. B) The percentage difference between the impulse applied by the participant and the robot, Δ Impulse (%), followed the same trends as the absolute difference and high impulse conditions induced larger differences in the two impulses. The cyan region indicates the error margin.
Δ Impulse (%) for SLow was on average negative and significantly smaller in magnitude than SHigh (p<0.01, Cohen’s d=0.57, Fig. 4B). For the Mass condition, Δ Impulse (%) for MLow was also negative and significantly smaller in magnitude than MHigh (p=0.018, Cohen’s d=0.53). Together, these results suggest that participants did not proportionately increase the impulse applied by the hand with an increase in object momentum, either when the momentum increased because of speed (SLow to SHigh) or “mass” (MLow to MHigh). This caused a larger mismatch applied between the impulse applied by the robot and the participant during high momentum blocks than the low momentum blocks.
Hand force was initiated with shorter time-to-collision for faster object speeds
DTCHand and DTCBody were significantly larger for SHigh than SLow. At the time when the hand force was initiated, DTCHand was 3.97±0.34 cm in the SLow sub-condition, and 4.37±0.37 cm in the SHigh sub-condition (p<0.01, Cohen’s d=0.37, Fig. 5A). Similarly, at the same time, DTCBody was 16.2±0.43 cm in the SLow sub-condition and 16.69±0.44 cm in the SHigh sub-condition (p<0.001, Cohen’s d=0.37, Fig. 5B). There were no significant differences for the two variables between the Mass sub-conditions. Secondary analyses revealed that there were also significant differences between SHigh and MHigh for DTCHand (p<0.01, Cohen’s d=0.94) and DTCBody (p<0.01, Cohen’s d=0.66). Together these results imply that faster object speeds caused participants to increase their hand force when objects were farther away from the hand and the body.
Objects traveling at faster speeds elicited a response when the object was farther away. Both DTCHand (A) and DTCBody (B) were larger larger for SHigh compared to SLow. There were no differences in the Mass sub-conditions. This suggests that object speed impacted when participants started to increase the hand force in preparation of the collision.
The time-to-collision between the object and the hand when the limb force was initiated (TTCHand) decreased from 0.188±0.012 s for SLow to 0.154±0.012 s for SHigh (p<0.01, Cohen’s d=0.95, see Fig. 6A). The difference between the Mass sub-conditions were small (Cohen’s d=0.16) and non-significant (p=0.29). Similarly, the time-to-collision between the object and the body (TTCBody) varied strongly with object speed. For the Speed condition, TTCBody decreased from 0.80±0.02 s for SLow to 0.60±0.01 s for the SHigh sub-condition (p<0.001, Cohen’s d=3.86). The differences between MLow and MHigh were again miniscule (Cohen’s d=0.05) and non-significant for the Mass condition (see Fig. 6B).
Time-to-collision between object and hand at hand force initiation was shorter for objects moving at high-speeds. TTCHand (panel A) and TTCBody (panel B) were significantly shorter for SHigh compared to SLow. No differences wer observed for the two Mass sub-conditions.
Taken together, these results suggest that in preparation for the collision between the object and the hand, when the object traveled at faster speeds, participants increased hand force above baseline levels much closer to the time of collision. These results do not support our hypothesis and suggest that alternative mechanisms might be responsible for limb dynamics in our task (Smeets et al. 1996).
Object speed modulates feedforward modulation of hand force in preparation for collision
We first looked at the hand force as a function of time leading up to and after the collision (see Fig. 7A). In the Mass condition, the temporal dynamics of the average hand force for MLow and MHigh were parallel all the way up to the collision and consequently the dF/dt was similar between the two sub-conditions (see right panel of Fig. 7B). For the Speed condition, dF/dt was significantly higher for SHigh compared to SLow (p<0.01, Cohen’s d=0.67, Fig. 7B).
Higher object speeds increased the rate of feedforward force increase. A) The mean ± SE of the force trajectories for all trials in a [-200, 200] ms window is shown for the Speed (left) and Mass (right) conditions. For the Mass sub-conditions, the mean force trajectories remain parallel (see right inset) throughout the pre-collision and collision durations, whereas for the Speed sub-conditions, SHigh force trajectory crosses the trajectory for SLow suggesting that participants increased hand force later in SHigh trials but increased the rate at which the force was increased. B) dF/dt was significantly higher for SHigh compared to SLow; there were no differences between the Mass sub-conditions. C) As expected, FPeak was significantly higher for the high momentum conditions (SHigh and MHigh).
For SLow and SHigh, FPeak was 5.83±0.16 N and 6.3±0.22 N, respectively. FPeak was significantly higher for SHigh than SLow (p<0.01, Cohen’s d=0.82, Fig. 7C). However, in terms of absolute magnitude, this difference was rather small (∼0.5 N, 8.6%) compared to the 43% increase in object momentum from the Low to the High sub-conditions. Similarly, FPeak was higher for MHigh (p<0.001, Cohen’s d=0.47, Fig. 7C) but the absolute differences were small. For MLow and MHigh, FPeak was 6.0±0.21 N and 6.31+0.22 N, respectively. This suggests that participants applied a marginally higher hand force during the high momentum conditions. However, the marginal change suggests that FPeak values may not be consequential to the overall task performance.
Together, these results suggest that object speed contributed to the rate at which participants increased hand force in preparation of the collision between the hand and the object. The results also partially support our prediction from the hypothesis that the amplitude of the feedforward motor response would scale with the momentum of the object, regardless of whether the momentum increased due to speed or mass. FPeak scaled with the object momentum, but this increase was too small to be of biological significance.
We also observed that the peak hand velocity prior to collision (VPeak) also scaled with object speed. Vpeak for SLow was 14.92 ± 1.63 cm/s and increased to 16.73±1.70 cm/s for SHigh (p<0.01, Cohen’s d=0.39). There were no differences between the Mass sub-conditions. Together, these results along with the time-to-collision results suggest that for faster moving objects, participants waited till the object was ∼30 ms closer to colliding with the hand, then rapidly increased the hand force, and moved the hand more vigorously towards the object in preparation of the collision.
Object speed modulated the timing of activation and co-contraction of the muscles prior to collision
Participants moved their hands toward the objects by extending their elbow and adducting their shoulders. Since we predicted that faster object velocities would initiate early motor responses, we predicted that the prime movers at the shoulder (anterior deltoids) and elbow (triceps) would “turn on” sooner for faster moving objects. In contrast to that expectation, we observed the opposite results.
The averaged muscle activity for both the conditions is shown in Figure 8. We found that EMGOnset for anterior deltoids was on average ∼25-30 ms closer to the collision for the SHigh sub-condition compared to the SLow sub-condition (p=0.015, Cohen’s d=0.76, Fig. 8A). We also observed that EMGOnset of triceps activity changed and was ∼10 ms closer to the time of collision for the SHigh condition. Surprisingly, this small change was significant and had a sizable effect size (p=0.003, Cohen’s d=0.54, Fig. 8C). All the other onset times did not reach significance and had small effects (Figs. 8B & D).
Faster object speeds caused temporal changes in muscle activation. For SHigh, the activity in the anterior deltoids increased above baseline (panel A) closer to the time of collision (∼30 ms). Though it appeared that the higher momentum in the Mass condition appeared to activate posterior deltoids earlier, this difference was not significant (p=0.19) and had a small effect (Cohen’s d=0.27) (panel B). No other differences were observed for the distal muscles in either the Speed or Mass conditions (panels C and D). The black vertical line indicates the time of collision between the hand and the object. The colored lines indicate EMGOnset. The red vertical lines indicate short- and long-latency windows.
Except for the triceps muscles in the Speed condition, we did not find any significant differences in the mean normalized EMG amplitudes of the four muscles in three windows around the collision time, [-100, 0], [25-50], and [50-100] ms. These windows correspond to the feedforward, short-latency, and long-latency windows, respectively. For the triceps muscles, the mean amplitude of activity was higher for SHigh in the feedforward (p=0.02, Cohen’s d=0.27), short-latency (p=0.01, Cohen’s d=0.33), and long-latency (p<0.01, Cohen’s d=0.43) windows.
We observed no significant differences in the feedforward co-contraction of the proximal muscles (anterior and posterior deltoids) between the sub-conditions for both Mass and Speed (see Fig. 9A and 9B). For the distal muscles, we saw a significant increase in the feedforward co-contraction of the distal muscles (biceps and triceps) for SHigh compared to SLow (p=0.023, Cohen’s d=0.52, see Fig. 9C). There were no differences in the feedforward co-contraction of the distal muscles for the Mass condition (see Fig. 9D).
Higher object speeds increased muscle co-contraction at the distal joint. At the proximal muscles, we saw an increase in co-contraction only in the long-latency window [50-100] ms (A) during the SHigh sub-condition. In contrast, participants increased co-contraction of antagonist muscles at the distal joints (C), during prior to collision [-100,0], during the short-latency [25-50] ms, and the long-latency windows. Participants did not increase co-contraction of muscles at either the proximal (B) or the distal joint (D) in the Mass sub-conditions. In fact, we observed a small decrease in co-contraction of distal muscles for MHigh during the short-latency window. The black vertical line indicates the time of collision between the hand and the object. The red vertical lines indicate short- and long-latency windows.
For the feedback responses in the short-latency window, we observed an increase in co-contraction for the distal muscles for SHigh compared to SLow (p=0.018, Cohen’s d=0.56, see Fig. 9C). We also observed a significant decrease in the co-contraction in this window for MHigh compared to MLow, but this is likely a false positive (p=0.014, Cohen’s d=0.19, see Fig. 9D). For the feedback responses in the long-latency window, we observed an increase in co-contraction for the proximal muscles (p=0.037, Cohen’s d=0.38, see Fig. 9A) as well as distal muscles for SHigh compared to SLow (p=0.011, Cohen’s d=0.69, see Fig. 9C).
It is important to note here that only object speed and not momentum contributed to the change in timing and increase in co-contraction prior to the collision. Higher object momentum in the SHigh and MHigh conditions resulted in stronger impulses applied by the robot during collision. But participants did not modulate the timing and amplitude of muscle activity differently between the Mass sub-conditions.
Feedforward modulation of hand force affects overall task performance in the Speed condition
The task that we developed with the intent of understand how object motion affects timing and amplitude of motor responses was not an easy motor task; we did not see any of evidence of improvement in task performance across trials or between blocks. But we still wanted to check if there were any correlations between the modulation of motor responses and overall task performance in the Speed and Mass conditions.
We first correlated TTCBody and TTCHand with dF/dt. We found strong correlations between TTCBody and dF/dt for all sub-conditions (Fig. 10A). Qualitatively, the slopes and intercepts for the Mass sub-conditions were similar, but different for the Speed sub-conditions. We also found correlations between TTCHand and dF/dt but they did not reach significance for the Speed sub-conditions (Fig. 10A). Again, the slopes and intercepts for the Mass sub-conditions were qualitatively very similar but they were different between the two Speed sub-conditions. Together, these results suggest that TTCBody was a relatively better predictor of how the limb motor response was modulated prior to the collision.
Rate of hand force increase impacted overall task performance. A) The time-to-collision was a strong predictor of the rate at which participants increased force prior to the collision between the hand and the object. The rate of hand force increase was strongly predictive of the peak feedforward force in both the Speed and Mass conditions (B). The Peak Feedforward force predicted the percentage difference in the impulse that the participants applied and the one applied by the robot for the Speed sub-conditions, but not the Mass sub-conditions (C). The cyan region indicates the error margin.
We then correlated the dF/dt and FPeak and found strong correlations between the two measures for all four sub-conditions (see Fig. 10B). We also observed very similar regression slopes for all the sub-conditions, but the higher impulse sub-conditions had slightly more positive intercepts. Though this suggests that for the same rate of force increase, participants applied slightly higher peak feedforward forces prior to the collision, these feedforward forces did not scale with the actual flat force applied by the robot during the middle 70 ms of the 90 ms collision. This force for the low momentum sub-conditions (both Speed and Mass) was 9.2 N and for the high sub-conditions was 11.3 N. However, FPeak was on average only 0.2-0.3 N higher for the High sub-conditions suggesting that participants increased the peak feedforward force in anticipation of the stronger collision but did not scale it to the actual magnitude of the robot applied force.
Finally, we correlated FPeak with Δ Impulse (%) and found moderate to strong and significant correlations for both the Speed sub-conditions, but not the Mass sub-conditions (see Fig. 10C). This suggests that the feedforward control of force may have influenced overall task performance in the Speed but not the Mass condition.
Discussion
In the current study, we used a virtual catching paradigm and simulated the mechanics of the interaction between the object and the hand based on Newton’s Second Law. We simulated the interaction such that the “momentum” (mass x velocity) of the object could be stopped by applying an equivalent mechanical impulse (integral area of force-time curve). If the applied impulse exceeded the object momentum, the object bounced back. If it was less than the object momentum, the object continued on its original trajectory. Participants also received performance feedback at the end of every trial.
In different conditions, we either varied the speed (Speed) or the mass (Mass) of the object to alter its momentum. We hypothesized that the amplitude of the feedforward motor responses would be modulated based on a forward-model of the mechanical interaction between the object and the hand.
The prediction from this hypothesis was that the amplitude of the feedforward motor response would scale with the momentum of the object, regardless of whether the momentum increased due to speed or mass. Our results partially supported this prediction (see Figs. 7C & 9C). The maximum feedforward force applied by the hand prior to the collision scaled as a function of object momentum (in both Speed and Mass conditions). In the Speed condition, faster object speeds also caused an increase in co-contraction of the distal muscles (triceps and biceps) in a 100 ms window prior to the collision. In the short-latency window (25-50 ms), we observed an increase in co-contraction of the distal muscles. In the long-latency (50-100 ms) window, we found an increase in co-contraction of both the distal and proximal muscles.
We also hypothesized that motor responses are timed based on a model-free online motion-processing system that likely uses an efference copy of smooth pursuit eye movements as a feedforward input to guide interceptive hand movements. The prediction from this hypothesis was that faster object velocities will initiate early motor responses. Our prediction was incorrect. In fact, we observed the exact opposite results. When the object traveled at faster speeds, participants increased hand force much closer to the time of collision suggesting alternative mechanisms might be at play that affect timing of motor responses.
Signal-dependent noise associated with visual tracking might influence timing of motor responses
How the visuomotor system times motor responses to intercept moving stimuli has been extensively studied (Van Donkelaar et al. 1992; Brenner and Smeets 1996; Lee et al. 1997; Port et al. 1997; Rushton and Wann 1999; Engel and Soechting 2000; Zago et al. 2004) and reviewed (Merchant et al. 2009; Zago et al. 2009; Brenner and Smeets 2018). These studies have shown that under different conditions of object velocity and limb movement (accelerating versus constant velocity objects, vertical versus horizontal object movement, arm movement versus button-press), different models selectively explain the timing of initiation of motor responses. The most prominent of these are the Tau model and the distance model (Port et al. 1997).
However, interception movements are usually understood to be an initiated limb movements toward the impending location of a massless target moving along a (un) predictable path on a computer screen or tablet device (reviewed in Brenner and Smeets 2018). Catching is a different skillset as the limb stays relatively immobile, the object has momentum (and mass), and the mechanics of the interaction between the limb and the projectile are of paramount importance. Thus, it is not surprising that Lacquaniti’s results on motor response timing invariance (Zago and Lacquaniti 2005) in a catching task are in stark contrast to the results of interception studies that have used a reaction-time paradigm (Van Donkelaar et al. 1992; Brenner and Smeets 1996; Engel and Soechting 2000; de Lussanet et al. 2004). Other interception studies using non-reaction time paradigms have also shown that based on object speed, participants use different strategies that involve initiating a manual response when either the time-to-collision or distance between the projectile and the body crosses a certain threshold (Lee et al. 1997; Port et al. 1997; Rushton and Wann 1999; Zago et al. 2004). One of the most prominent hypotheses generated from the interception studies is that during tracking movements, an efference copies of oculomotor commands associated with the smooth pursuit eye movement that track projectiles serve as feedforward input to guide limb movements (reviewed in Ilg 2008; de Brouwer et al. 2021). But movement kinematics and response times are only one aspect of the interception problem, real-world catching movements also require posture stabilization to absorb the kinetic energy of the projectile. Thus, producing stable motor output in real-world manual catching tasks is predicated on both the amplitude and timing of feedforward activation of muscles as well as modulation of feedback responses. This requires an experimental paradigm where the effects of manipulating projectile kinematics are tested on both limb kinematics and kinetics. Our experimental paradigm provides a tight controlled environment where we can systematically address this question.
The Tau model uses a simple calculation of the time-to-collision, the ratio of the distance between the object and the body/hand and the object speed; it also assumes that the object speed is constant. The Tau model is a special case of this calculation because it assumes that the object is headed towards the head. In that case, the ratio of distance and speed can be re-written as the rate at which the object’s image expands on the retina. This implies that the information required to initiate motor responses for approaching projectiles are contained within the stimulus itself. In other words, the motor system does not rely on sensory signals emanating from ocular or cephalic tracking of objects to initiate limb movement. However, when objects are not moving towards the head, the Tau model does not apply (Tresilian 1999) and sensory signals associated with object tracking would likely play a role.
We found no changes in TTCHand between the Mass sub-conditions suggesting that when target momentum was increased without changing its speed, the timing of the feedforward response did not change. Thus, only object speed influenced timing of motor responses. Previously, Port and colleagues showed that for objects moving at slow speeds, humans use a time-to-collision threshold to initiate limb movements, whereas for comparatively faster moving objects, they use a distance-to-collision threshold (Port et al. 1997). Our results did not support either of these strategies. If participants were to initiate movements at the same time-to-collision threshold, we would expect the TTCHand and TTCBBody to be proportionately longer for SHigh compared to SLow. This was not the case. The distance was longer but not enough and that resulted in TTCHand values that were ∼30-35 ms shorter for SHigh compared to SLow. These results were also supported by our EMG data (Figure 8). This suggests that participants did not use a time-to-collision threshold to initiate limb movements in our experimental task. So, why did participants wait 30-35 ms longer to initiate a motor response in SHigh blocks?
One possibility is that eye movements may have played a role. Though we did not record eye movements in this study, participants likely tracked the moving objects with smooth pursuit eye movements (SPEM) with high gains (Mrotek and Soechting 2007). The nervous system may have used efferent copies of the oculomotor commands associated with SPEM as inputs to set gains of feedforward and feedback motor responses (de Brouwer et al. 2021). It has been hypothesized that signal-dependent noise contaminates SPEM and causes larger fluctuations in the recorded gaze signals at higher SPEM velocities (Medina and Lisberger 2007; Joshua and Lisberger 2014). So, if efferent copies of oculomotor commands are used by the limb motor system to set gains of the motor responses, then averaging noisier oculomotor signals over longer periods of time during SHigh conditions may provide a more reliable estimate of true object motion and allow the nervous system to modulate limb movements better. Indeed, the performance of the participants improved slightly during SHigh compared to SLow (see Fig. 4A), but this association could be purely coincidental and needs further investigation.
Amplitude of motor responses exhibited momentum-dependent scaling
FPeak scaled with momentum in both the Speed and Mass conditions; participants increased the hand force right before the collision for objects with higher momentum, regardless of whether the momentum increased due to object speed or mass. This is consistent with the results of a previous study (Lacquaniti and Maioli 1989b). In contrast, the rate at which the participants increased hand force prior to the collision, dF/dt, only scaled with object speed. We correlated dF/dt and TTCHand and TTCBody (see Fig. 10A). We found strong and significant correlations between dF/dt and TTCBody for all sub-conditions, but not TTCHand. This suggests a coupling between the time at which the participants increased hand force and the rate at which participants increased the hand force prior to collision. When participants delayed the time of muscle force increase prior to collision (SHigh), they also increased the rate at which hand force changed.
The results on FPeak were not completely supported by the EMG data. The amplitude of muscle activities as well as indices of co-contraction at the shoulder and elbow joints, prior to the collision scaled with object speed, but not with object mass. These results are not consistent with the scaling of EMG with both object speed and mass that Lacquaniti and Maioli showed in a previous study (Lacquaniti and Maioli 1989b). It is unclear if the EMG scaling with mass seen in their study was because of gravitational acceleration or if the lack of scaling seen in our study is because of the constant object velocity. This needs to be further explored. However, taken together, these studies do suggest that the motor system scales the descending neural drive to the muscles based on a forward model of mechanical interaction during the collision.
Feedback motor responses exhibited only speed-dependent scaling. The amplitude of triceps activation and co-contraction of triceps and biceps was larger for SHigh compared to SLow within the short-latency and long-latency windows. There were no differences for any other muscles. There were also no differences between the two Mass sub-conditions. However, because the participants knew when the object would collide with the hand, these feedback responses were likely gated differently by the CNS compared to unpredictable perturbations that are typically applied at unexpected times prior to the collisions (Lacquaniti et al. 1993b). The differences between how feedback responses are modulated prior to and during the collision will be investigated in future studies.
Are efferent copies of oculomotor commands used for planning and execution of limb movements in an eye-centered coordinate frame?
The question of the reference frame in which reaching movements to static objects are planned has been addressed in many previous studies (Graziano 2001; Cohen and Andersen 2002; Kakei et al. 2003; Batista et al. 2007; Bernier and Grafton 2010; Beurze et al. 2010). The current understanding is that different neuronal populations along the parietofrontal pathway operate in different reference frames facilitating sensorimotor transformations in eye-centered coordinates for computing difference vector between objects and the hand, and in joint-centered reference frames for specifying the motor commands associated with making the movements. Whether similar neural mechanisms are also involved in planning and execution of virtual catching movements, is unclear.
In this study, we measured both TTCBody and TTCHand, as two different measures of time-to-collision. The latter implies that the nervous system computes the time-to-collision as a ratio of the difference vector between the object and hand locations and the object velocity, and the former that the time-to-collision was calculated as the ratio of the object distance along the y-axis and its velocity. Our analyses revealed that TTCBody, but not TTCHand, was strongly correlated with dF/dt for all Speed and Mass sub-conditions. As discussed earlier, if smooth pursuit eye movements (SPEM) tracked moving objects, then efferent signals associated with SPEM likely provided the nervous system with an estimate of time-to-collision in an eye-centered reference frame and with respect to the body, that was then used by the motor system to control the hand force rate prior to the collision.
Though this hypothesis is attractive, it does not explain many obvious facts. In our task, the object collides with the hand, and not the body. Therefore, there should be some mechanism with which the nervous system computes the time-to-collision between the object and the hand by estimating their relative distances and velocity. If so, which sensorimotor variables would be sensitive to those online transformations? Would those transformations affect both feedforward and feedback motor responses? This study was not designed to address these questions, but these questions merit further examination.
Neural correlates of motion-processing for limb motor control
These results suggest that the relative speed between the body and the object may serve as an input that might be continuously transformed by the limb motor system to adjust muscle activation and feedback gains (Zago and Lacquaniti 2005; Ilg 2008). But then which neural substrate(s) would support these visuomotor transformations? The most likely candidate is the MT/MST complex in humans. The middle temporal area (MT) in the posterior bank of the superior temporal sulcus receives direct input from motion sensitive cells in the primary visual cortex (Maunsell and Van Essen 1983; Movshon and Newsome 1996; Bisley and Pasternak 2000) and then relays that information to the medial superior temporal area (MST). The ventrolateral part of MST (MSTl) contains neurons with small receptive fields that are important for the execution of smooth pursuit eye movements and manual tracking movements (Newsome et al. 1988; Smith et al. 2006; Ilg and Schumann 2007). Both areas MT/MST share reciprocal connections with the frontal eye fields (FEF) that control eye movements (Fukushima 2003; Stanton et al. 2005) and are involved in direct visual sensory to oculomotor transformations (Gottlieb et al. 1993; Lawrence and Snyder 2009). Interestingly, FEF also shares strong reciprocal connections with the ventral part of the premotor cortex (PM) (Hutchison et al. 2012). Thus, it is likely that the MT/MST to FEF/PM pathway may facilitate modulation of the descending drive and feedback gains during interaction with moving objects.
Though this has not been probed directly, indirect support for this hypothesis came from an interesting study performed by Selen and colleagues (Selen et al. 2012). Participants viewed a random dot kinematogram and were asked to indicate the motion direction by moving a handle. During decision-making, perturbations applied to the arm at random times evoked short and long-latency reflexes (feedback responses). The gains of the long-latency reflexes were modulated by the strength and duration of the motion. The random dot task activates cells in the dorsal MST (MSTd) that have large receptive fields (Duffy and Wurtz 1991; Geesaman and Andersen 1996); these cells represent motion patterns elicited by multiple moving objects in optic-flow like conditions and integrate motion information over short time periods.
These results mirrored Lacquaniti’s studies where they showed that the amplitudes of muscle activation and feedback gains increased as the ball approached the hand. In contrast to the random-dot paradigm, catching movements involve tracking a moving object with smooth pursuit eye movements and a more ventrolateral region of MST, MSTl (reviewed in Ilg 2008; Spering and Gegenfurtner 2008). Together, these studies suggest that the MST complex may facilitate transformation of motion signals to modulate the descending drive and feedback gains in anticipation of collision between the body and a moving object. Long-latency reflexes involve spinal circuits and transcortical loops through the premotor and the primary motor cortices (reviewed in Forgaard et al. 2021). In conclusion, these studies suggests that a pathway from MST to PM may subserve transformations from visual motion to limb impedance modulation.
In summary, our study shows that object speed influences both the timing and amplitude of motor responses. We found that faster moving targets delayed the onset of the EMG and hand force response closer to the point of collision. We also found that object momentum scaled the amplitude of the feedforward force response regardless of whether the momentum was increased due to speed or mass. We also found that co-contraction of distal muscles increased only when the object momentum increased because of an increase in target speed. Our results suggest that limb motor responses may be modulated by slow eye movements.
Author contributions
AGG and TS conceived and designed research; AGG performed experiments; AGG and TS analyzed data; AGG, IK and TS interpreted results of experiments; AGG and TS prepared figures; TS drafted manuscript; AGG, IK and TS edited and revised manuscript; AGG, IK and TS approved the final version of manuscript.
Footnotes
Funding: AGG received partial support from the Universidad de Costa Rica. A portion of this work was supported by a grant from the University of Georgia Research Foundation, Inc. to TS.
References
