Abstract
The coordination between continuously-controlled and discretely-initiated orienting movements is a fundamental problem in motor control. This coordination is vital for maintaining high-acuity vision of moving objects, which requires foveating the object’s image through continuous smooth pursuit and discretely triggered saccadic eye movements. However, the decision mechanism underlying this coordination remains inadequately characterized. During these coordinated eye movements, pursuit is rapidly initiated by retinal image velocity while intervening saccades are triggered with highly variable latency contingent on both retinal image velocity and position. Here we propose a new computational strategy simulating the decision mechanism triggering saccades during pursuit. In this model, future position error and its associated uncertainty are predicted through Bayesian inference across noisy, delayed sensory observations of retinal image motion. This probabilistic prediction of position error enables the computation of the confidence that a saccade will be needed, triggering saccades upon accumulating confidence to a threshold. Saccade confidence corresponds to the relative certainty that the image is positioned outside the fovea (quantified through log probability ratio). This model predicts the occurrence and latency distributions of saccades across a range of step-ramp target trajectories and externally imposed sensory uncertainty levels (e.g. via Gaussian blurring of the visual target). We suggest this stochastic, predictive decision making process represents a fundamental principle for the neural coordination of movement.
Introduction
The coordination of continuous and discrete reorienting is a fundamental problem in neuroscience. This is further complicated by the inevitability of noise and delays in sensory observations (Faisal et al., 2008; van Beers et al., 2002). The oculomotor system has promising potential for investigating this motor coordination problem due to the relative simplicity in modelling the biomechanics of eye motion and the wealth of existing data concerning the underlying neurophysiology driving eye movements (Van Gompel et al., 2007). When viewing objects in motion, the oculomotor system coordinates two types of eye movements to reorient and actively stabilize a visual image onto the fovea (the region of the retina with the highest visual acuity). The first type of eye movement, saccades, are discretely triggered to rapidly reposition a visual image onto the fovea (Goffart, 2009). The second type, smooth pursuit, is continuously deployed to minimize the motion of a visual image on the retina (Keller and Heinen, 1991; Krauzlis, 2004). The oculomotor system judiciously coordinates if and when saccades are needed during pursuit, triggering spatially accurate saccades despite sensorimotor noise and delays (Orban de Xivry and Lefèvre, 2007; Quinet and Goffart, 2015). This decision process is also stochastic (Carpenter and Reddi, 2001; Ratcliff, 2001; Wolpert and Landy, 2012), resulting in trial-by-trial variability in the timing of saccades during pursuit (de Brouwer et al., 2002a). However, a quantitative explanation of the mechanism underlying this predictive, stochastic decision process remains poorly characterized. Studying oculomotor coordination of saccades during pursuit has the potential to provide general insights into decision making in motor coordination.
Previous studies have investigated how visual sensory signals correlate with saccade trigger during pursuit initiation and maintenance, finding that both visual position and velocity errors are important in this motor decision (Bieg et al., 2015; de Brouwer et al., 2002b, 2002a; Rashbass, 1961). To investigate the sensory basis of saccadic decision making during pursuit, experimentalists commonly employ a paradigm involving an abrupt change in target position and velocity (named step-ramp trajectories after their appearance on a position-time plot). It has been well established that the probability of saccade occurrence is minimized during pursuit initiation if the change in position and velocity are in opposite directions and balanced in magnitude such that the target recrosses the initial fixation position in 200 ms (Rashbass, 1961). Target trajectories that recross the fixation position after a slightly longer or shorter duration tend to evoke an intermediate probability of saccade occurrence with highly variable latency (Bieg et al., 2015). Similarly during pursuit maintenance, it was observed that the ratio between position and velocity error correlates well with the occurrence of saccades and roughly correlates with saccade latency (de Brouwer et al., 2002b). This parameter was originally coined as eye-crossing time, but henceforth referred to as time-to-foveation to parallel the time-to-collision parameter in studies of steering control (Regan and Gray, 2000) and the time-to-contact parameter in studies of limb control and interception (Chang and Jazayeri, 2018; Tresilian, 1995). While time-to-foveation is a useful behavioural correlate, it remains unsuitable as a decision variable due to its instability as velocity errors approach zero (time-to-foveation goes to infinity) and its inability to address how sensory noise is accommodated. Furthermore, the concept of time-to-foveation is based on the linear extrapolation and intersection of the eye and target trajectory which do not necessarily intersect in two-dimensional tracking. Thus, while existing datasets have catalogued saccadic decision making during pursuit, an underlying decision mechanism has not yet been proposed.
Probabilistic inference through sequential sampling has provided a successful framework for decision making under uncertainty, particularly in studies of perceptual decision making (Gold and Shadlen, 2001; Hanks and Summerfield, 2017) but less often applied to oculomotor control. In this framework, noisy evidence is sequentially sampled and linearly integrated until a fixed threshold is reached, terminating the decision process and triggering a response. Choices, response times, and neuronal responses in sensorimotor decision areas are consistent with this framework (Gold and Shadlen, 2007; Schall, 2001, 2000). Recently, an innovative stochastic model of pursuit was developed where noisy, delayed sensory signals are probabilistically estimated through Kalman filtering (Orban de Xivry et al., 2013). Their model allows for the closed-loop simulation of probabilistic visual motion estimation and pursuit dynamics, which opens the possibility to model the probabilistic sensory basis of stochastic saccadic decision making during pursuit.
To model the saccade trigger mechanism, we propose a decision process that accumulates evidence that a saccade is needed (saccade confidence) to a threshold. We define saccade confidence as the log probability ratio of the probability of the target being left vs right of the foveal center. This definition has roots in the sequential probability ratio test (Forstmann et al., 2016; Wald and Wolfowitz, 1948) and agrees with the proposition that the term confidence should reflect the probability that a decision/response is correct/appropriate given the observed evidence (Pouget et al., 2016). Modelling the decision variable triggering saccades with this definition of confidence allows small, low-uncertainty position errors to trigger saccades when sustained, while large, high-uncertainty position errors may fail to trigger saccades when transient. The model can simulate saccade latency distributions evoked by a range of step-ramp target trajectories matching human behavioural data (Bieg et al., 2015; de Brouwer et al., 2002b) and illustrates how predictive mechanisms can flexibly coordinate continuous and discrete reorienting movements. We suggest this stochastic, predictive decision mechanism represents a fundamental principle in the neural control of motor coordination.
Methods
Global Model Overview
The purpose of this model is to explain how catch-up saccades are triggered during smooth pursuit. We extend an existing Bayesian model of sensory motion estimation and smooth pursuit dynamics (Orban de Xivry et al., 2013) to develop a model of the decision mechanism triggering saccades. The model consists of three interconnected modules: a sensory pathway employing Kalman filtering for state estimation, a decision pathway employing bounded accumulation of saccade confidence and a motor pathway implementing the dynamics of pursuit and (simplified) saccades (Figure 1). An important addition to the sensory pathway of the model was the inclusion of linear motion extrapolation (Figure 1; Sensory Extrapolation) to update retinal state estimates, compensating for sensorimotor delays. We assume that the goal of a saccade is to reorient a visual image to a preferred retinal location, presumably the fovea (which defines the zero-position in our retinotopic coordinate system). The saccade decision mechanism relies on the probabilistic, predictive position error estimate to compute the log-probability ratio that the image is outside the fovea (which we define as saccade confidence). A saccade is triggered when saccade confidence is temporally accumulated to a threshold value. We use this model to simulate the occurrence and latency distributions of saccades during pursuit initiation and steady-state pursuit evoked by step-ramp target trajectories.
Sensory Pathway
The sensory pathway is an extension of the visual motion processing pathway previously described (Orban de Xivry et al., 2013). We denote matrices in bolded uppercase, vectors in bolded lowercase, and scalars in regular lowercase. The true retinal image state, δdet, is defined as the difference between the position, velocity, and acceleration of the target, t, and the eye, e: where PE is Position Error, RS is Retinal Slip, RA is Retinal Acceleration and k represents the current discrete time step in the numerical simulation.
The observed retinal state is delayed by 70 ms (Krauzlis and Lisberger, 1994; Tavassoli and Ringach, 2009) and corrupted by additive and signal-dependent noise: where Φmult and φadd are uncorrelated noise covariance matrices with values drawn at each time step, k, from Φmult∼N(0, σ2) and φadd∼N(0, σ2). These terms represent the signal-dependent and baseline levels of noise corrupting visual sensory information. Both behavioural analyses and electrophysiological recordings in the superior colliculus (encoding position error) and the middle temporal area (encoding retinal slip and acceleration) suggest that the information in visual signals driving saccades and pursuit are consistent with signal-dependent noise constraints (saccades: Sparks, 1986; Harris and Wolpert, 1998, 2006; van Beers, 2007; Goossens and van Opstal, 2012; pursuit: Maunsell and Van Essen, 1983; Lisberger and Westbrook, 1985; Nover et al., 2005; Osborne et al., 2005, 2007; Stein et al., 2005; Osborne and Lisberger, 2009).
Using the delayed, noisy observations of image motion, a Bayes-optimal probabilistic estimate of image motion, , is computed through Kalman filtering (Kalman and Bucy, 1961). This method of recursive Bayesian estimation combines noisy observations with priors of the noise characteristics and dynamics of the world (ie: a generative model) to estimate retinal state. The generative model contains information about the expected evolution of retinal motion, as well as the noise characteristics that map the underlying state into a noisy observation. In this generative model, the components of retinal state are assumed to be uncorrelated and thus it is expected that the evolution of retinal state can be described by a random walk: where λk is an uncorrelated, additive process noise with λk∼N(0, Q2) representing the state variability, a prior belief about how retinal state changes over time. The generative model also contains knowledge that the observed image motion at time k, , represents a noisy version of the retinal state, (Equation 5). where γk and vk are the expected, uncorrelated signal-dependent and additive noises covariance matrices with γk∼N(0, D2) and vk∼N(0, R2). These terms represent the brain’s estimate of the signal-dependent and baseline levels of noise corrupting sensory observations. For each component of δ, we set and . Thus, we set the internal estimates of sensory noise parameters (D2, R2) to accurately reflect the actual sensory noise , though an exact knowledge of these noise parameters is not crucial (Beck et al., 2012; Orban de Xivry et al., 2013).
The operation of the Kalman filter combines the current estimate of retinal state, , the current noisy observation, , and prior knowledge of the noise characteristics,(Q, R, D, Ω), to optimally estimate image motion at the next time step, (Izawa et al., 2008; Todorov, 2005) (Equations 6-8). where is the estimated error covariance, I is the identity matrix, and ηk is the internal noise of estimation with ηk∼N(0, Ω2), which represents noise in the computations estimating the retinal state. The values of Kalman filter parameters are listed in Table 1.
In the final stage of the sensory pathway, extrapolation, the components of estimated image motion driving the pursuit and saccade motor pathways are updated based on their expected rate of change and knowledge of the delays inherent to each respective sensorimotor loop. This allows for compensation of sensorimotor delays. It has been shown that catch-up saccades can accurately foveate moving targets at saccade offset, effectively predicting changes in position error due to retinal slip accruing during visuomotor delays and saccade duration (de Brouwer et al., 2001; Guan et al., 2005; Price et al., 2005; Quinet and Goffart, 2015). Similarly it has been shown that both pursuit behaviour and motion-tuned neuronal activity in the middle temporal area of macaques account for target acceleration (Bennett et al., 2007; Krauzlis and Lisberger, 1994; Lisberger and Westbrook, 1985). It is not clear if target acceleration is also accounted in saccadic motor system (Bennett et al., 2007), and although it has been shown that position error can weakly drive smooth pursuit (Blohm et al., 2005), neither of these interactions were modelled in this study. Thus, in our model, position error was extrapolated based on retinal slip , and retinal slip was extrapolated based on retinal acceleration to drive the saccadic and pursuit pathways respectively: where Tsacc = 130 ms and Tpurs = 70 ms represent the time duration of extrapolation for each respective signal. The rationale for the longer extrapolation duration for saccades is to account for the additional decision accumulation time, motor delay, and movement duration unique to the saccadic system.
Saccade Decision Pathway
The saccade decision pathway is inspired by the sequential probability ratio test and stochastic bounded accumulation models of decision making (Gold and Shadlen, 2007; Wald and Wolfowitz, 1948). This pathway computes saccade confidence and triggers saccades upon accumulating to a threshold. The first stage of this pathway reflects the probability of the image being right vs left of the foveal center using the predictive probabilistic estimate of position error: where x represents a variable of spatial integration. Recall that the estimated position error value used in this computation is a predictive estimate accounting for retinal slip.
The next stage is the computation and leaky temporal accumulation of saccade confidence (Ĉ), defined as the logarithm of the ratio between the above probabilities: where τS is the leakiness time constant, with τS = 25 ms and ωk is an uncorrelated noise term in the decision process with ωk∼N(0,0.12). This estimate of saccade confidence acts as a decision variable that upon threshold crossing triggers a saccade (with an additional motor delay of 40 ms). For example, a rightward saccade is executed after with a 40 ms motor delay, where θsaccade = 4.2. While outside the focus of this investigation, a simple model of saccade dynamics (described below) and saccade refractory period (200 ms) were implemented to illustrate how this saccade decision model can be implemented within a global framework for oculomotor control.
Pursuit Motor Pathway
The pursuit motor pathway transforms the predicted retinal slip, , into an eye velocity command that is sent to the premotor system and eye plant. It is adapted from the image velocity motion pathway and positive efferent copy feedback loop used by previous models (Krauzlis and Lisberger, 1994; Krauzlis and Miles, 1996; Orban de Xivry et al., 2013). This pathway transforms predicted retinal slip into a command for eye motion, E. The pathway (Figure 2, top left) contains a linear function (G) to modify the RS input into an eye acceleration command, a second-order filter (H to adjust the time course of inputs, and a variable gain element (A). The parameters were similar to Krauzlis and Lisberger, 1994; Orban de Xivry et al., 2013. where s represents the Laplace variable, and , with . The purpose of this noise term is to simulate the trial-by-trial variability in pursuit gain observed in human behavioural responses and thus capture a realistic level of pursuit variability.
The output of this image velocity motion pathway was sent to a leaky integrator with positive feedback. The leaky integrator is characterized by a single time constant, τp = 100 ms, that maintains eye velocity when . The positive feedback loop contains a linear gain element, .
Saccade Motor Pathway
After saccade confidence reaches decision threshold, the saccade motor pathway (Figure 2, Saccade Motor Pathway) transforms a desired saccade amplitude, , into an eye velocity command that is sent to the premotor system and eye plant. It should be noted that the purpose of this saccade model was not to exactly reproduce all aspects saccade dynamics, but rather to embed a model of saccade execution within our global framework. A motor delay of 40 ms was implemented between the time of saccade confidence threshold crossing and the execution of the saccade. The model of saccade dynamics is adapted from the local feedback model and bilateral burst neuron discharge rate pulse generator (Figure 2, PG) used by (Blohm et al., 2006; Jürgens et al., 1981; Scudder, 1988). In this pathway, the desired saccade amplitude is compared against an internal estimate of executed eye movement through negative feedback from a resettable integrator (Figure 2, RI), providing an estimate of motor error. This motor error is sent to a pulse generator, providing a saccadic motor command sent to the premotor system and eye plant. The pulse generator was based on the bilateral burst neuron discharge rate proposed by (Van Gisbergen et al., 1981): where the input, x, is motor error and the output, y, is a saccadic velocity command approximating the main sequence relationship between saccade amplitude, peak velocity, and duration (Bahill et al., 1975). The parameters used match Blohm et al., 2006 (ie: e0 = 1 deg; bm = 600 deg/s; and bk = 3 deg).
Final Common Motor Pathway
Eye velocity commands from the pursuit and saccade pathways are linearly added then sent through the premotor system to the eye plant. The premotor system consists of the sum of the motor command (with gain = T1 = 170 ms) and its integral; producing the pulse-step innervation pattern required to displace the eye and maintain eccentricity (Robinson, 1973). The eye plant is modelled as an overdamped, second-order system with time constants T1 = 170 ms and T2 = 13 ms. No additional motor noise was added to the eye plant, as the moment-by-moment variability in sensory observations and trial-by-trial variability in pursuit gain that we implemented provides a sufficiently realistic distribution of motor variability (Hu et al., 2007; Osborne, 2011; Osborne et al., 2005).
Simulations
The model was numerically simulated in MATLAB R2017a (Mathworks, Natick, MA, USA) using 1 ms discrete time steps. We simulated eye movements in response to step-ramp target motions selected to reproduce behavioural investigations of pursuit initiation (Bieg et al., 2015) and pursuit maintenance (de Brouwer et al., 2002b). For pursuit initiation, we simulated target steps between 1 to 12 degrees in one-degree increments, target velocities of ±10 deg/s and ±20 deg/s, and 100 repetitions at each step-ramp condition. For pursuit maintenance, the initial step-ramp was selected to recross the initial fixation position in 200 ms, thus minimizing initial saccade occurrence. The step sizes used were 2, 4, and 6 degrees with velocity changes of 10, 20, and 30 deg/s respectively in the opposite direction. The second step-ramp was selected using steps ranging from −20 to 20 degrees in increments of 2 degrees and velocity changes from −50 to 50 deg/s in increments of 10 deg/s, with 50 repetitions per double step-ramp condition. Simulations of increased sensory noise were implemented using the same double step-ramp conditions while modifying the variance of the additive and multiplicative noise in position estimation to 22 deg2 and 1.52 respectively. We analyzed the latency of the first occurring saccade after each step-ramp. Latency was determined using a 500 deg/s2 acceleration criterion on the simulated eye trace.
Results
The model of oculomotor coordination presented here relies on Bayesian estimation of stochastic sensory signals and bounded accumulation of saccade confidence to trigger saccades during pursuit for foveal stabilization of a moving visual target. Since the framework driving pursuit movements employed here has been previously described (Orban de Xivry et al., 2013), we will focus our description on the saccade decision and trigger pathway. The notable novelty in this model is the computation of saccade confidence from probabilistic estimates of predicted retinal state. We define saccade confidence as the log-probability ratio that the image position is outside the fovea. In this model, saccades are triggered when temporally accumulated saccade confidence reaches a constant threshold.
Computation of Saccade Confidence
The computation of saccade confidence transforms a probabilistic prediction of position error into the log-probability ratio of the target being right vs left of the foveal center, which is then temporally leaky integrated. This process is schematically illustrated in Figure 3, where a constant estimate of predicted PE is used to compute saccade confidence without decision noise. Figure 3A illustrates the probability density function for an estimated PE of 2 degrees with low uncertainty (blue) and high uncertainty (red). The probability of the target being right (left) of the fovea is the area under the probability distribution to the right (left) of zero (denoted in black dashed line). Saccade confidence is calculated by leaky integrating the ratio between these probabilities. Figure 3C illustrates the temporal evolution of saccade confidence based on the corresponding estimates of predicted PE. With low magnitude PE estimates, the saccade decision process is highly sensitive to the estimated uncertainty. Figures 3B illustrates an estimated position error of 5 degrees with low uncertainty (blue) and high uncertainty (red). Figure 3D demonstrates that at large magnitude PE estimates, the temporal evolution of saccade confidence is similar despite the difference in uncertainty. Thus, the estimation of saccade confidence accounts for both predicted PE and its uncertainty, but this computation is more sensitive to uncertainty for targets closer to the fovea.
Decision Process during Pursuit Initiation
To understand how sensory image motion drives saccadic decision making, Figure 4 illustrates examples of single trial simulations of oculomotor responses to two different trajectories of step-ramp target motion known to evoke and minimize saccade occurrence during pursuit initiation (red and blue, respectively) compared to a step trajectory (grey) (Figure 4A). These step-ramp target trajectories contain an abrupt, simultaneous position displacement and constant velocity shift that can be independently controlled by the experimenter. This paradigm is commonly used to investigate saccadic decision making during pursuit because it allows the experimenter to precisely control the retinal motion subsequent to the step-ramp onset (Bieg et al., 2015; de Brouwer et al., 2002a, 2002b; Rashbass, 1961).
In our model, noisy and delayed sensory observations are estimated through Kalman filtering (Figure 4B, 4C). To compensate for sensory delays, predictive retinal image state estimates are computed by extrapolating sensory estimates according to their first derivative and estimated delay (e.g. PE is updated by RS, figure 4D). This predictive PE estimate is then used to compute saccade confidence, which triggers saccades upon temporal leaky accumulation to a threshold (Figure 4E). Due to Kalman filtering, noisier signals (e.g. RS compared to PE) require more time for accurate estimation, which is comparable to behavioural data suggesting an asynchronous influence of RS compared to PE in saccade programming (Schreiber et al., 2006). Thus, the observation that saccade latency is longer than pursuit latency is explained in the model by an additional temporal accumulator in the saccade decision pathway, allowing time for accurate RS information to accrue and influence the saccadic decision. Thus, estimated RS can either enhance or attenuate the evolution of saccade confidence by updating the current PE estimate away from or towards the fovea, leading to a reduction in saccade latency or a suppression of saccade execution respectively.
We simulated oculomotor responses across a range of step-ramp trajectories then calculated the proportion of saccadic trials and mean saccade latency. This model captures the major trends of behavioural data, predicting a minimization of saccade occurrence for targets whose trajectories recross the initial fixation position in about 200ms (Rashbass, 1961), as well as saccades with long latency for target trajectories slightly outside this recrossing time (compare with Figure 2 from Bieg et al., 2015). However, the behavioural data from Bieg et al., 2015 shows small differences in saccade latency for target motion away from the fovea vs towards the fovea at large step sizes that was not captured in this model. Overall, the model reproduces the major trends in saccade occurrence and latency variability responses during visually guided pursuit initiation.
Decision Process during Steady State Pursuit
The model was used to simulate steady state pursuit using target trajectories with an initial step-ramp designed to minimize saccades followed by a second step-ramp to test saccadic decision making. An example of single trial simulations is illustrated in Figure 6, which contrasts smooth and saccadic pursuit in response to two different double step-ramp target trajectories. Consistent with saccadic decisions during pursuit initiation, RS estimates leading to an updating of PE towards foveal positions minimizes saccade confidence and subsequent saccade occurrence.
Across a range of double step-ramp target trajectories, we found that the sensory conditions preceding saccade trigger in simulations were highly comparable to human behavioural data. Figure 7 plots the values of RS and PE preceding each saccade trigger in simulations (left) and behavioural data (right). The region of this plot with very few saccade occurrences is described by de Brouwer and colleagues (2002b) as the smooth zone. The slope of the lines delimiting the smooth zone correspond to time-to-foveation values, demonstrating that the model emergently simulates similar correlations between time-to-foveation and saccade occurrence as human behaviour. Furthermore, the model simulated saccades of unusually long latency occurring when time-to-foveation was slightly outside this range (Figure 8). Thus, the model also explains saccadic decisions following a step-ramp perturbation during maintained pursuit.
New Model Predictions
A novel prediction from modelling results suggests that the absolute value of velocity change also correlates with saccade occurrence and latency beyond the relationship described by the time-to-foveation parameter. As the absolute value of the velocity change increases, the range of time-to-foveation values resulting in smooth trials tends to decrease (Figure 8). Since pursuit acceleration saturates at large values, larger velocity changes require longer acceleration durations, allowing more position error to accrue and thus increases the probability of saccade occurrence. Therefore, increasing the change in target velocity for a given time-to-foveation value will promote saccade occurrence.
A second novel prediction from this model is that increasing position uncertainty will impact saccade decision making during pursuit. Increasing position noise can be achieved experimentally through Gaussian blurring of the pursuit target and implemented in the model through an increase of the noise corrupting sensory observations. Specifically, we predict a decrease in saccade occurrence and an increase in the variability of saccade latency in response to step-ramp target trajectories with time-to-foveation near the smooth zone, but a minimal impact on saccade latency distributions for negative time-to-foveation trajectories (corresponding to a position step and velocity change in the same direction) (Figure 9, top row). This is because the smooth zone corresponds to retinal motion where RS updates PE closer to the fovea and the saccade confidence is highly sensitive to sensory uncertainty for foveal image position estimates (Figure 3). Thus, increasing sensory uncertainty should lead to a larger range of tolerable position error estimates that fail to trigger saccades.
Discussion
We have developed a model of the saccade decision mechanism during visually guided pursuit in which saccade confidence (defined as the log-probability ratio of position error to the right vs left of the fovea) is computed from predictive estimates of position error and temporally accumulated to trigger saccades upon threshold crossing. This decision mechanism reproduces the Rashbass paradigm, where saccades occurrence during pursuit initiation is minimized if the target recrosses the initial fixation position in 200 ms (Bieg et al., 2015; Rashbass, 1961). The model also reproduces the behavioural observation that saccade occurrence is minimized when the time-to-foveation following a step-ramp during steady-state pursuit is between 40 and 180 ms and qualitatively reproduces the distribution of long-latency saccades evoked by step-ramps in and around this time-to-foveation range (de Brouwer et al., 2002b). Additionally, the model makes the novel prediction that for the same time-to-foveation value, saccades occur more frequently and with shorter latency with increasing target velocity. The model also predicts that increasing position uncertainty will increase the range of tolerable position errors that fail to trigger saccades. Thus, the model provides a generalizable explanation for saccadic decisions during pursuit while fitting harmoniously within existing oculomotor models.
Model Limitations
The scope of this model is limited by only considering visual sensory inputs and omitting the influence of cognitive factors. For example, it has been shown that spatial attention is preferentially allocated ahead of tracking during pursuit (Chen et al., 2017; Khan et al., 2010). This could decrease the precision of sensory observations in unattended locations (Poletti et al., 2017), leading to a slower rise in saccade confidence and longer saccade latencies. This effect may underlie the differences in latency between forward and reverse saccades at large step sizes observed by Bieg et al., 2015. The allocation of attention also has more direct influences on saccade trigger (Kowler et al., 1995; Zhao et al., 2012), likely through influencing normative decision thresholds (Lo and Wang, 2006). Thus, cognitive factors likely modulate model parameters describing sensory noise and decision thresholds through mechanisms beyond the scope of this study, but the model accounts for the basic principles underlying these visually guided saccadic decisions.
Another important factor in human behavioural responses that was omitted in the model was mechanisms of learning and adaptation. This could also play a role in the differences between simulated results and Bieg et al., 2015. Since their paradigm only tested a single target speed for each individual it is conceivable that subjects improved their ability to appropriately accelerate their eyes at pursuit initiation (Dash and Thier, 2013) or more rapidly estimated RS by shifting their priors towards that target speed (Darlington et al., 2017) resulting in a different retinal image state evolution compared to simulations. In contrast, de Brouwer et al. (2002b) randomized the trial-by-trial target motion using a wide range of possible position steps and velocity shifts. This experimental paradigm was believed to minimize the influence of learning and anticipation and emphasize the visually driven aspects of saccadic decision making, providing a more ideal dataset to evaluate the model against, which qualitatively matches their behavioural data. Future studies may adapt/extend a similar memory pathway as developed by Orban de Xivry et al. (2013) to properly investigate saccadic decision making during memory-guided pursuit behaviours.
The model also omits the weak influence of PE on pursuit performance. For a given target velocity, initial pursuit acceleration also depends on its initial position error (Lisberger and Westbrook, 1985). Similarly, position error also influences pursuit deceleration and termination at motion offset (Pola and Wyatt, 2001). More recently, it has been shown that position error during steady-state pursuit can have a direct influence on the pursuit trajectory (Blohm et al., 2005). However, these influences only slightly contribute to the variability in pursuit response, whereas the sources of variability accounted in this model, sensory noise and trial-by-trial variability in pursuit gain, seem to have a more dominating influence (Lisberger and Medina, 2015; Osborne et al., 2005). Thus, we believe our simulations accurately reflect the magnitude of visually guided pursuit variability and the omission of a position error input to the pursuit system is minor.
Finally, our model makes no attempt to characterize the evolution of sensory and decision signals during saccade execution. It has been shown that visual acuity is reduced around the time of a saccade, a phenomenon known as saccade suppression (Krekelberg, 2010). The behavioural effect correlates with a suppression of neural activity in visual cortical areas (Bremmer et al., 2009). Saccade suppression has been successfully modelled as a consequence of optimal sensorimotor estimation in the face of signal-dependent noise and sensorimotor delay (Crevecoeur and Kording, 2017). However, we consider this outside the scope of our study since we emphasize investigating the decision mechanism leading to a saccade, not the subsequent sensory consequences of saccade execution.
Generalizability of Model
A major feature of this decision mechanism is the generalizability of saccade confidence as a decision variable compared to the previously hypothesized time-to-foveation parameter (de
Brouwer et al., 2002b). Recall that time-to-foveation is defined as - , which grows to infinity in magnitude as RS approaches zero. Thus, although the parameter correlates with saccade behaviour after an abrupt change in target motion such as in step-ramp trajectories, it is unsuitable as an actual decision variable implemented in the brain. In contrast, the saccade decision mechanism proposed here can explain saccade trigger to both moving and stationary targets.
Another feature of this model is the plausible generalizations that can be made to describe two-dimensional visual tracking compared to the time-to-foveation parameter. The concept behind time-to-foveation is based on the linear extrapolation and intersection of the eye and target trajectories in time, which does not necessarily have an intersection point in two-dimensional tracking. In our framework, a similar saccade confidence estimator that computes the log-probability ratio that the target is above vs below the fovea could be employed to account for vertical and oblique saccade trigger. While conceptually simple, this extension is suboptimal since it implies a loss of covariance information when considering the uncertainty in horizontal and vertical position error independently. A more sophisticated extension speculates that the probability ratio computations underlying the estimation of saccade confidence are implemented through distance dependent inhibition in topographic representations of position error (which would account covariance), similar to proposed dynamic neural field models of saccade generation (Quinton and Goffart, 2018; Trappenberg et al., 2001). Thus, while our current model is constrained to horizontal tracking, there are plausible methods of generalizing the model to account for two-dimensional tracking that merit further investigation.
Neurophysiological Implementation
Our model normatively outlines the computational steps underlying saccadic decisions, which has implications on the signals and connectivity of its neural mechanism. Firstly, the model suggests that motion and position information must converge in a predictive position error estimate which informs the saccade decision process. In macaques, this motion information is likely supplied by middle temporal area (MT) since electrical stimulation shifts saccade amplitudes according to the preferred motion direction of the stimulation site and influences saccade latency in a manner dependent on the trajectory of target motion (Groh et al., 1997). Position information is likely supplied by the superior colliculus (SC), as the spatial locus of activity in the SC correlates with position error during pursuit (Krauzlis et al., 2000) and inactivation of the rostral SC (corresponding to foveal position errors) results in stable eye position offsets during pursuit (Hafed et al., 2008). The frontal eye fields (FEF), characterized as a prediction map for oculomotor control (Crapse and Sommer, 2008), receives motion information from area MT (Ungerleider and Desimone, 1986) and position information from the superior colliculus (Lynch et al., 1994). Various neuronal subpopulations in the pursuit region of FEF have been shown to carry the necessary position and velocity signals for predictive saccadic decision making (Bakst et al., 2017). The lateral intraparietal area (LIP) is also implicated in position prediction, as repetitive transcranial magnetic stimulation suppresses the shifted position percept reported in the flash grab illusion (Edwards et al., 2017). Similarly, predictive eye position signals have been decoded from the ventral intraparietal area during pursuit (Dowiasch et al., 2016), which may also inform predicted position error. Thus, the computation of predicted position error is likely implemented through a distributed network of oculomotor regions.
Secondly, our model suggests neural correlates that integrate predicted PE over time that can trigger saccades upon threshold crossing. Neuronal activity in LIP during perceptual decision making tasks is consistent with temporal accumulation of evidence supporting a saccade to its movement field (Huk and Shadlen, 2005). In a motion discrimination task, LIP activity predicted monkey’s subsequent choice, and the rate of rise in activity correlates with the relative strength of the motion evidence (Shadlen and Newsome, 2001). Additionally, the time that LIP neurons reach a threshold firing rate predicts decision time (Roitman and Shadlen, 2002). Furthermore, during decision tasks based on probabilistic expectations of reward, LIP activity was consistent with temporal accumulation of expected reward (Platt and Glimcher, 1999). Similarly, neuronal activity in the saccade region of FEF is predictive of choice and latency consistent with the decision models of evidence accumulation to a fixed threshold (Hanes and Schall, 1996). Furthermore, the saccade and pursuit subregions of FEF have overlapping projections to the caudate nucleus (Cui et al., 2003) and models suggest this corticostriatal pathway terminating in the superior colliculus is capable of reading out threshold crossing to trigger saccades (Lo and Wang, 2006). Although further investigation targeting saccade decision signals during pursuit are needed, similar oculomotor networks may be responsible for both prediction and confidence accumulation in this normative decision mechanism.
Conclusion
Our model of saccade decision making during pursuit elegantly handles the constraints of delay and signal-dependent noise in the oculomotor system while reproducing known trends in saccade occurrence and latency across a range of target motion trajectories. The model illustrates how continuously controlled orienting movements like pursuit can be programmed directly from sensory estimates, while discretely triggered reorienting movements like saccades employ a second stage of evidence accumulation to determine if/when they should be executed. We suggest that this framework of prediction and confidence estimation represents a fundamental principle in stochastic decision making and sensorimotor coordination.
Acknowledgements
This work was supported by the National Sciences and Engineering Research Council (NSERC, Canada) and the Canada Foundation for Innovation (CFI). JDC would like to thank Brandon Caie, Alex Göttker and Julia Morris for valuable discussions and feedback.