The perceived position of a moving object is reset by temporal, not spatial limits

When the internal texture of a Gabor patch drifts orthogonally to its physical path, its perceived motion deviates dramatically from its physical path. The local position shifts accumulate to such an extent that a 45° oblique physical path appears to be vertical. However, at some point, a limit is reached and the path resets back to its veridical location, whereupon a new accumulation starts, making the new perceived path segment appear parallel to the pre-reset segment, but offset horizontally from it. Here, we tested whether spontaneous resets of this motion-induced position shift depend on the time or the distance over which position errors accrue, or both. We introduced a temporal gap in the middle of the path that forced the illusory path to reset back to its veridical physical position. This gap-triggered reset allowed us to measure the magnitude of the illusory offset up to that point. We found that perceived offset was less than expected for the angle of illusory drift, indicating that spontaneous resets had occurred prior to the gap-induced reset. The position offset decreased when the pre-gap duration increased but approximately doubled when the path length doubled. This pattern of perceived offsets is best accounted for by spontaneous resets that occur randomly over time at a constant rate, independently of the distance traveled. Our results suggest a temporal, not spatial, limit for the accumulation of position errors that underlies this illusion.


Introduction 50
A moving Gabor patch with orthogonally drifting internal texture is perceived to deviate 51 dramatically from its physical path. Compared with other well-known motion-induced position 52 shift effects, this 'double-drift' stimulus (Lisi & Cavanagh, 2015; also called 'the curveball 53 illusion', Kwon Lisi and Cavanagh (2015) demonstrated that the illusion is not just a 62 change in perceived direction but affects perceived position as well. In particular, they showed that 63 a temporal gap of 250ms in the middle of the path resets the position offset such that the post-gap 64 segment appears to start from its new physical location rather than from its previous perceived 65 location (Figure 2). They argued that if this illusion only involved a change of direction without 66 Figure 1. Double-drift stimulus. A Gabor patch that is moving obliquely (physical path) can be perceived to be moving vertically (perceived path) if its internal texture drifts orthogonally to the physical path (up and to the right).
affecting perceived position, the stimulus would appear at the same location after the temporal gap 67 (Lisi & Cavanagh, 2015). Kwon, Tadin, and Knill (2015) presented a computational model 68 according to which position and motion information are estimated based on past sensory signals 69 when the precision of current position information is low. They found that the illusion got stronger 70 the further in the periphery it was placed, and that the perceived speed of the Gabor's internal 71 motion slowed, suggesting that its energy was being captured to create the illusory motion 72 direction. Indeed, the illusion is strongest on an equiluminant background in the periphery where 73 position uncertainty is highest (Cavanagh & Tse, 2019). In this case, the target's motion starts to 74 contribute to its position estimate, generating displacements in the direction of the target's motion, 75 predicting where it should be next. However, the target's motion direction is taken to be a 76 combination of both the within-target texture motion and its translational motion, sending the 77 estimates of the new position off in the illusory, combined direction. When the same Gabor is 78 placed on a dark background, position uncertainty is low. In this case, internal motion does not 79 contribute to new position estimates and there is no illusion (Cavanagh & Tse, 2019). 80 Figure 2. Schematic demonstration of a position reset for a double-drift stimulus caused by a temporal gap. A Gabor patch moves downward 45˚ to the right with its internal motion orthogonal to its motion direction such that the direction of its motion path appears to be vertical. A blank temporal gap is introduced in the middle of the path and the Gabor reappears at the same physical location as before the gap and continues its movement in the same direction. Rather than starting from its last perceived pregap location, the Gabor appears to be shifted to the new physical location and starts to accumulate position errors from that location, creating a perceived offset between the two motion segments.
In this study, we further explored the nature of the motion-position integration process that 81 underlies the illusion. Kwon et al. (2015) and Shapiro et al. (2010) reported that the illusory shift 82 saturates into a curved trajectory as the path length increases. In contrast, we have frequently 83 observed something quite different -the path continues linearly first and then at some point, a 84 limit is reached. When that happens, the path resets back toward its physical location and a new 85 linear accumulation starts. These spontaneous resets appear to be similar to the forced reset created 86 by a temporal gap used by Lisi and Cavanagh (2015). In both cases, a new linear, illusory motion 87 segment begins parallel to the first, starting from the physical location (Figure 3). The 88 consequence of these resets is that the whole path appears to saturate into a curved trajectory with 89 averaging over trials, so that the existence of the individual resets from a linear path is masked. In 90 a separate study, Nakayama and Holcombe (2020) asked participants to trace or draw the motion 91 paths they saw for the double drift stimuli and the presence of the resets was clear in this case, 92 triggered by distracting flashes on the screen (Figure 4). 93 Figure 3. Schematic demonstration of a spontaneous position reset for a double-drift stimulus. A Gabor patch moves downward 45˚ to the right with its internal motion orthogonal to its motion direction such that the direction of its motion path appears to be vertical. A blank temporal gap (350ms) is introduced during the movement and the Gabor reappears at the same physical location as before the gap and continues its movement in the same direction. If there is a spontaneous position reset before the temporal gap (green arrow), the path resets back toward its physical location and a new accumulation starts, making a new perceived vertical segment parallel to the first but offset horizontally. The perceived position may either reset fully (A) or partially (B) back to the physical location. If the accumulation has been linear and free of loss before the temporal gap, the expected offset size (blue arrow) can be calculated from the angle of the illusory deviation. If the perceived offset (yellow arrow) is less than this expected offset size, it indicates that some position reset or saturation has occurred before the temporal gap.
The source of spontaneous position resets has not yet been examined. Here we investigated 94 whether the upper limit for the accumulation of the illusory position shift depends on the time 95 or/and the distance traveled by the double-drift stimulus. Specifically, if a reset is triggered once 96 the illusory offset exceeds a maximum distance from the Gabor's real position, regardless of the 97 time of accumulation, it suggests that there is a critical spatial error, or a 'red line', for the illusory 98 offset. Conversely, if the reset is triggered solely on the basis of the time since the illusory offset 99 began, it suggests a temporal limit on the accumulation of the position displacements. If this 100 temporal limit is in the range of seconds, it suggests that the accumulation process must logically 101 involve higher-order brain regions, as cells in early processing stages do not have integration time 102 windows on the scale of seconds; these are found instead in frontal areas that support visual 103 working memory maintenance (Funahashi, Bruce, & Goldman-Rakic, 1989;1990). Indeed, a 104 recent fMRI study using multivariate pattern analysis found that the perceived path of the double-105 drift stimulus did not share activity patterns with a matched physical path in early visual areas 106 traditionally associated with visual processing. This shared representation was only found in 107 anterior brain regions such as the frontal cortex (Liu, Yu, Tse, & Cavanagh, 2019). Lastly, both 108 distance and time may contribute to determine the moment of the reset. For example, the longer 109 the perceived position is beyond the "red line" for error, the more likely the reset. There are a 110 number of ways that space or time or both could influence the occurrence of resets. Here we will 111 try to evaluate only a few simple possibilities. 112 How can we measure these resets? Since a temporal gap in the path can force a reset, as 113 shown in Lisi and Cavanagh (2015), the perceived offset between pre-and post-gap motion path 114 should reveal the magnitude of the accumulation up to the reset. The present study thus measured 115 how much position error has accumulated up to a certain point by forcing a reset with a temporal 116 gap at that point. Specifically, we presented a double-drift stimulus over different time durations 117 and path lengths before the temporal gap, followed by a similar continued motion after the gap. 118 We compared the perceived offsets between the two path segments with the expected size of offset 119 that would be seen if no spontaneous reset(s) had occurred before the gap. Critically, to allow for 120 a prediction of what the perceived offset would be without resets, we asked each participant to first 121 adjust the speed of the internal motion so that the perceived path angle (ignoring any position resets 122 that they saw) was always, locally, 45°. If the accumulation is linear and free of loss during the 123 first segment, the offset of the perceived location from the physical location must be 70.7% (or 124 ) of the physical path length for a 45° angle of the illusory deviation. Anything less than 70.7% 125 (for this 45˚ illusion size) indicates that some spontaneous reset or saturation must have occurred 126 before the temporal gap (Figure 3). Overall, we found that the perceived offset was smaller than 127 expected and decreased for longer durations with the same path length, but approximately doubled 128 when the path length was doubled. The observed data were best explained by partial resets that 129 happen randomly over time with a fixed probability, regardless of the illusory offset from the 130 physical path. 131

Methods 132
Participants 133 Twenty students from Dartmouth College participated (Ten in the main experiment: 7 134 females; age range: 18-38, mean age = 23.4 ± 6.8; Ten in the control experiment: 8 females; age 135 range: 18-20, mean age = 18.9 ± 0.7) and all reported normal or corrected-to-normal vision. All 136 volunteered to participate and were naïve to the purpose of the study. Participants signed an the left of the screen throughout the experiment. The stimulus was a Gabor patch (a sinusoidal 148 grating within a Gaussian envelope) with a spatial frequency of 1 cycle/dva and 100% Michelson 149 contrast (Peli, 1990). The standard deviation of the Gaussian envelope was 0.15 dva. The Gabor 150 patch moved along a linear path at 45˚ either leftward or rightward from vertical. The midpoint of 151 the trajectory was placed at 5 dva to the right of the screen center so that the stimulus was 10 dva 152 away from fixation. The sinusoidal grating had the same orientation as the motion path. Stimuli 153 were presented on a uniform grey background with luminance of 53 cd/m 2 , equal to the mean 154 luminance of the Gabor. There were two parts of the experiment. In the first part, the Gabor patch 155 traversed back and forth along a linear path (external motion). The internal texture of the Gabor 156 also drifted in the orthogonal and downward direction of the external motion path (internal motion) 157 to produce a perceived vertical motion path. The internal motion reversed its direction each time 158 the external motion reversed. The speed of the internal motion was first presented at a random 159 frequency that ranged from 0.5 to 5.5 Hz on each trial and was adjusted, as described below, to 160 make the local angle of the Gabor path appear vertical for each path length, duration and direction 161 of internal motion. In the second part of the experiment, the same stimulus was used; however, it 162 was only presented for half a cycle on each trial, and always started moving downward from the 163 top position. The internal texture either drifted ('double-drift stimulus') or stayed static ('control 164 stimulus'). When the Gabor reached the midpoint of the path, there was a brief temporal gap of 165 350ms with the stimulus removed from the screen. After the blank gap, the stimulus would 166 reappear at the same height but at a shifted horizontal position (-3.5, -2.1, -0.7, 0.7, 2.1, 3.5 dva) 167 and continued its movement in the same direction for the same amount of time and path length as 168 before the gap. Three durations (1s, 2s, 3s) and two path lengths (2 dva, 4 dva) before the gap were 169 used in both parts of the experiment (Figure 5).

Procedure 171
Participants were instructed to fixate at the fixation point throughout all experiments. Each 172 participant completed two separate parts of the experiment. In the first part, a Gabor patch was 173 shown in the periphery and moved back and forth along a linear path with its internal texture 174 drifting orthogonally relative to the external motion. In each trial, participants adjusted the speed 175 of the internal drift by pressing the up arrow key to increase the speed and the down arrow key to 176 decrease the speed until the physical trajectory appeared locally vertical along its path, ignoring 177 any sidesteps from resets. The purpose of this task was to find the internal motion speed that 178 produced a perception of locally vertical motion for physical paths either 45˚ leftward or rightward 179 from vertical at each of the different external motion speeds used in the subsequent task. This 180 adjustment produces a constant perceived local path angle in all conditions. This allows us to 181 predict the expected amount of shift throughout so that the differences in perceived offset between 182 the two motion segments are not simply due to differences in the perceived path angle. Each 183 participant completed 15 adjustment trials for each external motion path orientation, stimulus 184 duration, and path length for a total of 180 trials divided in 2 blocks. The average of the internal 185 Figure 5. Stimulus demonstration and experiment conditions. A Gabor patch moved along a linear path downward at 45˚ to the right or left from vertical. Its internal texture also drifted in the orthogonal and downward direction of the physical path to produce a vertical perceived motion path. A blank temporal gap (350ms) was introduced during the movement and the Gabor reappeared at the same physical location as before the gap and continued its movement in the same direction for the same amount of time and path length as before the gap. The physical motion path had two possible orientations (leftward or rightward from vertical). The path before and after the gap had three possible durations (1s, 2s, 3s) and two possible path lengths (2 dva and 4 dva). If a spontaneous position reset occurs during the blank gap, the post-gap motion segment should appear parallel to the pre-gap segment but offset horizontally by less than the full amount expected. motion speed for each stimulus condition was calculated individually and was then used for that 186 participant in the following task for that same condition. See Figure 6 for group averaged internal 187 motion speeds for each condition as well as the expected angle of the illusory path derived from 188 the vector sum of the external motion speed and the adjusted internal motion speed, a model that 189 was found to produce the perceived motion direction when position uncertainty is high (Cavanagh 190 and Tse, 2019). In the second part of the experiment, in each trial, after a 400 ms fixation period, 191 the same Gabor patch as in the first task was shown in the periphery traversing its path once with 192 a temporal gap of 350 ms presented in the middle of the path during which nothing was presented.  We explored models that had spatial or temporal limits that would trigger a reset. The resets 220 could either reduce the offset from the physical location to 0 ('full resets') or reduce the offset by 221 a portion of the distance to last reset ('partial resets'). In all cases, we assume that rate of 222 accumulation of the illusory offset is linear (see Figure 3) and determined by the external speed 223 of the Gabor (path length, P, divided by pre-gap duration, T) and the local angle, α, of the illusory 224 path (which is titrated by changing the internal speed to be always 45°), such that the magnitude 225 of illusory offset, S, at time point t since last reset is given by: 226 (2) 227 We compared the perceived offset values for each stimulus condition with predictions from 228 two types of models: 1) a fixed time limit model or fixed space limit model where a reset happens 229 after a fixed time interval or a fixed distance that the illusory path has drifted away from the 230 physical path; 2) a random time limit or random space limit where a reset happens randomly with 231 a probability per second or per dva that the stimulus has drifted away from its physical path. to the control condition (Figure 7 and 8 right panel), indicating that the post-gap segment was 258 more likely to be perceived as shifted leftward back to its physical position after the temporal gap. 259  We first examined models that assumed that spontaneous resets occurring prior to the 296 temporal gap returned the perceived path 100% of the way to the physical location. For the fixed 297 space and time model, with no random factor, the predicted offset just oscillates back and forth 298 between 0 and the value set by the spatial or temporal limit (Figure 10A and B). These models 299 with a fixed limit are not very realistic as the resets occur at the same point on every trial. Although 300 observers often report seeing spontaneous resets (see 't Hart, Henriques, & Cavanagh, 2019), they 301 seem more variable. Nevertheless, the model predictions as shown in Figure 10A and B do 302 demonstrate what an individual trace might look like with resets. Thus, the predicted PSE from a 303 model with fixed time limit t would be a value between 0 and the expected offset after t seconds 304 without a reset (as given in Equation 2; Figure 10A). Similarly, the prediction with a fixed spatial 305 limit is just a value between 0 and the spatial limit s for both path lengths and all three durations 306 ( Figure 10B). Note that in these fixed models, the expected values of the perceived offset at each 307 time point do not change across simulation runs. As shown in Figure 10B, a fixed space limit 308 model with a spatial limit at 1.62 dva from the physical location is not a good match to the observed 309 data as there is no difference in illusory offsets between the three path durations and the predicted  The random models allow resets to occur probabilistically at some rate of resets per time 320 interval or per spatial interval. As shown in Figure 10C and D, the average predictions of offset 321 for these models increase smoothly due to the averaging over 10,000 runs of simulation with resets 322 occurred randomly over time; each individual repetition had a sawtooth pattern of drift values like 323 those in Figure 10A and B. Specifically, for the random time limit model with a fixed probability 324 of reset over time, the rate that yielded the lowest RMSE was 0.8/s (on average one reset per 1.25s). 325 1: 1 2 :

Results were highly similar across all observers and the observed bias in
1 3 As Figure 10C shows, similar to the fixed time limit model, predicted PSEs from this model 326 decreased as path duration increased with fixed path length and doubled as the path length doubled. 327 This pattern of results is expected from a model with random resets that are purely determined by 328 the time the drift is away from the physical path since the number of resets will increase with time 329 regardless of the path length (Table 1). With a fixed duration giving the same number of resets, 330 doubling the path will double the PSE since the rate of drift increases directly with the external 331 speed. However, unlike the fixed time limit model, the predicted PSE was smaller for 1s and the 332 decrease in PSEs flattened over time for longer durations (Figure 10C; ratio of 1s to 2s to 3s of 333 predicted PSEs is 1:0.7:0.5). The predicted PSEs from this model with a rate of one full reset per 334 1.25s matched the observed data better than the fixed time limit model (ΔAICc = 14.37). 335 For the random space model ( Figure 10D) with a constant probability over distance away 336 from the physical path, the probability (p) that yielded the lowest RMSE was 0.6/dva (on average 337 one reset per 1.7 dva). First, the predicted PSEs remained the same for the three path durations 338 since the number of resets will stay the same as long as the drift distance is the same regardless of 339 the path duration (Table 1). Also, unlike the case in the random time model, the predicted PSEs 340 were on average less than doubled when the path length doubled (ratio of 4-dva to 2-dva of 341 predicted PSEs is 1.4). This is because the number of resets doubles as the path length doubles, 342 preventing the PSEs from doubling (Table 1). Thus, the predicted PSEs from this model at 0.6/dva 343 did not fit the observed data as well as the random time model (ΔAICc = 2.4). 344 Lastly, we examined the same types of fixed and random limit models with partial resets 345 where the perceived path returned only part way to the last reset (Figure 11 and Table 1). The 346 pattern of results was similar to those with full resets. The best fit model had resets occurring 347 randomly at a rate of 1.8/s (on average one reset per 0.56s) and each reset returned 68% of offset 348 to last reset. Predictions from this model matched the data better than the fixed and random space 349 models when they were allowed to have partial resets (ΔAICc > 7). Predictions from the fixed time 350 model with partial resets (65% partial resets every 0.67s) also matched the data as well as the 351 random time model with partial resets (ΔAICc = 0.19). As expected, the average number of resets 352 of the best fit model with partial resets is higher than that with full resets to compensate for the 353 smaller resets (Table 1). Thus, the model that fits the observed PSEs best was the one where reset was partial (68%) 355 and was set by random events at a rate of one every 0.56s (1.8 per second) Table 1. Mean number of resets for each path duration and length across 10,000 runs of simulation of the best fit models with full and partial resets and model performance.  increases (linearly) and as a result, for a fixed path length, the perceived offset must decrease with 369 duration. This matches the effect of duration in the data with path length fixed (Figure 9). In 370 contrast, a purely spatial limit would predict the same illusory offset for a fixed path length 371 independently of the duration of the path as the resets would be triggered purely by distance (since 372 the illusory direction was titrated to be 45° for all stimulus conditions, the expected drift is set only 373 by the path length and is therefore of path length). Second, with a purely temporal limit, the 374 number of resets would stay the same for a given duration regardless of the path length. Doubling 375 the path length would then double the illusory offset as the illusory drift accumulates at twice the 376 rate when the path is twice as long (again, the expected drift is of two times of the path length), 377 as observed in the data. A pure spatial limit model would predict more resets with increasing path 378 length and that would keep the drift value from doubling. Thus, our results are best explained by 379 a pure time limit where partial resets that return the perceived path 68% of offset to last reset that 380 happens randomly, with a fixed rate across time on average once every 0.56s, independently of the 381 magnitude of the Gabor's perceived shift away from its physical location. 382 Although the fixed time model with partial resets also fit the data relatively well, models smooth trajectory that is curved across time. We also tested models that combined both space and 389 time limits but already with only 6 data points, our current models stretch the ability of the data to 390 constrain the models. With even more free parameters, the marginal gain with combined space and 391 time limits is very small and they did not perform noticeably better. 392 Our results are best explained by resets occurring randomly over time with underlying 393 linear illusory paths. These models predict curved paths when the position offsets are averaged 394 over trials (Figure 10 and 11) and are quite similar to the curved paths described by Shapiro et al. 395 (2010) and Kwon et al. (2015), although they predict that the perceived path is curved on each 396 transit whereas we assume linear paths interspersed with resets. The data in our experiments here 397 do not differentiate between these two alternatives. The demonstration movie from Lisi and 398 Cavanagh (2015) shows a gap-triggered reset of linear path to almost all observers. Hand tracing 399 data from 't Hart, Henriques, and Cavanagh (2019) and Nakayama and Holcombe (2020) supports 400 the existence of spontaneous resets along with an otherwise a linear path (see Figure 4). However, 401 the resets, when they do occur, are not very salient and observers are often left with a feeling that 402 the path shifted but without knowing when that happened, so more studies will be required to reach 403 a clear conclusion. Cavanagh (2015) further showed that the deviation is not only in direction but also in the perceived 408 position of the stimulus, so that a brief temporal gap in the middle of the path would reset the 409 illusory position offset back to its physical location. They proposed that the large and sustained 410 perceived position displacement for the double-drift stimulus reflects the accumulation of local 411 position shifts driven by the motion of its internal texture, shifting the perceived path far from the 412 physical path (Lisi & Cavanagh, 2015). Note that in their study, the perceived offset between the 413 pre-and post-gap segments was not significantly different from the expected offset (1.41 dva) for 414 a 2dva motion path with 45˚ illusory angle (Lisi & Cavanagh, 2015), unlike our results in which 415 we see a decrease in perceived offset for this condition (Figure 9). We assume that this difference 416 was due to a difference in the speed of internal motion used in the two studies. In our study, we 417 asked the participants to adjust the speed of the internal motion until the motion path appeared to 418 be, locally, 45˚ away from its physical path for each external speed condition, ignoring any shifts due to spontaneous resets. These titrated internal motion speeds were all shown to be around the 420 expected illusory angle (45˚) derived from a vector sum model of the adjusted internal motion 421 speed and the external motion speed ( Figure 6B). Importantly, the internal motion speed that 422 yielded a 45˚ illusory angle for a 2 dva/s motion path, which was then used in the subsequent reset 423 test, was around 2 Hz in our study. However, Lisi and Cavanagh (2015) used a 3 Hz internal 424 motion, which should yield a 56˚ illusory angle and an expected offset of 1.66 dva if there were 425 no resets before the gap based on a vector sum model. Thus, if we assume that the perceived motion 426 direction of the double-drift stimulus was produced by a vector sum of the external and internal 427 motion speed when uncertainty is high as shown in Cavanagh and Tse (2019), it is possible that 428 Lisi and Cavanagh (2015) also got less than the expected offset at the moment of the temporal gap. 429 The double-drift stimulus has a much longer and more sustained motion-position 430 integration process than that of other well-known motion-induced position shift effects. For 431 example, the "flash-grab" illusion shows integration period of only about 80 to 100 ms (Cavanagh 432 these working-memory-related frontal regions that act as 'visual buffers' where the position errors 450 are accumulated and maintained over longer durations. happen spontaneously, and if so, whether they depend on the time or/and the distance traveled by 512 the double-drift stimulus. By introducing a temporal gap, we measured the size of the 513 accumulated illusory offset up to the gap for different path lengths and durations. We found that 514 the perceived offset at the gap was smaller than the expected offset size if no reset had occurred 515 before the gap, suggesting that spontaneous resets had occurred before the gap. In addition, the 516 offset size decreased for longer durations at the same path length and approximately doubled 517 when the path length doubled. The observed data are best explained by resets that are determined 518 purely by the time since the perceived path drifted away from the physical path, regardless of the 519 distance traveled, with the partial resets occurring randomly at a constant rate over time (on 520 average once every half second). 521

Acknowledgments 523
This material is based upon work funded in part by National Science Foundation Award (1632738)