Abstract
When performing a long chain of actions in rapid sequence, future movements need to be planned concurrently with ongoing action. However, how far ahead we plan, and whether this ability improves with practice, is currently unknown. Here we designed an experiment in which healthy volunteers produced sequences of 14 finger presses quickly and accurately on a keyboard in response to numerical stimuli. On every trial, participants were only shown a fixed number of stimuli ahead of the current keypress. The size of this viewing window varied between 1 (next digit revealed with the pressing of the current key) and 14 (full view of the sequence). Participants practiced the task for five days and their performance was continuously assessed on random sequences. Our results indicate that participants used the available visual information to plan multiple actions into the future, but that the planning horizon was limited: receiving information about more than 3 movements ahead did not result in faster sequence production. Over the course of practice, we found larger performance improvements for larger viewing windows and an expansion of the planning horizon. These findings suggest that the ability to plan future responses during ongoing movement constitutes an important aspect of skillful movement. Based on the results, we propose a framework to investigate the neuronal processes underlying simultaneous planning and execution.
Significance Statement Although skill learning has typically focused on the training of specific movement sequences, practice improves performance even for random sequences. Here we hypothesize that a fundamental aspect of skilled sequential behavior is the ability to plan multiple actions into the future, both before and during execution. By controlling the amount of visual information available for motor planning, we show that people plan at least three movements beyond current action and that this planning horizon expands with practice. Our results suggest that coordinating ongoing movement and planning of future actions is central to a wide range of cognitive and motor tasks, providing a new perspective on the neural implementation of motor planning in the context of sequential behavior.
Introduction
Humans exhibit a wide range of behaviors, from whole-body activities like running or riding a bike, to fine dexterous skills like writing or typing on a keyboard. Many of such skills share one common feature: they are comprised of a series of separate motor elements that are strung together in quick succession to form longer and more complex sequences of movements (Lashley, 1951). When learning a new sequential skill, people usually need many hours of practice to achieve fluidity in performance (Ericsson et al., 1993). With practice, sequence production becomes quicker, more accurate, and less effortful (Verwey, 1994; Rhodes et al., 2004; Diedrichsen and Kornysheva, 2015; Krakauer et al., 2019), leading in the long run to the skillful behaviors typically observed in elite athletes (Yarrow et al., 2009).
Previous studies of motor sequence learning have largely focused on the training of specific movement sequences (Cohen et al., 1990; Willingham, 1999; Verwey, 2001; Verwey and Abrahamse, 2012; Kornysheva et al., 2013, 2019; Verwey et al., 2014; Wong et al., 2015b; Mantziara et al., 2020). However, many sequences we execute in everyday life are not fully predictable and practice improves performance even for random or untrained sequential movements (Waters-Metenier et al., 2014; Wiestler et al., 2014; Ariani et al., 2020). Some of these sequence-general improvements arise because participants learn to translate individual visual stimuli into motor responses and to execute these responses more quickly (Ariani and Diedrichsen, 2019; Hardwick et al., 2019). Such improvements in single responses benefit the production of all sequences, including random ones.
In the present study, we focus on a second core ability that benefits the production of unpredictable sequences: the ability to plan future movements ahead of time. Planning of movements before their initiation, here referred to as preplanning, has been studied extensively (Rosenbaum, 1980; Rosenbaum et al., 1987, 2007; Churchland et al., 2010; Cisek and Kalaska, 2010; Kaufman et al., 2014; Wong et al., 2015a; Haith et al., 2016). However, long or complex movement sequences are unlikely to be fully preplanned, so planning of the remaining elements must continue throughout sequence production – a process that we have recently named online planning (Ariani and Diedrichsen, 2019). Take the example of a basketball player dribbling up the court. The player needs to control a continuous flow of movements (e.g., to keep the dribble alive) while scouting the court and planning future movements depending on the actions of both teammates and opposing defenders. Some evidence for online planning has been observed for a range of behaviors, such as reading (Rayner, 1998, 2014; Rayner and Reingold, 2015), walking (Matthis and Fajen, 2014), sequential reaching (Säfström et al., 2013, 2014) and path tracking (Bashford et al., 2018). However, to what extent the motor system plans upcoming movements during sequence production (i.e., the horizon of online planning) remains poorly understood.
Here we asked 1) how far the benefit of planning ahead extends beyond current execution, and 2) whether this planning horizon can be improved with practice. To answer these questions, we used a discrete sequence production (DSP) task, in which participants performed random sequences of 14 keypresses with their right hand in response to numerical cues. We manipulated how many digits participants could see ahead of the current keypress. Viewing window size ranged from 1 (only the next movement is cued, as in the serial reaction time task, SRTT) to 14 (the entire sequence shown at once, as in the DSP task). Participants practiced producing varying sequences over 5 days. This design allowed us to examine the horizon of both pre- and online planning in sequence production, as well as the influence of practice on the planning horizon.
Methods
Participants
Seventeen right-handed neurologically healthy volunteers (8 women, 9 men; age 18–36 years, mean 25.81 years, SD 5.09 years) were recruited for this study. Handedness was assessed with the Edinburgh Handedness Inventory (mean 82.81, SD 18.07). Individuals participated in 5 sessions of practice (2 hours each, on 5 separate days). All participants provided written informed consent and were naive to the purposes of the study. Two participants abandoned the study after the first session of practice. One participant had an unusually high error rate (> 30%, while every other participant managed to keep the error rate < 20%, as per instructions). These 3 participants were excluded from successive analyses (final N = 14). For one of the remaining 14 participants, age and handedness data was missing. All experimental methods were approved by the Research Ethics Board at Western University.
Apparatus
Participants placed their right hand on a custom-made keyboard (Fig. 1A), with a force transducer (Honeywell FS series) mounted underneath each key. The keys were immobile and measured isometric finger force production. The dynamic range of the force transducers was 0-16 N and the resolution 0.02 N. A finger press/release was detected when the force crossed a threshold of 1 N. The forces measured from the keyboard were low pass filtered, amplified, and sent to PC for online task control and data recording. Numerical stimuli were shown in white against a black background, horizontally aligned in a single line, and spanned ~36° of visual angle for an entire sequence.
Procedure
In each of the five practice sessions, participants sat in front of a computer screen with their right hand on the keyboard. The task required participants to produce sequences of keypresses in response to numerical cues appearing on the screen (numbers 1 to 5, corresponding to fingers of their right hand, thumb to little finger, respectively) as quickly and accurately as possible (Fig. 1A). On every trial, only a fixed number of digits ahead of the current press position (viewing window size, w) were revealed to the participants, while the rest were masked with asterisks (Fig. 1B). The masked digits were revealed to the participant as they proceeded, from left to right, with the presses in each sequence. The window size varied within the domain of w = {1, 2, 3, 4, 5, 6, 7, 8, 14}, and was randomized across trials within every block. As an attentional pre-cue, during the inter-trial interval (ITI, 1.5 seconds) participants were presented with a fixation cross on the location of the first digit in the sequence. With every press, subjects received feedback about the correctness of their action: the white numbers turned either green or red and were accompanied by a corresponding sound for correct and incorrect presses, respectively.
To motivate participants to improve in the task, they were rewarded with points based on their performance after each trial. Points were awarded on the basis of sequence movement time (MT) and execution accuracy. MT corresponded to the time interval between making the first press in the sequence to releasing the last press in the sequence. Accuracy was calculated as 1 – error rate (proportion of error trials in a block) in percentage. Specifically, a trial was considered an error if it contained one or more incorrect presses, for which participants received 0 points. Correct sequences were rewarded with at least 1 point. Finally, participants were awarded 3 points if 1) a sequence was correct and 2) MT 5% or more faster than a specific time threshold. This time threshold was designed to get increasingly difficult adjusting to every subject’s speed throughout training. It would decrease by 5% from one block to the next if two performance criteria were met: median MT in the current block faster than best median MT recorded hitherto, and mean error rate in the last block ≤ 15%. If either one of these criteria was not met, the thresholds remained unchanged. At the end of each block, participants received feedback on their error rate, median sequence MT, total points obtained during the block, and total points obtained during the session. Subjects were asked to try to maintain an error rate below 15%.
In the original design, we intended to compare also how the ability to plan ahead might affect partially familiar (structured) sequences. Therefore, each one of the 5 practice sessions consisted of 8 blocks (27 trials each) of 14-item sequences and 3 blocks (60 trials each) of specific short 3-/4-item segments that composed the structured sequences. One-third of the trials in the sequence blocks were randomly generated by random shuffles of the digits 1 to 5. The remaining two-thirds of the trials were structured sequences. As the results from the structured sequences turned out to be hard to interpret, the present paper focuses only on the completely unfamiliar, random sequences.
Data analysis
Data were analyzed with custom code written in Matlab (The MathWorks, Inc., Natick, MA). To evaluate the speed of sequence production, we inspected the time intervals between different keypresses. Reaction times (RT) were defined as the time from stimulus onset to first press (i.e., the first crossing of the 1 N force threshold). Note that participants were not instructed to react particularly fast. Instead, they could take as much time as they wanted until they felt ready to start. MTs were defined as the time between the first press and the release of the last press in the sequence (i.e., the time between the first and the last crossing of the force threshold). Finally, we calculated inter-press intervals (IPI) between subsequent pairs of presses in the sequence (i.e., the time interval between every two consecutive crossings of the force threshold). Unless otherwise noted, we used within-subject repeated measures ANOVAs and 2-sided paired samples t-tests for statistical inference in assessing the effects of viewing window or practice on RT, MT, and IPI. Error trials were excluded from data analysis. To provide meaningful error bars for within-subject comparison, the standard error for each condition was calculated on the residuals after subtracting the mean across conditions for each participant. This way, the error bars visualized the size of the relevant error term in a repeated-measures ANOVA.
To describe the relationship between MT and the viewing window size, we used the following exponential model: where MT’ is the predicted MT for a given viewing window size w. Note that for w = 1, the function reduces to the initial value of the exponential, MT’ = a + c. The asymptote is given by c and the slope by b. This model was then fit to the MT data of each participant using Matlab’s nlinfit() function, which implements the Levenberg-Marquardt nonlinear least-squares algorithm. We determined the effective planning horizon (w*), by finding the window size for which the predicted MT of the participant had dropped 99% of the difference between w = 1 and the asymptote, i.e., by solving the equation for w: where the 0.01 arises from the criterion of the 99% drop (i.e., 1% above the MT’ asymptote). The improvement in effective planning horizon with practice was then assessed by fitting the model to the data of each participant on each day and comparing w* between day 1 and day 5 with a within-subject 2-sided paired samples t-test. While the use of a 99% criterion is arbitrary, changes in this criterion only scale the effect planning horizon by a specific value but do not change the outcome of the statistical analysis.
Results
Preplanning of future movements speeds up sequence production
First, we assessed the benefit of being able to plan future finger movements on sequence production. To determine this, we varied the amount of available information and tested how this window size affected the speed of performance. On average, across all days of practice, larger window sizes produced shorter MTs (Fig. 2A), as confirmed by the highly significant main effect of window size on MT in a repeated measures ANOVA (F8,104 = 176.980, p < 10e-10). This finding suggests that the availability of visual information allows for preplanning of sequential actions into the future, which in turn reduces MT. Interestingly, this benefit appeared to plateau around a window size of 3 or 4. Indeed, when we compared the MT of each viewing window to the average MT for larger window sizes, we found a significant difference for w = 3 vs. w > 3 (t13 = 4.644, p = 4.591e-04), but not for w = 4 vs .w > 4 (t13 = 2.083, p = 0.058). To obtain an individual measure of the planning horizon, we fit an exponential model to the MT curve of each participant (Fig. 2B, see Methods).
Next, we decided on an arbitrary criterion on the exponential (99% of the MT drop to the asymptote) to establish the individual effective horizon of each participant. This analysis revealed a mean effective planning horizon of 3.58 ± 0.28 items ahead of the current item, indicating that, on average, participants were able to plan at least 3 keypresses into the future.
Practice expands the planning horizon
We then asked whether practicing sequences would affect the ability to plan future movements by comparing performance at the beginning (day 1) and at the end (day 5) of practice (Fig. 3A). We observed that MT improved across all window sizes (main effect of day: F1,13 = 18.004, p = 0.001). Significant improvements were found even for a window size of 1 (MT difference day 1 vs. day 5: 586 ± 262 ms; t13 = 2.234, p = 0.022). This condition was, in essence, a serial reaction time task, where each cue was only presented after the preceding key was pressed, meaning that participants were forced to serially cycle through the planning and execution of every press, with no possibility for planning ahead. Therefore, MT improvements for a window size of 1 must be a consequence of 1) better stimulus identification, 2) better stimulus-response (S-R) mapping, or 3) better execution (i.e., motor implementation) of single responses.
If learning was restricted to improvement in any of these three processes, we would predict equal MT improvement across all window sizes, given that stimulus identification, S-R mapping, and execution are necessary steps across all viewing windows. Contrary to prediction, we found a significant interaction between window size and stage of practice (day 1 vs. day 5; F8,104 = 3.220, p = 0.003). Furthermore, when we directly inspected the MT improvement (percentage change relative to average MT for each horizon, Fig. 3B), we found significantly larger gains for larger viewing windows (w = 2 vs. w = 1: t13 = 3.338, p = 0.005; w = 3 vs. w = 2: t13 = 2.722, p = 0.017), until again the gains plateau for w = 4 or larger (w = 4 vs. w = 3: t13 = 0.113, p = 0.912). Thus, although responses to single items improved with practice, this improvement cannot explain why performance benefits were more pronounced for larger window sizes. Instead, the additional performance benefit must arise because participants became more efficient at using the advance information provided by larger viewing windows.
In addition to the evidence for more efficient planning of future elements, we also found evidence that participants planned further into the future. When we determined the effective planning horizon for each participant and day (Fig. 2B) using an exponential fit (see Methods), we found that the planning horizon expanded from 3.20 to 3.88 digits ahead of current action between day 1 and day 5 (paired-samples t-test, t13 = 2.840, p = 0.014, Fig. 3C). Thus, participants not only became more efficient in planning but planned sequential actions further into the future.
Note that faster MTs for larger window sizes did not occur at the expense of reduced accuracy in performance. On average, the percent accuracy of presses remained roughly constant around 85-90% across all viewing window conditions. We found no significant main effect of window size (F8,104 = 1.182, p = 0.317), practice stage (F1,13 = 0.325, p = 0.578), or interaction between the two factors (F8,104 = 0.548, p = 0.818).
Taken together, these results show that participants became faster in sequence production by getting better at 1) making single responses (involving stimulus identification, S-R mapping, or execution) and 2) exploiting available information to plan more upcoming movements in advance.
Reaction times increase with the amount of preplanning
If participants invested time in preplanning the first few elements of each sequence, then we would expect this to be reflected in their reaction times: namely, participants should start a sequence earlier when presented with a smaller window size, and later for larger window sizes, since they would be preparing more of the upcoming keypresses. Even though fast RTs were not required by the task, participants likely tried to balance the benefit of getting more points with the benefit of finishing the experiment more quickly. On average across all days (Fig. 4A), larger viewing windows resulted in slower RTs. A repeated-measures ANOVA substantiated this effect (F8,104 = 4.563, p = 8.726e-05).
However, as observed for MTs, RTs appeared to plateau for window sizes larger than 3. Thus, even though participants could see more than 3 elements on the screen and had virtually unlimited time to preplan, they initiated the sequence in approximately 700-800 ms from cue onset.
Did this dependence of RTs on the amount of available information change with practice? When we compared RTs across early and late stages of practice (Fig. 4B) we found no indication that, late in practice, participants waited longer to initiate a sequence (F1,13 = 0.012, p = 0.913), or that their strategy changed over time (no interaction between practice stage and window size: F8,104 = 1.187, p = 0.314). Thus, even though over time participants did not spend more time preplanning, the MT results indicate that motor planning improved, as participants could make use of more visual information in roughly the same amount of preparation time (i.e., with comparable RTs).
Planning ahead continues during sequence production
So far, our results have indicated that participants improve their ability to perform random sequences of finger movements by becoming more efficient in using the information provided by larger window sizes. However, it remains unclear whether participants got better at planning movements before sequence production (preplanning), during sequence production (online planning), or both. To distinguish the contributions of preplanning and online planning to performance improvements, we examined the intervals between individual presses in a sequence (i.e., the IPIs). The rationale behind this analysis is that short IPIs reflect an increased readiness to press (i.e., better planning) than long IPIs. If all keypresses were equally well prepared (e.g., as in the case of w = 1, which does not allow participants to plan ahead), then all IPIs within a sequence should roughly have the same duration depending on the serial RT (null hypothesis, Fig. 5A). Alternatively, if only early presses in a sequence can be fully preplanned, while later presses can only be minimally planned, or are completely unplanned, then early IPIs should be significantly shorter than later IPIs, which will revert to serial RT speed (Fig. 5B). Finally, if online planning continues in parallel with execution, we should expect an effect of window size also on mid to late IPIs (Fig. 5C).
In light of these predictions, we first inspected the IPIs averaging across practice stages (Fig. 5D). For a window size of 1, all IPIs had approximately the same duration (~500 ms), reinforcing the idea that for W = 1, each keypress is selected, planned, and executed independently. In contrast, for window sizes larger than 1, we found a clear effect of IPI placement (i.e., finger transition number within the sequence) on IPI duration (F12,156 = 33.111, p < 10e-10). Specifically, the first and last IPIs were consistently performed much faster than the middle IPIs, regardless of the size of the viewing window (W > 1). For W = 2, the first IPI (first 2 finger presses) was faster than subsequent IPIs; for W = 3, the first two IPIs (first 3 finger presses) were faster than subsequent IPIs. For W > 3, this preplanning advantage appeared to be spread over the first 3 finger transitions. This pattern of results indicates that the initial speed up reflects the fact the visible digits can be preplanned during the reaction time, and hence are executed faster.
Consistent with RT and MT data, preplanning does not seem to improve further beyond a window size of 3. This observation reinforces the idea that participants preplanned at least the first three movements of each finger sequence. Once all preplanned keypresses are executed, planning must continue online, slowing down later IPIs. Thus, the slower IPIs in the middle of the sequence mostly reflect limits in the speed of online planning. When we restricted our analysis to these middle IPIs (transitions 5 to 12), the differences between w = 1 and w = 2 (t13 = 19.557, p = 5.037e-11), between w = 2 and w = 3 (t13 = 5.013, p = 2.374e-04), and between w = 3 and w = 4 remained significant (t13 = 2.182, p = 0.048). This indicates that, just like preplanning, online planning benefits from having visual information about up to 3 presses into the future, thus highlighting clear parallels between the two processes.
We also observed that, consistent across all window sizes greater than 1, the last IPI was executed much more quickly than preceding IPIs. Currently, we can only speculate about the reasons for this result. One idea is that participants tend to select and plan the last 2 presses as a unit. These presses can then be executed very quickly, as no more movements need to be planned after those two (which frees up planning capacity). Alternatively, participants could optimize the last two presses from an execution biomechanics perspective. Given that no subsequent movements are needed, participants do not have to maintain a specific hand posture that would be required for fast execution of successive movements. Instead, they are free to optimize their hand posture for comfort and speed only in regard to making the last two presses.
Both pre- and online planning improve with practice
Finally, we asked whether practice effects on MT are more likely related to improvements in preplanning, online planning, or both. From day 1 to day 5 (Fig. 6A), we observed significant main effects of practice stage on IPI duration on both early (IPI 1-3: F1,13 = 17.623, p = 0.001) and middle IPIs (IPI 5-12: F1,13 = 15.988, p = 0.002). To quantify the relative contributions of preplanning and online planning, we carried a separate analysis (Fig. 6B) averaging across IPIs that were more likely preplanned (IPI 1 for w = 2, IPI 1-2 for w = 3, and IPI 1-3 for w ≥ 4), or not (the remaining IPIs for each viewing window condition, which had to be planned online). For w = 1, only the first press, but not the first IPI, can be preplanned. Therefore, we cannot attribute any of the observed improvements to either sequence preplanning or online planning. Instead, eventual improvements need to arise from improved visual identification, S-R mapping, or execution. We computed the IPI difference between day 1 and day 5 for these three categories, normalized it by the average IPI duration across days (separately for each category), and plotted it against viewing window size (Fig. 6B). This analysis confirmed that IPIs got faster with practice even for w = 1 (one-sample t-test vs zero difference: t13 = 2.305, p = 0.038). Additionally, we found clear further improvements in IPI duration for w > 1: compared to w = 1, these effects were present both for the IPIs that were likely preplanned (t13 = 4.028, p = 0.001), and for those that relied on online planning (t13 = 6.009, p = 4.379e-05). There was no significant difference between preplanning and online planning in terms of learning improvements (F1,13 = 1.141, p = 0.305), nor was there an interaction between planning process and viewing window (F2,26 = 1.000, p = 0.382). Thus, preplanning and online planning appear to have similar capacity limits and to benefit similarly from practice in sequence production.
Discussion
The ability to move and simultaneously plan future movements is a fundamental yet underappreciated faculty of the human brain. By manipulating the amount of visual information available for motor planning in a discrete sequence production task, we show that participants planned multiple actions (at least 3) into the future (Fig. 2-4). Sequence practice led to larger gains in speed for larger window sizes (MT difference, Fig. 3), as well as increases in the horizon of sequence planning (exponential fit, Fig. 3). In-depth analysis of the inter-press intervals (Fig. 5-6) revealed that enhanced planning of future actions was present both before (preplanning) and during (online planning) sequence production.
Fast sequence production and the speed of online planning
Before voluntary movements can be performed, they need to be planned (Keele, 1968; Keele and Summers, 1976; Kerr, 1978; Rosenbaum, 1980; Bock and Arnold, 1992; Crammond and Kalaska, 1994, 2000; Cisek and Kalaska, 2002, 2004), at least to some degree (Cisek and Kalaska, 2010; Ames et al., 2014). However, many real-life motor skills require quick sequences of movements that are not always predictable. Proficiency in such skills depends on our ability to select and plan future movements both before and during sequence production. To investigate this ability, we used a viewing window paradigm that varied the amount of information available for planning the next movements. We replicate previous findings of anticipatory planning in the context of sequences (Rosenbaum et al., 1987; Rhodes et al., 2004; Herbort and Butz, 2009), and longer reaction times when more information for planning is available (Henry and Rogers, 1960). Critically, once such preplanning reaches capacity, the execution of later elements in the sequence slows down, which we interpret as evidence that successive movements need to be planned online. By varying the available time for preplanning, in a previous paper (Ariani and Diedrichsen, 2019) we showed that this was the case even for relatively short (e.g., 5-item) and well-known (e.g., trained) sequences – only the first 3 elements were fully planned prior to execution. Further evidence for online planning comes from a wide range of activities in which visual information is used for planning. For example, when participants make anticipatory eye movements to future targets in reading (Rayner, 1978), sequential reaching (Säfström et al., 2014), and object manipulation (Johansson et al., 2001). More directly, a recent unpublished study tackled the horizon of online planning by restricting the viewing window in a continuous manual tracking task (Bashford et al., 2018). Together, these studies support our view that the ability of the motor system to deal with a stream of incoming stimuli while producing motor responses (i.e., online planning) enables skillful performance of movement sequences.
Motor planning has a limited capacity
We found that the span of the planning horizon (~3-4 movements) was smaller than the typical amount of information that can be stored in short-term memory (Miller, 1956; Cowan, 2010). However, more recent theories of short-term memory posit that capacity is not limited by a fixed number of items, but rather by finite attentional resources that can be flexibly allocated across multiple items (Luck and Vogel, 1997; Bays and Husain, 2008). Similarly, characterizing the planning horizon as having a hard, discrete limit (i.e., a specific number of movements) may not be the best description. A more realistic model would assert that cognitive resources are mainly allocated to the next upcoming movement, with a decaying distribution for elements further into the future (Fig. 7A). Such distribution is consistent with the Competitive Queueing hypothesis (Averbeck et al., 2002, 2006; Rhodes et al., 2004; Kornysheva et al., 2019; Mantziara et al., 2020). Eventually, subjects run out of resources and start executing responses. Completing preceding movements frees up new resources that can be allocated to plan successive movements online. In sum, the discrepancy between the short-term memory span and the planning horizon may reflect the fact that planning a movement takes up more central resources than remembering a digit.
The horizon of motor planning expands with practice
If the capacity of motor planning depends on a soft, flexible horizon, we can ask whether this limit can be improved with practice. In agreement with a previous study (Bashford et al., 2018), we found that practice had expanded the span of the planning horizon. Our conclusion was based on two key observations: 1) the benefit of seeing further ahead was greater later in practice (significant interaction between w and day on MT); 2) the influence of window size on MT can be described with an exponential function whose decay rate decreased with practice (change in the slope of the exponential). Speed improvements that are independent of the amount of available information can be attributed to improved stimulus identification, S-R mapping, or implementation of single responses (Haith et al., 2016; Ariani and Diedrichsen, 2019; Hardwick et al., 2019). As participants become more fluent at translating numbers on the screen into finger movements, each individual press is executed more quickly, thus contributing to faster sequence production across all window sizes. However, the greater performance benefits for larger window sizes together with the expansion of the effective planning horizon indicate that participants improved their ability to make better use of advance information. Unlike previous studies that examined sequence-specific effects in sequence production (Verwey, 2001; Verwey and Wright, 2004; Wiestler and Diedrichsen, 2013; Ariani and Diedrichsen, 2019; Berlot et al., 2020), here we focused on random sequences. Note that, because of this, the observed practice effects cannot be explained by the formation of specific chunking structures previously proposed as a way to deal with the complexity of planning long movement sequences (Ramkumar et al., 2016; Popp et al., 2020). Instead, we found that even without prior experience with a specific sequence, people can improve in the motor planning processes that underlie sequence production. In other words, practice effects are not only about learning what sequence to produce, but also about learning how to coordinate execution and planning efficiently.
Implications for the neural control of sequential movements
The present study provides behavioral evidence that online planning constitutes a central component of motor skill. It also raises the question of how planning and execution processes can simultaneously occur in the brain without interfering with each other. One possibility is that planning and execution take place in separate but communicating anatomical areas, such as the dorsal premotor cortex (PMd) for planning and the primary motor cortex (M1) for execution. However, several studies have reported signals related to movement planning also in brain structures responsible for movement (Crammond and Kalaska, 2000; Ames et al., 2014, 2019; Elsayed et al., 2016; Ariani et al., 2018), with signals often mixed even within single neurons (Evarts and Tanji, 1976; Riehle and Requin, 1989; Alexander and Crutcher, 1990; Prut and Fetz, 1999). Therefore, a more likely scenario is that planning and execution occur in overlapping neuronal populations, but occupy orthogonal subspaces of the multidimensional neuronal code (Fig. 7B), such that planning activity does not trigger motor output (Kaufman et al., 2014; Lara et al., 2018; Zimnik and Churchland, 2020).
Although most neurophysiological studies have focused on planning-related signals before movement onset (preplanning), how the neural substrates for planning change when the same neuronal population has to concurrently control an ongoing movement (online planning) remains an open question. Our behavioral results highlight notable similarities between preplanning and online planning: both processes led to faster performance when participants had a chance to plan up to 3 upcoming sequence elements, with diminishing gains for larger window sizes. Additionally, practice-related improvements were comparable between early IPIs (mostly preplanning) and late IPIs (online planning). These similarities suggest that preplanning and online planning may rely on the same neural process (i.e., motor planning) happening in different contexts, either in isolation before movement initiation or in parallel with execution.
Finally, we showed that advanced planning does not stop at the next movement (+1) but continues for subsequent movements as well (+2 and beyond). How are these parallel planning processes organized at the neural level? Again, one possibility is that they are implemented in separate neural populations in different brain areas, or at least orthogonal subspaces (Fig. 7B). Alternatively, multiple future movements could be planned as an integrated packet (i.e., as a movement chunk), such that there is only one, shared, planning subspace, with specialized code for specific transitions (i.e., 2-3) in movement sequences (Fig. 7C). Evidence for such an integrated code has been reported in studies of the SMA (Tanji and Shima, 1994; Hoshi and Tanji, 2004), which found neurons sensitive to specific sequences of actions.
In conclusion, our study provides a behavioral foundation for the investigation of the neural control of sequential movements, both by establishing online planning as an important element in skilled motor performance and by developing a behavioral paradigm that allows the detailed study of its neural underpinnings.
Acknowledgements
This work was supported by a James S. McDonnell Foundation Scholar award and a NSERC Discovery Grant (RGPIN-2016-04890) to JD, and the Canada First Research Excellence Fund (BrainsCAN). J.A.P is funded by the Canada Research Chairs program. The authors thank Eva Berlot for comments on earlier versions of the manuscript.
Footnotes
Disclosures. The authors declare no conflicts of interest.
We have added detailed implications of our behavioral results for the study of the neural control of sequential movements. Using a neural state-space approach, we propose testable hypotheses about the neural implementation of the ability to plan ahead during movement execution. The novel experimental paradigm described in this paper lays the foundation for future work to directly test these hypotheses in animal models.