Abstract
Skill learning involves the formation of stable motor patterns. In musical and athletic training, however, these stable patterns can also impede the attainment of higher levels of performance, and hence constitute a motor habit. We developed an experimental paradigm to induce a specific motor pattern in a sequence production task and investigated how it affected subsequent optimization over a 3-week training period. Participants initially practiced small segments of 2 to 3 finger movements, which were then combined to form longer sequences. This initial training induced a persistent chunking behavior, with shorter inter-press-intervals within a chunk and longer ones at chunk boundaries. We were able to induce chunking that was either beneficial or detrimental to performance, and could show that the degree to which these detrimental chunk structures were maintained, predicted lower levels of final performance. We also identified two optimization processes by which participants overcame the detrimental motor habits.
Introduction
Humans are capable of astonishing feats of motor skill in athletics, musical performance and dance. But what does it take to become an expert? The first obvious factor is practice: it is estimated that 10,000 hours of training are necessary to develop high-level motor skills (Ericsson et al., 1993; Hayes, 2013). Perhaps the motor system simply needs to gather a large amount of experience to optimize the motor commands that are necessary to achieve skilled movement. Thus, given enough practice, motor expertise may emerge automatically.
However, simply practicing for many hours will not automatically lead to expert performance. There are numerous examples in which motor skill acquisition is slow or fails (Haith & Krakauer, 2018). This is sometimes attributed to the formation of habits: automatic (Hélie, Waldschmidt, & Ashby, 2010; Moors & De Houwer, 2006) and highly entrenched behavioral patterns that resist change through retraining (Ashby, Ell, & Waldron, 2003; Graybiel & Grafton, 2015; Hardwick, Forrence, Krakauer, & Haith, 2017; Jager, 2003; Seger & Spiering, 2011).
Animal models have been integral to the study of habit formation and its neural underpinnings (Jog, Kubota, Connolly, Hillegaart, & Graybiel, 1999; Robbins & Costa, 2017; Smith & Graybiel, 2014, 2016; Wickens, Horvitz, Costa, & Killcross, 2007). However, the majority of animal experiments investigating this concept have focused on habits in the context of action selection – i.e. choosing what action to perform. In contrast, our paper addresses the question of habits in motor performance – i.e. habits in how to perform a chosen action. For example, a tennis player could have a habit in action selection, whereby she always chooses a forehand over a backhand to return a serve. Independently, she could have a motor habit, whereby she executes the forehand without rotating her hips.
Critical to the definition of a habit is that the behavior is maintained, even though it is no longer adaptive (Adams, 1982; Dezfouli & Balleine, 2012; Dickinson, 1985). Most experiments, therefore, demonstrate the existence of a habit by teaching subjects a behavior under one reward contingency and show that it persists even when the reward contingency switches (Ashby et al., 2003; Smith & Graybiel, 2013b). In a similar way, we define motor habits here as a stable way of performing an action that is maintained, even if it prevents optimal performance. This does not imply that motor habits always have to be dysfunctional. Their automatic nature can be beneficial by increasing processing speed (Hardwick et al., 2017) or by reducing cognitive load (Haith & Krakauer, 2018; Hélie & Cousineau, 2011). Thus, habits can be either functional or dysfunctional, but their defining criterion is that they are resistant to change even under circumstance where a change would be beneficial.
To investigate the influence of habit formation on motor skill learning, we introduce a novel experimental paradigm that enables us to induce beneficial and detrimental motor habits and to test whether participants can overcome these habits with practice. As an experimental model of skill acquisition, we used the discrete sequence production task (DSP), in which participants perform an explicitly known series of finger presses as fast as possible (Abrahamse, Ruitenberg, de Kleine, & Verwey, 2013; Verwey, 2001). Learning in this task depends on both cognitive and motor processes (Diedrichsen & Kornysheva, 2015; Wong, Lindquist, Haith, & Krakauer, 2015). Initial performance relies strongly on forming a declarative memory of the sequence and can, therefore, be sculpted through explicit instructions (de Kleine & Verwey, 2009; Verwey, Abrahamse, & de Kleine, 2010; Verwey, Abrahamse, & Jiménez, 2009). An important determinant of the structure of declarative memory is “chunking” – the process by which participants separate a long sequence into smaller subsets (Verwey, 1996; Verwey & Dronkert, 1996) to aid memorization and improve performance by reducing memory capacity demands (Halford, Wilson, & Phillips, 1998; Miller, 1956; Wymbs, Bassett, Mucha, Porter, & Grafton, 2012).
The structure of the declarative memory representation of a sequence may then constrain subsequent motor optimization processes (Bo & Seidler, 2009; Seidler, Bo, & Anguera, 2012). For example, it has been suggested that sequential movements may be optimized within a chunk, but not across chunk boundaries (Ramkumar et al., 2016). We hypothesized therefore that the initial cognitive chunking of the sequence influences the learning of execution-related skills in subsequent motor training. That is, we tested the hypothesis that cognitive chunking can evolve into a motor habit.
We trained participants to perform the same set of 7 sequences, each consisting of 11 isometric keypresses. Training occurred on 14 separate days, spread over 3 weeks. In the induction phase (Fig. 1a) we imposed a specific chunk structure by instructing participants to practice a set of 2-3 digit chunks. They then learned the 11-digit sequences as being composed of four of the pre-trained 2-3 digit chunks. We induced chunk structures that were designed to be either aligned or misaligned with biomechanically easy or difficult finger transitions within the sequence and therefore were predicted to be beneficial or detrimental to performance. Each participant learned 3 sequences using the misaligned chunk structure and 3 distinct sequences with the aligned chunk structure (Fig. 1d). To test for patterns of spontaneous chunking, a separate group of participants (control) was trained on the same sequences but did not receive chunk training and instead practiced the complete sequences during the initial training.
In the subsequent optimization phase (Fig. 1a), participants were instructed to improve their performance through practice. During this phase, we did not make any further mention of chunks (see Methods). Because participants performed the same sequences (but differed in the chunk structures with which they had learned the sequences), we could draw strong inferences about whether their performance was dictated by biomechanical demands (which were identical across participants) or whether it was affected by the chunk structure that was imposed during the induction phase. Using a Bayesian model, we estimated changes in chunk structure. This allowed us to investigate three questions: First, can explicit instructions at the beginning of training lead to stable motor performance patterns? Second, to what degree are these patterns maintained if they impede the participants’ ability to reach skilled performance – i.e. can these patterns be considered a motor habit? Finally, what are the optimization processes that allow participants to overcome bad habits (misaligned chunk structures) through practice?
Results
Over 15 days we trained 32 participants to produce sequences of 11 isometric keypresses from memory on a keyboard-like device. Participants were rewarded with points for executing sequences as fast as possible while keeping the proportion of incorrect keypresses in each block of trials below 15%. We maintained the participants’ motivation by gradually decreasing the movement time (MT) threshold at which they received points.
We manipulated how participants memorized the sequences by splitting the sequences into several chunks, each composed of 2-3 keypresses. We wanted to test whether the different ways of chunking (hereafter “chunk structures”) imposed in the induction phase (Fig. 1a) would affect performance optimization in the subsequent two weeks of training. On the first day, we trained participants to produce eleven 2-3 press chunks in response to a visually presented letter. For example, “A” corresponded to the chunk “3,2,1” (middle finger, index finger, thumb). At the end of the first day, participants could reliably produce the chunks from memory with an average accuracy of 92.7%. On days 2-4, we combined these chunks to form 7 different 11-press sequences. Each sequence was associated with a specific character symbol (e.g. $ see Supp. Table 1). At the end of day 4, participants were able to recall all sequences from memory using the sequence cue with an accuracy of 92.5%.
Chunk induction induces a stable motor pattern
To assess whether the imposed chunk structure influenced participants’ motor behavior, we examined inter-press time intervals (IPIs). An increased IPI is commonly taken as a sign of a chunk boundary, as the cognitive processes (memory recall, action selection) involved in switching from one chunk to another require additional time (Verwey, 1999; Verwey et al., 2010). Hence, we would expect our participants to exhibit shorter IPIs between keypresses that belonged to a chunk imposed during day 1 (within-chunk IPIs) and larger IPIs for the boundaries between chunks (between-chunk IPIs). We indeed found significantly longer between-chunk IPIs compared to within-chunk IPIs in the first few days of training (Fig. 2a: days 2-4: t(31) = 7.728, p = 5.098e-09).
In the optimization phase (day 5-14), we ceased to cue sequences using the alphabetic letters associated with the chunks. Instead, participants were asked to recall the sequences from memory in response to the symbolic sequence cues (e.g. “$”). From this point forward, no further reference to the imposed chunk structure was made. Across days 5-10, the difference between the within- and between-chunk IPIs remained stable; t(31) = 7.165, p = 2.351e-08 (Fig. 2a). Importantly, this difference cannot be attributed to biomechanical difficulty of the finger transitions. The within-chunk IPIs for one group were the between-chunk IPIs for the other group and vice versa; IPIs that were within-chunk for all participants (e.g. the first and last IPI of a sequence) were excluded from this analysis. In summary, even though after day 4 we cued the sequences only with a single symbol, participants persisted in performing the sequences consistent with the chunk structures imposed early in training.
In the last four days of training (days 11-14) we tested whether the persistence of the imposed chunk structure reflected a motor habit or whether it reflected memory recall. We split each experimental group into two subgroups: half of the participants continued to perform the sequences from memory, while the other half were cued using the numbers (Fig. 1a) that indicated the necessary keypresses, therefore removing any memory recall demands. Both the memory (t(15) = 4.865, p = 2.059e-04, Fig. 2b) and the cued subgroup (t(15) = 3.403, p = 0.004) showed a significant difference between the within- and between-chunk IPIs and there was no reliable difference between the two subgroups in this effect (t(30) = −0.749, p = 0.460). Thus, removing the requirement for memory recall did not abolish chunking. Because none of the subsequent analyses showed any significant difference between the two subgroups, we will report their combined results for the remainder of the article. Overall, these results suggest the explicit chunk training early in learning established a stable performance pattern that outlasted 10 days of subsequent practice.
Misaligned chunk structure impairs performance
To show that this initial instruction led to the emergence of a motor habit (as defined above), we needed to not only show that this initial instruction induced a stable temporal pattern of IPIs, but also that this pattern was maintained when it leads to slower execution speeds than other patterns. We therefore designed chunk structures that were predicted to be either beneficial or detrimental to performance. These predictions were based on a separate experiment (see Methods), in which we trained 7 participants on all possible 2 and 3 keypress combinations over the course of 3 days and measured their execution speed. Transitions between two adjacent fingers could be performed faster than two repeated presses of the same finger (t(6) = 13.965, p = 8.404e-06; see Fig. 1b). Given that the 2-3 press sequences hardly taxed the cognitive system, these results can be taken as a characterization of the biomechanical constraints of our specific task (see Methods).
We used these results to design two different ways of separating the sequences into chunks. In one case, we placed chunk boundaries so that they were aligned with digit transitions that were performed more slowly (as measured in the independent dataset) – i.e. they were preferentially placed between digit repetitions (Fig. 1c). The time required to perform these difficult finger transition can therefore simultaneously be used to recall a next chunk, which should benefit overall performance. In the misaligned chunk structure, we placed chunk boundaries at digit transitions that can be performed quickly, thereby breaking up transitions between adjacent fingers or runs (Fig. 1c). Participants would, therefore, have to slow down their performance at these fast transitions to recall the next chunk, which should slow overall performance. Each participant learned 3 of the 7 sequences with a misaligned chunk structure and 3 sequences with an aligned chunk structure, with the assignment counterbalanced across groups. For the last remaining sequence, both ways of chunking were predicted to be equally fast, as both possible chunk structures were aligned with the biomechanical requirements (neutral chunk structure, Fig. 1d & Supp. Table 1).
We predicted that training on the misaligned chunk structure would lead to poorer performance. To quantify performance, we used movement time (MT), the time between the first finger press and the last finger release. Indeed, in the induction phase (days 2-4), the sequences instructed with the misaligned chunk structure were performed slower than the sequences instructed with the aligned chunk structure (one-sample t-test: t(31) = 2.693, p = 0.006; Fig. 3a). Hence, we were not only able to manipulate how participants performed a sequence, but also how well they could perform it. This difference was maintained in the second week of training (days 5-10: t(31) = 2.313, p = 0.014). Importantly, this shows that the stable pattern of IPIs indeed constitutes a motor habit.
Misaligned chunk structure is changed more rapidly
Interestingly, the difference in performance between the sequences instructed with the aligned compared to the misaligned chunk structure was no longer significant in the last four days of training (days 11-14: t(31) = 0.764, p = 0.225; Fig. 3a). This suggests that participants were able to overcome the “bad” habit of a misaligned chunk structure to some degree. To investigate this, we separated the IPI analysis (Fig. 2a) by whether the intervals came from sequences that were instructed using an aligned or misaligned structure. While the difference between within- and between-chunk IPIs for “aligned sequences” was stable over the entire training period, the difference for the “misaligned sequences” disappeared in the last four days of training (Fig. 3b). The three-way interaction between day x within/between x instruction (aligned or misaligned) was significant (F(12,372) = 19.790, p < 1e-16). Thus, in the last four days of training participants seemed to diverge from the misaligned chunk structure while maintaining the aligned chunk structure.
Tracking changes in chunking
A disadvantage of the above analysis, however, is that we cannot discern how participants restructured their chunking and whether they completely abandoned the misaligned chunk structure. For a clearer understanding of how participants changed their chunk structure, we used a Bayesian model to estimate the probability of each possible chunk structure, given the observed series of IPIs, on a trial-by-trial basis (Acuna et al., 2014). The state variable in this Hidden Markov Model indicates which of the 1023 possible chunk structures is present on each trial. Using the expectation-maximization (EM) algorithm (Dempster, Laird, & Rubin, 1977; Welch, 2003), we simultaneously estimated the 9 free parameters of the model (for details see Methods), and the posterior probability for each possible chunk structure on each trial. We accounted for the effects of biomechanical difficulty by regressing out the patterns of IPIs across finger transitions (Fig. 1b) from each block before modeling. Importantly, our version of the model could capture separate learning-related changes to the within- and between-chunk intervals (Fig. 4a). Our method, therefore, allowed us to estimate participants’ chunk structure independently of the overall speed of performance. We confirmed this independence using simulated data (see Methods).
Figure 4b shows two examples of individual participants and sequences. In the first panel, the participant chunked the sequence according to the initial instructions at first, then inserted 1 or 2 additional chunk boundaries, and at the end of training performed the sequence as a single chunk. In comparison, the other participant maintained the instructed chunk structure for most of the training period.
Movement towards a single chunk structure
To characterize changes in chunk structure, we first defined a metric that quantified the difference between two chunking structures: we counted the number of chunk boundaries that differ – i.e. the number of chunks that needed to be split or merged to transform one chunk structure into the other (Fig. 4b - distance). We then used this measure to calculate the distance between the chunk structure used by the participant and three reference structures of interest: (1) the aligned-, (2) misaligned, and (3) a structure that consisted of a single chunk. These distances defined a coordinate system that enabled us to visualize changes in chunk structure. We then projected participants’ estimated chunk structure into this space (Fig. 5a). On the horizontal axis, we plotted the expected distance of participants’ chunk structure to the single-chunk structure. Given the definition of our distance, this measure simply counts the number of chunk boundaries. On the vertical axis, we plotted how close the estimated chunk structure was to the aligned and misaligned chunk structure.
Previous literature has suggested that participants group smaller chunks together with training (Kuriyama, Stickgold, & Walker, 2004; Ramkumar et al., 2016; Sakai, Kitaguchi, & Hikosaka, 2003; Song & Cohen, 2014; Verstynen et al., 2012; Verwey, 1996; Wymbs et al., 2012), a process that may help to improve performance (Abrahamse et al., 2013; Ramkumar et al., 2016; Verwey, 1999, 2001; Verwey et al., 2010; Verwey & Wright, 2014). In nearly all previous studies, however, the estimated number of chunks is biased by the overall movement speed. Using a modified probabilistic model (see Methods), we were able to disambiguate the two and critically test this assumption. We estimated the number of chunk boundaries for each participant averaged across sequences (the neutral sequence was excluded). Interestingly, on the 2nd day, participants separated sequences into more chunks than the 4 chunks we instructed (Fig. 5a, t(31) = 4.224, p = 0.0002). This tendency continued on day 3, where participants tended to subdivide the sequences into even smaller chunks (Fig. 5b; day 2 vs. 3: t(31) = 2.023, p = 0.052). After day three the number of chunk boundaries decreased as shown by a significant effect of day in a repeated measures ANOVA (F(11,341) = 11.710, p < 1e-16). However, even in the last phase of training, participants performed the sequences with an average of 2.9 chunk boundaries (we instructed 3 chunk boundaries). Thus, while there was a clear tendency towards merging chunks after an initial increase, participants did not perform the sequence as a single chunk, even after 3 weeks of practice.
Participants abandoned the misaligned chunk structure to a greater degree
Next, we probed how much participants diverged from the initial instructions. Participants slowly changed their chunk structure for both aligned and misaligned instructed sequences with training. The average distance to the instructed chunk structure increased systematically over time (repeated measures ANOVA, effect of day, F(12,372) = 7.055, p < 1e-16, Fig. 5c).
Consistent with our IPI analysis (Fig. 3b), we observed that participants abandoned the misaligned chunk structure to a greater degree than the aligned chunk structure (Day x Instruction interaction: F(12,372) = 5.610, p < 1e-16). In the last four days of training, the sequences that were instructed with the misaligned chunk structure were more dissimilar to the misaligned chunk structure than the sequences that were instructed with the aligned chunk structure were to the aligned chunk structure: t(31) = 2.294, p = 0.029 (Fig. 5c). Additionally, we found a significant Day x Instruction interaction (F(12,372) = 2.215, p = 0.011) for the distance to a single chunk (Fig. 5b), suggesting a stronger tendency towards performing a sequence as a single chunk when trained on the misaligned chunk structures. Together these results indicate that participants changed their chunking behavior more readily for sequences that were trained using the misaligned chunk structure than when trained using the aligned chunk structure.
Despite the divergence from the misaligned chunk structure with training, our analysis also revealed that participants did not overcome the influence of the instruction completely. In the third week, sequences produced after training with a misaligned chunk structure were still performed closer to the misaligned structure than to the aligned structure (t(31) = 6.962, p < 1e-16). This shows that even training on misaligned chunk structures had a lasting influence on participants’ motor behavior.
Chunk structure “crystallizes” with training
Would longer training allow participants to completely overcome the influence of the instruction and to perform all sequences as a single chunk? Although experiments with longer training are necessary to provide a definitive answer, our data indicate that this process, if occurring, may take a very long time. The amount of change in the chunk structure for each sequence reduced dramatically in the last week of training, suggesting that a stable motor habit formed. This phenomenon is akin to the development of an invariant temporal and spectral structure in bird-song, a process that has been termed “crystallization” (Brainard & Doupe, 2002). To estimate crystallization, we calculated the distance between the chunk structures from one day to the next (Fig. 5d) and within each day from one trial to the next (Fig. 5e). The analysis was performed separately for each sequence and participant. Overall, both the day-to-day distance (F(11,330) = 18.794, p < 1e-16) and the trial-by-trial distance decreased significantly across training days (F(12,456) = 13.245, p < 1e-16). Therefore, participants appeared to settle onto a stable pattern in the last week. Consequently, additional training would likely only lead to slow changes in their chunk structure.
In summary, our analyses provide a clearer picture of how chunking changes with learning. Firstly, in line with previous research (Kuriyama et al., 2004; Ramkumar et al., 2016; Sakai et al., 2003; Song & Cohen, 2014; Verstynen et al., 2012; Verwey, 1996; Wymbs et al., 2012) participants gradually moved towards performing the sequence as a single chunk by dividing the sequence into fewer chunks. Secondly, participants diverged from the instructions over time with a quicker deviation from the misaligned chunk structure. Nevertheless, they did not completely overcome the initial instruction, nor did they perform the sequences as a single chunk at the end of training. Considering that the chunk structure crystallized in the last four days of training, these results demonstrate the formation of a stable motor habit that is still influenced by the initial instruction.
Spontaneously emerging chunk structures
To investigate how participants would spontaneously chunk the sequences, we tested an additional control group (N=8), which did not receive any explicit chunk training. Rather, participants were presented with the entire sequences on the first day and had to memorize them without any reference to chunks (see Methods for details). Even though memorization was more difficult, the control group did not differ significantly from the experimental groups in terms of their explicit knowledge on day 4 (t(36) = 1.288, p = 0.206), or in their overall MT across training (main effect of group: F(1,38) = 0.101, p = 0.753; interaction between group and day (F(1,38) = 1.387, p = 0.168).
Similar to the experimental groups, the control group initially subdivided the sequences into small chunks and then slowly combined them into larger chunks. The distance to a single chunk structure decreased significantly over days (F(12,84) = 17.977, p < 1e-16), and reached a level that was not statistically different from the experimental participants on the last day of training (t(38) = −0.940, p = 0.353). Interestingly, on the first day, the control group performed the sequences closer to the misaligned chunk structure than to the aligned chunk structure (t(7) = −2.799, p = 0.027). With training, participants then moved closer to the aligned chunk structure, as indicated by a significant change in the difference between the distance to the aligned and misaligned chunk structure across days (F(12,84) = 5.303, p < 1e-16). The control group also showed clear crystallization over time (see Figure 5d&e). Compared to the experimental groups, control participants showed a higher day-to-day and trial-by-trial change in the beginning of training, which then reduced more quickly (Group x Day interaction; day-to-day: F(11,330) = 3.780, p = 4.003e-05; trial-by-trial: F(12,456) = 4.254, p = 2.167e-06). In summary, the control group showed similar behavioral patterns to the experimental participants, indicating that similar processes of habit formation are also at play in the absence of explicit instructions.
Two optimization processes correlate with faster final performance
How did these changes in chunk structure determine how fast participants could execute the sequences at the end of training? We first asked whether performing the sequences using larger chunks would facilitate performance. For each participant, we therefore regressed the MT for 6 sequences (last 4 days, excluding the neutral sequence) against the corresponding distance to the single chunk structure (Fig. 6a). The majority of the participants showed a positive relationship between the number of chunks and MT: a one-sample t-test indicated that the individual slopes were significantly greater than 0 (Fig. 6a, t(31) = 6.104, p = 4.560e-07). This significant relationship was also found for the control participants (Fig. 6b, t(7) = 3.429, p = 0.006). Thus, performing the sequences with fewer chunks led to better performance.
Secondly, we investigated whether performing the sequences in alignment with the biomechanical constraints was also beneficial. We regressed the MT for 6 sequences in the last four days of training against the corresponding distance to the aligned chunk structure. On average the individual slopes again were significantly greater than 0, both for the experimental (Fig. 6c; t(31) = 2.220, p = 0.017), and control group (Fig. 6d, t(7) = 2.720, p = 0.015). Finding a better way of chunking (for the same number of chunk boundaries) therefore also improved performance.
To visualize the relationship between the chosen chunk structure and the MT in the last four days of training, we plotted the MT and chunk structure for each sequence and participant in the 2-dimensional space defined in Fig. 5a (Fig. 7). This visualization clearly shows the performance benefit from being closer to a single chunk as well as being closer to the aligned chunk structure.
Overall, these results suggest that the two optimization processes - joining chunks and aligning the remaining chunk boundaries with biomechanical constraints - positively influence participants’ ultimate performance. Furthermore, sequences for which participants could not develop a better way of chunking were performed substantially slower.
The amount of musical training did not systematically affect chunk behavior
One possible confound is the amount of musical training that participants’ were exposed to prior to the experiment. We found that participants with piano experience (55%) performed the sequences faster compared to the non-experienced participants (t(38) = 6.297, p = 2.227e-07). This is perhaps not surprising, given the similarity of our task with playing the piano. Furthermore, the number of practice years significantly correlated with MT (t(38) = −3.850, p = 4.401e-04). Importantly, however, the amount of participants’ prior musical experience neither influenced the distance to the instructed chunk structure in the last week of training (t(30) = −0.291, p = 0.773; practice years: t(30) = 0.059, p= 0.954) nor the distance to a single chunk (t(38) = −0.602, p = 0.551; practice years: t(38) = −0.380, p = 0.706). Therefore, musical training did not seem to have a qualitative influence on participants’ chunking behavior.
Discussion
In this study, we utilized chunking as a tool to investigate the role of motor habits in skill learning. We influenced the structure of the initial declarative sequence representation by manipulating how participants memorized them (Park, Wilde, & Shea, 2004). By experimentally imposing two different chunk structures on the same physical sequence, we could make causal inferences on the effects of cognitive chunking on motor skill development. This is an important advance over previous observational studies (Ramkumar et al., 2016; Wright, Rhee, & Vaculin, 2010; Wymbs et al., 2012), which did not experimentally control how participants chose to chunk the sequence.
This paradigm yielded three main results. First, consistent with previous studies (de Kleine & Verwey, 2009; Verwey et al., 2010, 2009; Verwey & Dronkert, 1996), our data demonstrate that a stable chunking pattern can be induced through cognitive manipulations during sequence learning. Importantly, participants did not completely overcome this imposed chunk structure, even after 2 weeks of additional training. Participants’ chunk structure crystallized towards the end of training, making it unlikely that the influence of the initial instruction would disappear completely with longer practice. Finally, the chunking structure remained stable, even when the task changed from a memory-guided to a stimulus-guided task. Thus, the initial instruction led to the formation of specific motor patterns that were still clearly measurable after three weeks of training.
Second, we tested whether this stable pattern of chunking could be considered a motor habit. To do so, we designed two different ways of instructing the sequence, one aligned and the other misaligned with its biomechanical requirements. This manipulation either facilitated or impeded performance in the first two weeks of practice. We could show that participants did not overcome the misaligned structure completely, even though it was detrimental to their performance. Thus, the stable chunking pattern meets the requirements (as laid out in our definition) for being called a motor habit. Therefore, we believe that studying sequential chunking can provide valuable insights into the neural systems underlying motor habits. Indeed, it has recently been suggested that chunking plays an integral role in the formation and expression of habits (Dezfouli, Lingawi, & Balleine, 2014; Graybiel, 2008) and is neurally represented in the dorsal lateral striatum as action “start and stop signals” (Barnes, Kubota, Hu, Jin, & Graybiel, 2005; Graybiel, 1998; Jin, Tecuapetla, & Costa, 2014; Smith & Graybiel, 2013a, 2014).
Finally, our results also indicate that the “bad” habit was not completely immutable. Participants were able to modify the misaligned chunk structure, and did so more rapidly than the aligned chunk structure. As a consequence, the performance detriment imposed by the misaligned instruction was no longer significant on the group level in the last week of training.
We identified two ways by which participants overcame the limitation induced by the bad habit. After initially breaking up the instructed sequences into 5 chunks on average, participants then joined chunks together, decreasing the amount of additional time spent on chunk boundaries. While previous research has suggested that the size of chunks increases with training, these findings were usually conflated with the overall speed of the action (Solopchuk, Alamia, Olivier, Ze, & Zénon, 2016; Song & Cohen, 2014; Wymbs et al., 2012). Using a Bayesian model to assess chunk structure independent of performance, we could demonstrate a positive relationship between chunk concatenation and execution speed, both in the experimental as well as in the control group that developed a chunking strategy without explicit instructions. However, our results also indicate that participants did not merge all sequences into a single chunk after 3 weeks of training, but on average subdivided each sequence into 3-4 chunks. This suggests that the number of motor actions that can be joined in a single chunk may be limited (Langan & Seidler, 2011; Ramkumar et al., 2016; Verwey & Eikelboom, 2003; Verwey, Lammens, & Van Honk, 2002).
A second (and novel) finding was that participants also optimized performance by rearranging chunk boundaries in a biomechanically efficient manner. Consistent with our prediction based on the difficulty of individual digit transitions, placing chunk boundaries at digit transitions that take more time to execute resulted in faster performance for the full sequence. This optimization process was also observable in the control group that memorized and practiced sequences on their own terms.
Conversely, we observed that sequences that were not chunked in line with these strategies were performed slower. Therefore, if a more beneficial way of chunking was not found, participants still showed a detriment, suggesting that other learning mechanisms cannot fully make up for a persistent bad habit. Considering that participants’ behavior became highly invariant in the last week of practice, we predict that some bad habit will remain and continue to influence participants’ performance even after prolonged training.
In many motor tasks, there are numerous strategies and processes that can lead to excellent performance (Verstynen et al., 2012; Verwey et al., 2010). Examining Figure 7, one can observe that the shortest MTs were achieved anywhere in the space between the aligned and single chunk structure. Occasionally, good performance was also reached in other positions in chunk space. Participants adopted quite idiosyncratic chunk structures for each sequence at the end of training. This suggests that there may be considerable inter-individual variability in which technique works best for reaching a high level of performance. While we based our biomechanical constraint estimates on a representative sample, it might not perfectly reflect the constraints experienced by each participant. Alternatively, a number of ways of chunking may work approximately equally well, such that the cost of changing an established habit may outweigh the small benefit that could be gained from changing the structure. A similar observation can be made in sports, where even top-ranked athletes use slightly different techniques to reach similar levels of performance. This variation may reflect individual biomechanical differences but also differences in instruction and training combined with subsequent habit formation.
The establishment of a novel experimental paradigm to study motor habit formation will allow us to explore ways to encourage learners to abandon or change a current habit. While our attempt at accelerating this process by changing the task from a memory-based to a stimulus-based task was ultimately not successful, there are many other techniques that would be possible. In many disciplines, teachers have developed ways to help students overcome bad habits. For instance, the Hanon piano exercise helps students play difficult passages of a musical piece by breaking up learned phrases into new chunks to explore different rhythms. Playing a passage slower than intended has also been suggested to break bad habits (Chang, 2004). Overall, the general advice from the diverse literature on learning piano is to diversify training and to practice with careful awareness to prevent bad habits from forming (Sadnicka, Kornysheva, Rothwell, & Edwards, 2018). This suggests that changes in context and the exploration of novel ways of moving can aid performance and the abandonment of bad habits.
While our experimental design enabled us to manipulate participants’ habits in a laboratory setting, sequence learning only captures a specific aspect of motor skill acquisition. Nevertheless, similar persistence of habits has been observed in other motor learning paradigms (Diedrichsen, White, Newman, & Lally, 2010). In bimanual coordination, for instance, Park et al. (2013) showed that an acquired pattern stayed remarkably stable even over 8 years of not performing the task.
The current study shows that motor habits can be cognitively induced and can remain stable for extended time periods, even though they may prevent further performance gains. Furthermore, the study provides the first insights into the learning processes that are involved in overcoming a detrimental habit. Our experimental paradigm allows the further study of how we can aid the abandonment of bad habits.
Methods
Participants
Forty neurologically healthy participants were recruited for the study (30 females; ages: 19 to 33). Thirty-two were randomly split into two experimental groups and the remaining eight participants were assigned to the control group. All participants were right-handed based on the Edinburgh Handedness Inventory and completed informed consent. On average, participants had received 4.68 (± 5.55) years of musical training, with 55% percent having played the piano for more than half a year. The study protocol was approved by the ethics board of the University of Western Ontario.
Apparatus
A custom-built five-finger keyboard was used. The keys were not depressible, but were equipped with a force transducer (FSG-15N1A, Sensing and Control, Honeywell) underneath each key which reliably measured participants’ isometric force production with a repeatability of <0.02N and a dynamic range of 16N (Wiestler & Diedrichsen, 2013; Wiestler, Waters-Metenier, & Diedrichsen, 2014; Yokoi et al., 2017). The signal was amplified and sampled at 200 Hz.
Discrete sequence production task
We used a discrete sequence production task (DSP), in which participants had to execute sequences of 2, 3, or 11 keypresses as fast as possible while keeping their error rate under 15% within each block. A trial was termed erroneous if participants pressed a wrong key anywhere within the sequence. No pause between presses was required and thus some co-articulation between fingers emerged with faster execution. A finger press was detected when the given finger produced a force above 3N. Subsequently, a finger was detected as released when the force of the same finger fell below 1.5N. In order for a subsequent finger to be registered as pressed the previous finger had to be released. This rule prevented participants to press with more than 2 fingers at once. The force magnitude applied to each key was represented by 5 lines on an LCD monitor, with the height of the line representing the force in the corresponding finger. A white asterisk (memory-guided conditions) or digit (cued condition) for each finger press was presented above the lines. Immediately after the press threshold was reached, participants received visually and auditory feedback. If the press was correct, the white cue changed its color to green and a sound was presented. If the press was incorrect, the cue turned red and a lower-pitch sound was presented. After each trial, participants received points based on their accuracy and movement time (MT; the time between the first press and last release). Correct sequences performed faster than the current MT threshold were rewarded with 1 point. MTs that were 20% faster than the threshold were rewarded with 3 points. Incorrect presses or MTs exceeding the threshold resulted in 0 points. At the end of each block, participants received feedback on their error rate, median MT, points obtained during the block, and total points obtained during the session. In order to maintain motivation, we adjusted the MT threshold by lowering the threshold by 500ms after each block in which the participants performed with an error rate of 15% or lower and had a median MT faster than the current threshold. This manipulation resulted in an approximately stable overall success rate of 61% SD: 13% (0.27% 1pt, 0.34 % 3pt) across the entire experiment.
Study design
To impose a particular way of chunking, we first had participants memorize and perform smaller 2-3 press chunks. These chunks were then combined to form the training sequences. All participants were trained on the same 7 sequences, each consisting of 11 digit presses (see suppl. Table 1). Each participant completed 14 training sessions in total: one session per day across a 3-week period (excluding weekends). Each session lasted approximately 1 hour, excluding the two initial sessions and the last session which took 2 hours. Participants completed at least 10 blocks of 28 trials per training day. Each block comprised 4 repetitions of each of the 7 sequences. Each trial started with the visual presentation of the sequence to be executed and was completed once the participants pressed the amount of presented numbers (irrespective of whether the pressed keys were correct or incorrect).
To verify that the chunking behavior was influenced by the instruction, we used two different ways of chunking. We split each sequence either into one 2-digit and three 3-digit chunks (2-3-3-3, misaligned) or into three 3-digit chunks and one 2-digit chunk (3-3-3-2, aligned). Each participant practiced half of the sequences with one chunk structure and the other half of the sequences with the other chunk structure. This resulted in in two different sets of chunks (suppl. Table 2). The control group did not receive any explicit chunking instructions.
Days 1-4: Chunk induction & initial sequence learning
Experimental group
At the beginning of training the experimental groups were pre-trained on a specific set of eleven 2- or 3-digit chunks (2 two-press chunks and 9 three-press chunks). Participants received one of two different sets of chunks (suppl. Table 2). Each chunk was consistently associated with a letter of the alphabet (A-K). Participants were explicitly told to learn this association. Each chunk was presented twice in succession. In half of the blocks, on the first trial of each chunk presentation, the numbers corresponding to the finger presses accompanied the letter on the screen while on the second trial participants had to recall the presses solely based on the letter (numbers were interchanged with stars). This trial order was reversed on every second block. To ensure that participants had memorized the chunks we added speeded recall blocks at the end of days 1 and 2. After practicing the 2-3 press chunks on day 1 and at the beginning of day 2, participants trained on the seven 11-press sequences. Each sequence was associated with a symbol (e.g. $; suppl. Table 1). Each sequence was presented twice in succession and participant had to perform the sequences from memory using the sequence cue on one trial or with the help of the chunk letters on the next trial. We tested participants’ sequence knowledge with a self-paced recall block at the end of days 2-4 (The first two participants did not perform the recall blocks).
Control group
The control group did not receive any chunk training but instead trained directly on the seven 11-press sequences. On the first day, the control participants practiced the sequences using the digits presented on the screen. We matched the amount of training across groups by ensuring that all participants were required to produce the same number of finger presses. On the first day, the control participants were not aware that they would have to memorize the sequences later on. On days 2-4 they were then instructed to memorize the sequences using the same sequence cues as the experimental groups and were subsequently tested on their sequence knowledge. The rest of the experimental design was identical for all groups.
Days 5-10: Optimization - Memory Recall
For the days 5-10 of training participants practiced exclusively on the entire eleven-press sequences and chunks were no longer mentioned or trained on. Each sequence was presented twice in succession and participants had to recall the sequence from memory on both trials using the sequence cue.
Days 11-14: Optimization - Memory recall or cued presentation
During the last four days of training, half of the experimental participants performed the sequences from memory while for the other half and for the control participants we removed the sequence cue and presented participants with the actual numbers that corresponded to the sequences (Fig. 1a). Participants completed an additional generalization test on day 15. The results of this test are not reported in this article.
Biomechanical baseline study
We conducted a separate study to determine the influence of biomechanical difficulty on the finger transition speed. 7 participants (5 females, ages: 21-27) participated in this 3-day study. Participants executed all possible two-finger transitions (25) and three-finger transitions (125), each 8 times per day (each sequence was presented twice in a row). Each day participants completed 8 blocks with 150 trials each. The setup and motivational structure were identical to the main experiment. We found that on our device, finger repetitions (e.g. 2-2) were executed more slowly than presses of neighboring fingers (e.g. 2-1) To press the same finger twice, the force applied to the key had to first exceed the press threshold (3N), then go below the release threshold (1.5N) and then cross the press threshold again. This rapid alternation of forces takes time to produce. In contrast, for two adjacent fingers, the second finger can be already pressed before the finger is released (have already reached the press threshold), making it easier to rapidly produce this force pattern. Even though participants improved the overall speed from 622ms on the first to 522ms on the third day, the 5×5 pattern of relative IPI was stable across both participants (average correlation r = 0.689) and days (r = 0.894).
Aligned vs. misaligned chunk structures
To determine how to design our sequences and chunk structures to aid or impede performance we used the finding from the biomechanical baseline study that finger repetitions (e.g. 11) are performed slower than presses of adjacent fingers (e.g. 12).
We designed the sequences such that they would include both fast transitions (runs e.g. 123) and slow finger repetitions (113; suppl. Table 1). Depending on which chunk structure was instructed, these transitions would either fall on a chunk boundary or lie within a chunk. We counterbalanced this within and between participants, meaning that each participant trained on 3 sequences with the aligned chunk structure and 3 with the misaligned chunk structure (suppl. Table 1). One control sequence was added which included a within-chunk run for both groups.
Statistical Analysis
We recorded and analyzed the force traces for each finger. For each trial, we calculated the reaction time (RT, time between presentation and first crossing of the threshold), movement time (MT, time between first press and last release) and inter-press-intervals (IPIs; time between force peaks of two consecutive presses). All analyses were performed using custom-written code for MATLAB (the MathWorks). We excluded trials that contained one or more incorrect presses from our analyses, as well as trials with an MT or a press with an IPI three standard deviations above the mean. The data were analyzed using mixed-effects analysis of variance (mixed ANOVA), Pearson’s correlation and paired and one sample t-tests. All t-tests were two-sided. A probability threshold of p<0.05 for the rejection of the null hypothesis was used for all statistical tests. For the regression analyses as well as for calculating the MT difference between the sequences with misaligned and aligned instruction we normalized the data for each participant by subtracting the mean performance for each day due to a wide range of performance speeds.
Probabilistic model for estimating chunk structure
We used an extended version of a Bayesian model of chunking behavior, developed by Acuna et al. (2014). The algorithm uses a Hidden Markov Model to estimate the posterior probability that a specific chunk structure is active on a given trial. As we had 10 digit transitions, each of which could either coincide with a chunk boundary or not, we had to consider 210-1= 1023 possible chunk structures. Between trials, the hidden Markov process could either stay in the same chunk structure with probability p or jump to any other chunk structure with probability (1-p)/1022. The IPIs were modeled as a Gaussian random variable, with a different mean and variance, depending on whether the transition was within- or between-chunks. In contrast to Acuna et al.(Acuna et al., 2014), where learning effects were removed in a preprocessing step using a single exponential, we modeled the learning within our model with two separate exponentials for the IPI mean. This captured the faster reduction in the between- compared to the within-chunk intervals (Fig. 2a). The inclusion of separate learning curves for within- and between-chunk IPIs into the model allowed us to estimate participants’ chunk structure independently of the overall performance speed. This is an important advance over previous methods that used a constant cutoff value to distinguish between within- and between chunk intervals. For these methods, faster performance would automatically decrease the number of chunk boundaries detected. To confirm that our algorithm did not show this bias, we simulated artificial data using parameter estimates for individual participants. We simulated sequences that switched between 4 different chunk structures, each of which contained 4 chunks. Even though IPIs decreased by about 300ms with learning, the estimated average number of chunks remained stable across the entire simulated experiment (average distance to single chunk: 3.35).
The model did not use errors and IPIs covariance structure, as these did not relate systematically to the imposed chunk structure even early in training. We used an Expectation-Maximization algorithm to simultaneously estimate the posterior probability of each chunk structure for each trial, as well as the 9 parameters of the model: 3 parameters each for the exponential curve for the within- and between-chunk IPIs, 1 variance parameter for each, and the transition probability p.
As a preprocessing step, we regressed the IPIs for each subject against the average biomechanical profile, which was estimated as the average IPI profile for all possible 2 digit-presses from our biomechanical baseline experiment (Fig. 1b). The fitted values were removed from the IPIs. Removing temporal regularities that could be modeled with biomechanics alone should result in chunking estimates that more closely reflect cognitive and learning influences. Qualitatively comparable results were also obtained using the raw IPIs, without biomechanical factors removed.
Expected distance
We quantified how much participants changed their chunking behavior over time by calculating the expected distance between two estimated chunk structures. The distance between two chunk structures, d(i,j), was defined as how many of the 10 transitions would have to change from a chunk boundary to a non-boundary (and vice versa) to transform one structure into the other (for an example, see Fig. 4b). A distance of 0 would indicate no change and the average distance between two randomly chosen chunk structures is 5. Because we did not know for certain which chunk structure participants adopted in each trial, we calculated the expected distance. For this, we first calculated a 1023 × 1023 matrix containing the distances between any chunk structure i, and chunk structure j. From the posterior probability distribution, we could then derive how likely each of these chunk structure changes was, p(i,j). The expected value of the distance was then simply calculated as
Code availability
Bayesian algorithm code available on GitHub: https://github.com/jdiedrichsen/chunk_inference.
Custom MATLAB code is available from the corresponding author on request.
Data availability
The datasets generated during and analyzed during the current study are available from the corresponding author on request.
Author contributions
N.J.P. and J.D. designed the experiment. N.J.P, A.Y. and J.D. programmed the experiment. N.J.P collected the data and analyzed the data together with J.D., N.J.P., J.D., and P.L.G. wrote the manuscript.
Acknowledgments
This work is supported by a James S. McDonnell Foundation Scholar award, a Natural Sciences and Engineering Council of Canada (NSERC) Discovery Grant (RGPIN-2016-04890) and the Canada First Research Excellence Fund (BrainsCAN) to J.D., a NSERC Discovery Grant (RGPIN 238338) and a Canadian Institutes of Health Research Grant (PJT-153447) to P.L.G. We thank Aaron L. Wong for helpful comments on earlier versions of this manuscript.
Footnotes
Conflict of interest The authors declare no conflict of interest.