Abstract
Humans are highly attuned to patterns in the environment. This ability to detect environmental patterns, referred to as statistical learning, plays a key role in many diverse aspects of cognition. However, the spatiotemporal neural mechanisms underlying implicit statistical learning, and how these mechanisms may relate or give rise to explicit learning, remain poorly understood. In the present study, we investigated these different aspects of statistical learning by using an auditory nonlinguistic statistical learning paradigm combined with magnetoencephalography. Twenty-four healthy volunteers were exposed to structured and random tone sequences, and statistical learning was quantified by neural entrainment. Already early during exposure, participants showed strong entrainment to the embedded tone patterns. A significant increase in entrainment over exposure was detected at central sensors, reflecting the trajectory of learning. While source reconstruction revealed a wide range of brain areas involved in this process, entrainment in right temporo-parietal and frontal areas as well as left pre-central and frontal areas significantly predicted behavioral performance, especially in the last third of stimulus exposure. These results give insights into the dynamic relation between neural entrainment and explicit learning of triplet structures, suggesting that these two components are systematically related yet dissociable. Neural entrainment reflects robust, implicit learning of underlying patterns, whereas the emergence of explicit knowledge, likely built on the implicit encoding of structure, varies across individuals and may depend on both sufficient exposure time and attention.
1. Introduction
Living in a dynamically changing environment, humans and other animals are highly attuned to structure in their surroundings. They are able to extract patterns from their surroundings automatically, incidentally and through mere exposure by using a process called statistical learning (Saffran et al., 1996a). The first experiment on statistical learning demonstrated that infants were able to extract embedded patterns from a continuous stream of speech input by becoming sensitive to the transitional probabilities between neighboring syllables (Saffran et al., 1996a). This finding suggested that statistical learning plays an important role in language acquisition, and spurred a large body of additional work in this area, not only in developmental populations (e.g. Pelucchi et al., 2009; Fló et al., 2019; Benitez et al., 2020), but also in adults (e.g. Saffran et al., 1996b; Cunillera et al., 2009; Batterink and Paller, 2017). Importantly, many subsequent studies have shown that statistical learning is not restricted to language, but also occurs to nonlinguistic auditory (e.g. Saffran et al., 1999; Gebhart et al., 2009), visual (e.g. Turk-Browne et al., 2005; Bulf et al., 2011) and cross-modal stimuli (e.g. Cunillera et al., 2010; Paraskevopoulos et al., 2018).
One key benefit of using nonlinguistic stimuli to investigate statistical learning in the auditory domain is that it can be assumed that learners are “blank slates,” with learning not heavily influenced by prior knowledge. In a recent study, Siegelman and colleagues (2018a) demonstrated that statistical learning to linguistic stimuli (i.e., syllables) is strongly shaped by learners’ existing phonotactic knowledge and expectations about the input. To demonstrate this, they measured internal item consistency on one linguistic and two nonlinguistic (auditory and visual) statistical learning tasks. Results for items in the nonlinguistic tasks were highly correlated, while the linguistic task had a low internal item consistency (Siegelman et al., 2018a). Additionally, their experiments showed a strong within-subject correlation in performance in the two nonlinguistic tasks. This correlation across modalities was not present for a visual and a linguistic statistical learning task (Siegelman and Frost, 2015). These results suggest that basic, domain-general, statistical learning computations may be tested most reliably using stimuli that do not strongly invoke prior knowledge.
Traditionally, statistical learning has been tested after a certain exposure period to structured stimuli— for example, with a two alternative forced choice (2AFC) task in adults (e.g. Gebhart et al., 2009) or with looking-time paradigms in infants (e.g. Saffran et al., 1999). These tests are limited in that they only reveal learners’ ability to recognize the preceding input, and do not capture other, possibly more implicit, aspects of learning (e.g. Batterink et al., 2015a). In addition, while these behavioral outcome measures provide a one-time snapshot of learners’ knowledge after the learning process, they neglect the temporal dynamics of learning—that is, the same level of performance on these tasks can originate from completely different learning trajectories (Siegelman et al., 2018b). In contrast, methods that involve monitoring during the learning period have been argued to provide a more complete picture of statistical learning. At the behavioral level, reaction time approaches can be used for such monitoring (Siegelman et al., 2018b). Another, possibly even more advantageous approach is the use of neuroimaging methods to capture statistical learning—such methods can track learning processes without the requirement of an overt behavioral task, and can also shed light on the neural mechanisms and/or neural structures involved in learning.
Both Electroencephalography (EEG) and Magnetoencephalography (MEG) have been used to track statistical learning during the learning period. For example, Cunillera et al. (2009) found an increase in the amplitude of the event-related N400 component during exposure to an artificial language compared to a random syllable stream. Using MEG, Paraskevopoulos et al. (2012) demonstrated a mismatch response to part-triplets compared to triplets during exposure to a statistically regular stream, even though participants’ post-exposure recognition was at chance level. This finding suggests that neural measures can be more sensitive indices of learning than post-exposure behavioral measures (see also Turk-Browne et al., 2009). Another effective approach, which has a high signal-to-noise ratio and is well-suited to capture the neural response to a continuous sensory stream, involves the measurement of neural entrainment. Neural entrainment refers to a property of the electromagnetic activity of the brain to resonate at the same frequency as an ongoing rhythmic stimulus. This neural response can be quantified by either investigating changes in the power (e.g. Buiatti et al., 2009; Farthouat et al., 2017) or inter-trial-phase coherence (ITC; e.g. Batterink and Paller, 2017, 2019) at the frequency of the stimulus and of larger embedded patterns in a statistical learning stream. Studies using this approach have demonstrated that neural entrainment to hidden patterns increases over the exposure period (Batterink & Paller, 2017, 2019) and also predicts performance on behavioral tests administered after the learning period (Batterink & Paller, 2017, 2019; Buiatti et al., 2009). Taken together, these results suggest that neural entrainment reflects the successful perceptual grouping of raw stimulus elements into cohesive units that occurs during statistical learning (e.g., syllables into words). Importantly, such methods allow for the quantification of statistical learning during the acquisition process, independent of participants’ later behavioral responses.
In addition to shedding light on the time course of learning (Batterink and Paller, 2017, 2019), a number of other neuroimaging studies have revealed which areas in the brain are active during statistical learning tasks. Previous studies using functional magnetic resonance imaging (fMRI) and functional near infrared spectroscopy (fNIRS) have yielded mixed results. While most authors agree on the importance of the superior temporal cortex in statistical learning, other findings implicate the premotor cortex (Cunillera et al., 2009), the inferior frontal cortex (Abla and Okanoya, 2008; Karuza et al., 2013), or the supramarginal gyrus (McNealy et al., 2006). However, because fMRI and fNIRS rely on indirect measurement of neural activity (through blood oxygenation levels), these results cannot directly be related to time-locked activity of neural populations, as reflected by neural entrainment. Attempting to gain both temporal and spatial information, Farthouat et al. (2017) used MEG to assess nonlinguistic auditory statistical learning, presenting embedded tone triplets in a continuous stream. They detected an increase of power at the frequency of tone triplets from the third minute of exposure on, and were able to reconstruct this increase to the left posterior temporal sulcus and supplementary motor area. Nevertheless, participants’ behavioral responses in a 2AFC task were at chance level, precluding a direct link between neural responses and behavioral measures of learning (Farthouat et al., 2017).
These results, which include traditional behavioral measures of statistical learning as well as neuroimaging measures, collectively highlight two important components of statistical learning: (1) the dynamic learning trajectory, which may be assessed through sensitive neural measures and occur even in the absence of behavioral evidence of learning, and (2) the ability to explicitly express the learned information, which seems to be present in some studies (or some participants) but not in others. Insight into how these two components are related is key to gaining a deeper understanding of the underlying dynamics and neural mechanisms of statistical learning. To this end, some previous studies have attempted to link neural responses with subsequent behavior. For example, in a linguistic statistical learning task Karuza et al. (2013) showed that neural activation in the left inferior frontal gyrus was related to participants’ behavioral learning trajectory. Similarly, Abla et al. (2008) reported that N400 amplitudes elicited by the first tone of a tone-triplet were highest within the first ∼7 min of exposure in participants classified as high learners based on 2AFC performance, compared to middle and low learners. In addition, as mentioned previously, several studies have found a positive association between neural entrainment during learning and subsequent performance on behavioral statistical learning tests (Batterink et al., 2017, 2019; Buiatti et al., 2009).
In the current study, we combined the strengths of multiple approaches in order to investigate statistical learning over time and space, and its reflection in behavior. We used a nonlinguistic task that is less likely to be influenced by learners’ prior knowledge. In addition, we used MEG to monitor participants’ statistical learning to embedded triplet sequences, which provides sufficient temporal resolution to measure neural entrainment, and additionally allows us to examine the neural sources that are most relevant for this entrainment. We tested participants’ learning of the tone triplets at the behavioral level by using a familiarity rating task and a speeded response time task, which assesses reaction times to individual tones presented in different positions within the triplets. We expected triplets to be rated as most familiar and reaction times to tones that occurred in later triplet positions to be faster as a result of increased predictability (Batterink and Paller, 2017), providing behavioral evidence of statistical learning.
Neural entrainment was quantified via the comparison of ITC at the triplet and tone frequency between two stimulation conditions, in which tones were organized in repeating triplets, or in pseudorandomized order (Batterink and Paller, 2017). If statistical learning is present, we expect ITC at the triplet frequency to be higher in the structured condition compared to the random condition, reflecting stronger neural phase-locking to the embedded triplets in the structured condition. In addition, this phase-locking value at the triplet frequency is expected to increase over the course of the exposure period, reflecting the progression of learning (Batterink and Paller, 2019). Furthermore, we aimed to elucidate the neural sources of the neural entrainment, providing further insight into the core substrates of statistical learning. We expected the superior temporal cortex to be the major hub (McNealy et al., 2006; Abla and Okanoya, 2008; Cunillera et al., 2009).
Lastly, we explored correlations between performance in the behavioral tasks and neural entrainment. We incorporated results at both the sensor and source level across different phases of exposure to gain insight into the relationship between neural entrainment, the progression of learning, and explicit knowledge. We hypothesized that stronger neural entrainment to tone triplets should predict better behavioral performance on the post-learning tests.
2. Material and Methods
2.1. Participants
Participants were 24 healthy volunteers (12 male) between 20 and 37 years old (mean age 27.54 years, SD=9.96). All participants were right handed and had normal hearing abilities. They received 10 € per hour for their participation. The local ethics committee of the Medical Faculty of the University of Tübingen approved the study (No. 231/2018BO1).
2.2. Materials and Design
Auditory stimulation consisted of 12 pure sinusoidal tones between 261.63 and 932.33Hz. The 12 tones corresponded to notes C, D, E, F#, G# and A# from the 4th and 5th octave of a standard piano (261.63Hz, 293.66Hz, 329.63Hz, 415.3Hz, 466.16Hz, 523.25Hz, 587.33Hz, 659.26Hz, 739.99Hz, 830.61Hz, 932.33Hz).The tones were organized into two different types of sequences, one so-called “structured” condition and one “random” condition (Figure 1). In both conditions, the 12 tones were grouped into four triplets of three tones; for better perceptibility, tones within a triplet never spanned more than one octave. In the structured condition, the same four triplets repeated over the course of a stimulation block. There were three different structured sequences, counterbalanced across participants, to control for the position of each individual tone within the triplet. Across the three counterbalanced sequences, each tone occurred in each triplet position (first, second or third). The random condition was also composed of triplets that did not span more than one octave, but apart from this constraint, the composition of the triplets was random and changed constantly throughout the exposure block. In both conditions, each tone occurred an equal number of times. Triplets were presented in pseudorandom order, with the constraint that neither the same tone, nor the same triplet, could repeat consecutively. Each sequence consisted of 2400 tones. Tones had a duration of 300 ms and were presented every 333 ms (i.e., with an inter-tone interval of 33 ms), yielding a total duration of 13.32 minutes per stimulation block.
Summary of the experimental protocol. A) Auditory stimulation and hypothesized neural entrainment effects, as measured by MEG. The notes in the top rows represent the different tones, with the curves in the two bottom rows showing the expected peak frequencies of neural entrainment. If statistical learning of the underlying triplets occurs in the structured condition, stronger neural entrainment at the triplet frequency (1.0Hz, grey dotted line) is expected relative to the random condition. B) Rating task performed after the exposure phase in the structured condition. Four triplets, four part-triplets and four non-triplets were rated on a scale from 1 (very unfamiliar) to 4 (very familiar). C) Target detection task performed after the rating task. Participants were presented with a target tone, which occurred four times within a stream of 36 triplets. Detection was indicated by a button press, and reaction time was measured.
Following the exposure phase, participants performed two behavioral tasks, designed to measure statistical learning of the structured sequence (cf. Batterink and Paller, 2017). In the rating task, participants were asked to rate 12 items for familiarity on a scale from 1 (very unfamiliar) to 4 (very familiar). Four of the items were triplets that were previously presented in the structured condition, four were part-triplets that consisted of a pair of tones from one of the triplets and a third tone that did not belong to the triplet, and four were non-triplets that were never presented in the structured block. In the target detection task, participants were asked to detect target tones that occurred within short tone sequences that followed the same triplet structure as the original structured stimulation block. For each trial, participants were presented with a target tone, followed by a short stream of tones. Each stream consisted of four repetitions of the four tone triplets from the structured condition (presented with a slightly longer inter-tone interval of 66 ms to facilitate the task), yielding a total of four targets per stream. A total of 36 streams were presented, with each tone serving as the target three times (48 syllables per triplet position). Target detection was indicated by a button press, with both speed and accuracy emphasized in the instruction.
2.3. Procedure
Data were recorded using a 275-sensor, whole-head MEG system (VSM Medtech, Port Coquitlam, Canada) installed in a magnetically shielded room (Vakuumschmelze, Hanau, Germany). Auditory stimulation was produced by a loudspeaker outside of the shielded room and conducted through tubes into the shielded room. Tones were presented through earplugs connected to the tubes at an intensity of 70dB.
After arrival at the MEG laboratory, participants were fully informed about the experimental procedure and signed a consent form to confirm their voluntary participation. They filled out a short questionnaire, and completed a short hearing assessment to confirm normal hearing. Participants then changed into metal-free clothes. They were seated in a height-adjustable chair and instructed to fixate on a cross displayed in front of them during the whole recording. Participants were told that they would hear sounds through their earplugs, but that they had no particular task to perform related to those sounds. The MEG signal was recorded continuously, with a sampling rate of 585.94Hz.
During the exposure phase, participants listened to the random condition first, followed by the structured condition. Two short breaks were included within each exposure block. After both exposure blocks, participants performed the rating task, followed by the target detection task. The target detection task was preceded by a short practice trial, with syllables rather than tones, to familiarize participants with the task. Both behavioral tasks were performed in the MEG chair to keep the environment consistent, but MEG was not recorded during these tasks. Finally, after both tasks were completed, participants were asked whether they heard any difference between the two exposure blocks.
2.4. Data Analysis
For analysis of the MEG data, MATLAB version 2016b (The MathWorks, Natick, MA) and the MATLAB-based open-source software package Fieldtrip (Oostenveld et al., 2011) were used.
2.4.1. Preprocessing
MEG data were band-pass filtered between 1-30Hz. Channels containing a high level of noise (overall root mean square (RMS) > 1 pT or no signal (overall RMS of 0.01 aT) were removed. Data were time-locked to the onset of each triplet and extracted into epochs containing 12 triplets (12000 ms), leading to epochs overlapping for 11/12 of their length (cf. Batterink and Paller, 2017). This process led to a total of 766 epochs for evaluation (800 triplets formed by the 2400 stimuli, resulting in 799 overlapping epochs, excluding 11 epochs per block near the end of a stimulation block to ensure an equal epoch length of 12000 ms). Epochs were corrected for a 32 ms sound output delay, and epochs containing artifacts with an amplitude over 4 pT were removed. This led to an average of 703.19 remaining epochs (SD=94.83).
2.4.2. Computation of inter-trial phase coherence (ITC)
We quantified neural entrainment at the tone and triplet frequencies by computing ITC across the overlapping epochs. ITC is a measure of event-related phase-locking, with ITC values ranging from 1, reflecting completely phase-locked activity, to 0, indicating activity that is completely phase random. To the extent that statistical learning occurs, we expected to observe higher phase-locking at the triplet frequency, reflecting greater neural entrainment to the underlying triplet structure (Figure 1). ITC was computed using a continuous Morlet wavelet transformation from 0.2-9.2Hz, centered at every second.
To examine the temporal trajectory of the ITC in a fine-grained way, as a neural index of learning over time, ITC values were calculated on “bundles” of 12 consecutive epochs (e.g., 1-12, 13-24, 25-36, …755-766). As ITC values show greater fluctuation when computed over fewer trials, the resulting time course was smoothed over 5 bundles and the first and last bundle omitted. This resulted in 61 bundles that represent the time course of ITC over the whole recording. In the case of epochs removed during artifact rejection that resulted in a lower number of bundles, remaining bundles were shifted to fit the temporal structure by comparing the mean time within a bundle to the mean time at that bundle position and performing a shift, in case this time was more fitting to the subsequent bundle position.
2.4.3. Source Localization
To examine neural entrainment effects at the source level, we used the Dynamic Imaging of Coherent Sources (DICS) approach (Gross et al., 2001). In this approach, we created a surrogate signal, consisting of the combination of a 3Hz (tone frequency) and 1Hz (triplet frequency) sinusoidal and added it as an extra channel to the data. This allowed us to calculated coherence of the MEG signal with the tone and the triplet frequencies at the source level. A fast Fourier transform was calculated to obtain data in the frequency domain, which was then used as input for the DICS source analysis model. A common spatial filter for the structured and random conditions was calculated on the appended data to make conditions comparable. To project the source data, we used the ‘fsaverage’ brain, which is an average brain mesh provided by Freesurfer (Fischl et al., 1999; Dale et al., 2012), as a template brain and used SUMA (Saad and Reynolds, 2012) processing to obtain a decimated standard white/gray matter boundary mesh from which we could calculate a volume conduction head model with the ‘single shell’ method (Nolte, 2003). The MEG data were coregistered to the template head via the three fiducial points (nasion, left and right preauricular points). We obtained 2004 cortical surface vertices that were further used as MEG sources (cf. Hegner et al., 2018). For naming detected brain areas, we used the Destrieux et al. (2010) atlas.
2.4.4. Statistical Testing
Statistical tests were performed using R (R Core Team, 2019) and SPSS (IBM, 2017).
For the rating task, familiarity ratings were analyzed using a repeated-measures ANOVA with category (Triplet, Part-Triplet, Non-Triplet) as a within-participant factor. A linear contrast was used to test whether familiarity ratings decreased linearly across the three categories. Following previous studies (Batterink & Paller, 2017, 2019), a “rating score” was calculated for each participant, by subtracting the average score given for part-triplets and non-triplets from the average score given for triplets.
For the target detection task, for each participant, mean reaction times to detected targets were calculated within each triplet position (first, second, and third). Responses that did not occur within 0-1200 ms of a target were considered to be false alarms. The number of correctly detected tones and false alarms was quantified within each participant to get an estimate of task performance. To examine the hypothesis that reaction times should decrease linearly as a function of triplet position, reaction times were analyzed using a repeated-measures ANOVA with triplet position (initial, medial, final) as a within-participants factor, using a linear contrast. In an exploratory step, number of misses were compared in the same fashion.
ITC differences between conditions (structured versus random) at the triplet and tone frequencies were tested with a two-sided student t-test. Development of ITC over time was tested with a mixed effects model using ITC value as the dependent variable, bundle as continuous predictor, and participant as a random effect. In a second step, interaction terms between sensor location (frontal, central, parietal, occipital, right temporal, left temporal) and bundle were added to the model to test for possible interaction effects. Statistical tests at the source level were performed in MATLAB with a cluster based permutation test (Maris and Oostenveld, 2007). Cluster threshold was set to α=0.05 for initial testing and to α=0.01 for detecting more specific regions of importance. Relations between ITC and the rating score as well as between differences in coherence with the surrogate signal at the triplet frequency on the source level and the rating score were explored using Pearson correlations.
The calculation of relationships between neural entrainment and behavioral outcomes were exploratory and therefore not corrected for multiple comparisons. Follow up analyses on the mixed model results, examining ITC over time, were also exploratory and not corrected for multiple comparisons. In all other cases, results were Bonferroni corrected for multiple comparisons, with significance values set to p<0.05.
3. Results
3.1. Behavioral Results
In a subjective report after the experiment, 7 out of 24 participants reported that they did not hear any difference between the two conditions.
In the rating task, participants rated triplets (M = 3.07, SE = 0.13) as more familiar compared to part-triplets (M = 2.91, SE = 0.12) and scrambled non-triplets (M = 2.79, SE = 0.12). This was shown by a significant linear effect (F(1,23) = 4.64, p = 0.042). The test for a category effect was only marginal (F(1,46) = 2.79, p = 0.075), showing that the effect of category on familiarity is small in magnitude for non-linguistic stimuli. The rating score was significantly above zero, providing evidence of above-chance learning (M = 0.23, SE = 0.10, t(23) = 2.15, p = 0.042). Sixteen participants showed a rating score. > 0 and were classified as “learners” in subsequent analyses, while the remaining eight participants with a rating score <= 0 were classified as “non-learners.”
In the target detection task, no significant effect of triplet position on reaction times was found (Triplet Position Effect: F(2,46) = 0.78, p = 0.44; linear contrast: F(1,23) = 0.64, p = 0.43). There was a significant effect of triplet position on accuracy, such that the number of misses (out of 48) increased with later triplet positions (Position Effect: F(2,46) = 16.2, p < 0.001; linear effect: F(1,23) = 36.7, p < 0.001). However, this effect was unexpected and in the opposite-to-predicted direction. Overall, participants performed poorly on the task, detecting only 63% of tones, compared to the syllable version of the task where 83-89% accuracy has been reported (Batterink & Paller, 2017, 2019). In addition, participants made an average of 102.5 false alarms (std = 49.4), which is much higher than in the linguistic version (∼12-19 false alarms, using the same total number of targets). Based on these results, we conclude that this version of target detection task, as implemented with tones rather than syllables, was too difficult to reveal significant learning effects. In the subsequent correlation analyses with MEG neural entrainment effects, only the rating score was used as a behavioral measure of learning.
3.2. Neural Entrainment at the Sensor Level
Across the exposure period and across all MEG channels, ITC at the triplet frequency was significantly higher in the structured compared to random condition (t(23) = 4.38, p < 0.001; Figure 2). ITC at the tone frequency did not differ between conditions. This significantly higher ITC in the triplet frequency in the structured compared to the random condition was confirmed for both participants classified as learners (t(15) = 3.36, p = 0.009) and non-learners (t(7) = 3.05, p = 0.037; Figure 2). When analyzed within each of the three blocks (separated by the short breaks), ITC at the triplet frequency significantly differed between conditions in blocks one and three (t(23) = 4.58, p < 0.001, and t(23) = 5.9, p < 0.001, respectively), but did not significantly differ in block two (t(23) = 1.84, p = 0.24; Figure 2).
Left: Inter-trial-coherence (ITC) at the whole brain level over the whole recording. Triplet frequency: 1Hz, Tone-Frequency: 3Hz. Significant difference was found between structured and random conditions at triplet frequency (** p<0.001). No difference was found at tone frequency. Right upper: ITC at triplet frequency over 3 recording blocks. Structured and random conditions differed significantly in blocks 1 and 3 (** p<0.001). Right lower: ITC at triplet frequency in random and structured blocks in learners (n = 16) and non-learners (n = 8). ITC differed significantly between conditions in both groups.
A fine-grained, bundle-by-bundle analysis of the time course of ITC at the triplet frequency in the structured condition revealed no significant overall effect of time across all sensors (F(1, 8006) = 2.09, p = 0.15). However, a significant sensor region x time effect was found, indicating that the effect of time on ITC differed across the scalp (F(5,8066) = 10.0, p < 0.001). Follow-up analyses indicated that ITC increased significantly over central sensors (F(1,1332) = 4.92, p = 0.027; parameter estimate = 0.000198, SE = 0.000089), and increased marginally over left temporal (F(1,1332) = 3.75, p = 0.053; parameter estimate = 0.000200, SE = 0.000103) and frontal sensors (F(1,1332) = 3.31, p = 0.069; parameter estimate = 0.000158, SE = 0.000087). In contrast, ITC at right temporal and parietal sensors did not significantly change over time (both p > 0.14), and unexpectedly significantly decreased over occipital sensors (F(1,1280) = 4.23, p = 0.040; parameter estimate = −0.000158, SE = 0.000077; Figure 3).
MEG index of learning over time in the structured condition, as captured by the time course of ITC at the triplet frequency, shown over different sensor locations (central, frontal, left temporal, right temporal, occipital, parietal) Each plot depicts an average over the sensor locations marked in red. * depicts significant change over time (increase or decrease), ∼ depicts marginally significant change.
3.3. Neural Entrainment at the Source Level
An initial test (cluster threshold α=0.05) revealed significant differences between the structured and random conditions in coherence with the surrogate signal at the triplet frequency (“triplet coherence”) across a wide range of cortical areas. When the test was repeated with a more stringent cluster threshold of α=0.01, clusters showing significant condition differences in triplet coherence were restricted to fewer areas, as shown in Figure 4A. The largest cluster spans from the right superior temporal lobe over the supramarginal and subcentral gyrus towards the inferior frontal gyrus (cluster 2). On the left side, this homologous cluster is smaller, consisting of the superior temporal gyrus and part of the inferior frontal gyrus (cluster 5). Additionally, the right inferior fronto-polar gyrus and superior parietal lobe and the left middle and superior frontal and precentral gyrus showed significant coherence differences between conditions. Structured and random conditions did not show any significant differences in coherence with the surrogate signal at the tone frequency, independent of applied cluster threshold (“tone coherence”).
A) Areas where triplet coherence significantly (cluster threshold α<0.01) differed between structured and random conditions. Color gradient represents t-values. B) Correlations between difference values obtained from clusters depicted in A and rating score, our main behavioral measure of learning. Top: correlation across all clusters; bottom: indication for correlation with individual clusters for the entire recording and the third recording block (*=significant positive correlation, ∼=marginally significant positive correlation)
3.4. Relationship Between Neural Entrainment and Behavior
3.4.1. Neural Entrainment at the Sensor Level
To examine the relation between neural entrainment at the sensor level, ITC values at the triplet and tone frequency were correlated with the rating score. Critically, across the whole recording sequence and all sensors, the rating score significantly correlated with ITC values at the triplet and tone frequency, in the structured condition only (r(22) = 0.44, p = 0.033; r(22) = 0.44, p = 0.033). Follow-up exploratory analyses examined ITC correlations at both the triplet and tone frequencies, within each of the three exposure blocks individually. Results are shown in Table 1. The general pattern of results suggests that there is a more consistent relation between neural entrainment and behavioral learning later in the exposure period (block 3), compared to in earlier blocks. In addition, whereas tone-frequency entrainment in both structured and random conditions predicted rating score (suggesting a non-specific learning effect), the correlation between triplet-frequency entrainment and rating score was significant only in the structured block.
3.4.2. Neural Entrainment at the Source Level
At the source level, the difference in triplet coherence between structured and random conditions, across all significant clusters (α=0.01; see Section 3.3) strongly correlated with the rating score (r(22) = 0.57, p = 0.004; Figure 4B). Follow-up tests of individual clusters revealed significant correlations in the right temporo-parietal (r(22) = 0.52, p = 0.01; cluster 2), right fronto-polar (r(22) = 0.51, p = 0.011; cluster 3) and left fronto-parietal cluster (r(22) = 0.45, p = 0.029; cluster 4).
Next, we applied this same analysis to the triplet coherence and tone coherence within each of the three recording blocks, using the clusters depicted in Figure 4A. Only the difference in triplet coherence from block 3 significantly correlated with the rating score (r(22) = 0.56, p = 0.004), mirroring the general pattern of sensor-level results reported in section 3.4.1. Regarding the individual clusters within this block, the right temporo-parietal (r(22) = 0.55, p = 0.006; cluster 2), left fronto-parietal (r(22) = 0.53, p = 0.008; cluster 4) and right superior parietal cluster (r(22) = 0.41, p = 0.045; cluster 1) showed significant correlations with the rating score (Figure 4B).
4. Discussion
The present study showed that participants were able to learn nonlinguistic statistical regularities simply through passive exposure to the stimulus stream, in the absence of explicit instructions to detect the patterns. Participants demonstrated learning at both the behavioral and neural levels. At the behavioral level, participants rated triplets from the stimulus stream as significantly more familiar than non-triplets. At the neural level, learning was evidenced by robust neural entrainment to the embedded triplets across all sensors. This triplet entrainment was significantly greater in the structured condition compared to the random condition and emerged early on during exposure. In addition, triplet entrainment significantly increased over the course of exposure, albeit weakly and in limited regions. Concerning the neural sources of these entrainment effects, a broad range of regions spanning temporal, frontal and parietal cortices showed sensitivity to the hidden structure, which is generally in line with previous literature. Neural entrainment in these identified clusters also robustly predicted subsequent behavioral performance, with this brain-behavior relationship becoming stronger later in exposure. These results suggest that successful neural entrainment forms a basis for subsequent expression of explicit forms of knowledge, with the increasingly strong relationship between neural entrainment and behavior over exposure reflecting the emergence of explicit forms of knowledge from earlier representations in implicit memory.
4.1. Behavioral Results
As a group, participants showed significant behavioral evidence of learning as measured on the rating task, even though 7 out of 24 participants claimed that there was no difference between random and structured blocks. Participants’ ratings distinguished among triplets, part-triplets and non-triplets, although the differences in ratings were small and variable across participants. Linguistic statistical learning studies using a similar rating task have shown stronger learning effects (e.g. Batterink and Paller, 2017, 2019), which may be related to facilitation of learning enabled by verbal encoding strategies (Siegelman et al., 2018a). In contrast, nonlinguistic auditory statistical learning studies have reported both stronger (Abla et al., 2008; Abla and Okanoya, 2008; Gebhart et al., 2009) and weaker, chance-level behavioral learning effects (Paraskevopoulos et al., 2012; Farthouat et al., 2017). These previous studies used a 2AFC task to test participants’ learning abilities, testing the contrast between triplets and either non-triplets (Abla et al., 2008; Abla and Okanoya, 2008) or part-triplets (Gebhart et al., 2009; Paraskevopoulos et al., 2012; Farthouat et al., 2017). Additionally, exposure time to the structured stream varied from 10 min (Farthouat et al., 2017) to 40 min (Gebhart et al., 2009). These variable results across studies emphasize that task difficulty (differentiating triplets from part-triplets or non-triplets) as well as exposure time likely play a crucial role in the recognition of tone structures. Therefore, even if a neural signature is visible, the ability to explicitly express knowledge acquired during statistical learning is not present automatically.
Effects in the target detection task were counterintuitive, showing a lower number of misses for the first tone in a triplet. Based on previous studies using the same task with language stimuli (e.g. Batterink et al., 2015b; Batterink et al., 2015a; Batterink and Paller, 2017, 2019) and abstract shapes (Turk-Browne et al., 2005; Kim et al., 2009), we had expected a facilitation for the third tone of a triplet, as it is most predictable. However, the low task accuracy, as well as subjective reports from participants, suggest that the task was too difficult to reveal expected learning effects. Moreover, memory for the specific tones may have been too weak to drive expectations for the third stimulus and support faster responding, as occurred in prior statistical learning studies with more complex stimuli. We speculate that the higher accuracy for the first tone of a triplet could have been caused by improved memory or improved perceptual separation for these less predictable tones, which could have in turn facilitated recognition. As the positions of individual tones in the structured condition were counterbalanced across participants, better accuracy for first tones cannot be due to stimulus-specific properties of certain tones.
4.2. Neural Entrainment
Neural entrainment to triplets was significantly enhanced in the structured compared to random tone streams. This condition difference was visible at the whole brain level and already highly significant within the first 5 minutes of exposure. Interestingly, higher triplet-frequency entrainment in the structured compared to the random condition did not depend on above-chance behavioral performance, as it was present over all participants, including those classified as non-learners. This finding suggests that these neural entrainment effects reflect a robust, perhaps compulsory neural response to repetitive structure in an input stream.
In the structured stream, ITC at the triplet frequency increased over the exposure period over central MEG sensors, though this increase was not strong enough to reach significance at the whole brain level. The observed increase over central MEG sensors fits with previous neural entrainment results on linguistic statistical learning, where an increasing trajectory was reported over centro-frontal midline EEG electrodes (Batterink and Paller, 2017, 2019). Across studies, this increase in neural entrainment to the underlying triplets may reflect a shift in perception and encoding from individual stimulus units to more integrated items, which occurs gradually over the course of exposure. Unexpectedly, in the present study, sensors over the occipital cortex showed a significant decrease over time, which could possibly reflect disengagement of regions that are not involved in the statistical learning task.
An advantage over previous EEG-based neural entrainment studies of statistical learning (e.g., Batterink and Paller, 2017, 2019; Buiatti et al., 2009) is our use of MEG, which allowed us to localize specific cortical regions involved in learning. Overall, results are in line with previous statistical learning studies, while also revealing additional brain areas that have not been previously discussed in this context. Our localization analyses revealed a network of regions spanning the superior temporal, inferior frontal and parietal cortex that showed robust differences in triplet coherence between the two conditions (Figure 4A). These regions fit well with previous literature (McNealy et al., 2006; Abla and Okanoya, 2008; Cunillera et al., 2009; Farthouat et al., 2017). In both hemispheres, the clusters that span the superior temporal to the inferior frontal cortex reflect the auditory processing hierarchy, and include Heschl’s gyrus, which is known to be important in pitch and melody perception (e.g. Patterson et al., 2002; Schneider et al., 2005). This most likely reflects a domain-specific component of auditory statistical learning, and suggests that even regions involved in early sensory processing may be sensitive to statistical regularities. The cluster spanning the precentral gyrus towards the superior frontal gyrus is in line with findings by Cunillera et al. (2009) and Farthouat et al. (2017), who found the premotor cortex to be important for statistical learning. Based on these results, the authors argued for an audio-motor interface theory of speech learning, wherein speech perception and production are linked and thereby facilitate learning (Cunillera et al., 2009). This theory may possibly also be applied to learning of tone sequences (Farthouat et al., 2017) as not only the memory for words but also the memory for tone patterns could be mediated by the phonological working memory loop (Baddeley et al., 1998). Interestingly, the left lateralization of this precentral cluster, which Cunillera et al. (2009) connected to speech production, also holds for our nonlinguistic study as well as that of Farthouat and colleagues’, suggesting that this left-lateralization is not strictly specific to linguistic processing.
The two clusters that we detected in the orbitofrontal and superior parietal cortex have not been reported in prior statistical learning literature. The orbitofrontal cortex is known to play a role in impulse control, especially studied in the context of drug addiction (e.g. London et al., 2000; Goldstein et al., 2001), reinforcement learning (e.g. Rolls, 1996) and decision making (Bechara et al., 2000). The current study used a passive learning paradigm and therefore did not include any overt task that could directly link statistical learning to the aforementioned prefrontal cognitive control processes. Nevertheless, given that the orbitofrontal cortex is involved in a variety of processes related to learning and working memory, it is plausible that it also plays a role in statistical learning. The superior parietal gyrus has been implicated in attention shifting (Vandenberghe et al., 2001). As clusters reflect differences in triplet coherence between structured and random conditions, a possible explanation for the involvement of the superior parietal gyrus could be an increase in attention to the triplets, once participants detected the regularity in the structured condition. The lack of regularity in the random condition could lead to decreased attention.
We observed bilateral clusters along the auditory processing stream, whereas some previous studies found left hemispheric lateralization in these regions (McNealy et al., 2006; Abla and Okanoya, 2008; Karuza et al., 2013). However, both McNealy et al. (2006) and Karuza et al. (2013) used linguistic stimuli, which may be expected to result in greater left-lateralization compared to our nonlinguistic paradigm. Regarding lateralization for non-linguistic stimuli, previous results are inconsistent. Abla and Okanoya (2008) used a similar tone paradigm in a fNIRS study and reported a left-lateralization. In contrast, Farthouat et al. (2017) did not find a clear lateralization of statistical learning effects using MEG, and Janacsek et al. (2018) even emphasized the role of the right hemisphere, applying transcranial direct current stimulation during statistical learning. In sum, results of our study together with previous literature do not support the assumption of a lateralization of statistical learning per se. It is more likely that lateralization is a result of earlier sensory processing, influenced by acoustic features (e.g., high temporal resolution of speech and high spectral resolution of musical tones) as described by Zatorre and colleagues (2002).
4.3. Relationship Between Neural Entrainment and Behavior
Interestingly and as predicted, participants’ neural entrainment measured during the structured exposure block strongly correlated with their behavioral learning outcomes, as measured by performance on the rating task. Even though neural entrainment was also present in participants who were not able to recognize the learned triplets, its strength predicted later performance on the behavioral task. This correlation suggests that neural entrainment is at least partially related to the emergence of explicit knowledge, with successful neural entrainment perhaps forming a basis for later, explicit forms of knowledge. This brain-behavior correlation was observed at both the sensor and source level—that is, for triplet-ITC across all electrodes and for triplet coherence in the brain areas depicted in Figure 4A. At the source level, this relationship was most pronounced in temporo-frontal areas in the right hemisphere and fronto-parietal areas in the left hemisphere. The correlation with entrainment in temporal regions implicates domain-specific mechanisms, and suggests that areas involved in auditory processing also play a role in the formation of more consciously available representations. The spread of this cluster towards the frontal cortex suggests the additional engagement of higher cognitive functions in this process. The correlation with precentral regions points towards a possible role of the previously described audio-motor interface not only in neural entrainment of statistical regularities but also in the formation of explicit knowledge from this implicitly available information. Cunillera et al. (2009) used this framework in the context of speech acquisition. Its role in the context of our nonlinguistic paradigm suggests that the involvement of the motor system is not language specific but rather general to an auditory statistical learning task.
In the present study, the coupling between neural entrainment and behavior became stronger over exposure, reflecting the emergence of explicit forms of knowledge from earlier representations in implicit memory. As pointed out in the discussion of our behavioral results, prior results suggest that length of exposure may critically influence whether participants are eventually able to explicitly recognize the tone sequences. Our ITC results indicate that neural entrainment happens very fast, but the results cannot pinpoint exactly when learning at the neural level is transformed into explicit, consciously accessible knowledge. Nonetheless, our finding that the degree of neural entrainment in the last third of exposure correlated most strongly with explicit behavioral knowledge suggests that at least 10 minutes of exposure are necessary for the emergence of explicit representation of the underlying structure, at least in the case of this nonlinguistic auditory paradigm. Altogether, these results point to rapid sensitivity of specific cortical brain regions to hidden statistical structure, followed by the gradual emergence of knowledge that can be expressed explicitly, which occurs in a majority of participants but not all.
The variability between participants’ performance in the behavioral task shows that exposure time is most likely not the only factor crucial for forming explicit representations. In addition to ITC at the triplet frequency in the structured condition, ITC at the tone frequency in both the structured (overall) and random condition (block 3) also correlates with the rating score (Table 1). Given that a higher ITC at the tone frequency (independent of condition) may reflect greater attention to the presented sounds, this correlation with tone frequency suggests an effect of general attention on learning. It may be that there is a minimum degree of attention necessary to support behavioral learning, and that participants classified as non-learners may have allocated insufficient attention to the speech stream to achieve demonstrable explicit knowledge.
Taken together, these results suggest that neural entrainment reflects a basic, implicit form of learning that is observed from very early on during exposure. In contrast, the process of building up on this implicit learning to make learned structures explicitly available requires both sufficient exposure and a certain degree of attention.
4.4. Limitations and Future Directions
The current study used a rather short exposure time, and it is not clear whether a longer exposure time would allow all participants to achieve explicit knowledge of the underlying structure. In addition, we did not control for the attentional level of the participants, so we are not able to determine the influence of attentional control on the process. To test the role of exposure time and attention on statistical learning, future studies that systematically manipulate these variables – like the study by Batterink and Paller (2019), who compared full and divided attention – will be necessary. Gaining further insights into the dynamic transition from an implicit, neural representation of structure towards an explicitly available one may be used to optimize learning strategies, both in the auditory domain as well as in other modalities. Furthermore, these results could lead to the development of new strategies for individuals with learning impairments.
5. Conclusions
MEG is well suited to detect nonlinguistic statistical learning at both the sensor and source level. Combined with behavioral data, these results shed light on different aspects of statistical learning. The present study showed that a wide range of brain areas are involved in this process of learning, not only those primarily related to auditory functions. It also provided evidence that even if effects on the neural level were rather strong, the more explicit expression of this kind of nonlinguistic learning is difficult for participants. Nevertheless, these two components are tightly linked, as shown by the correlation results. The nature and timing of these correlations suggest that both sufficient exposure time and a certain degree of attention are necessary to build on the implicit encoding of an underlying structure represented by neural entrainment and to form explicitly available knowledge.
7. Author Contributions
Julia Moser: Conceptualization, investigation, methodology, formal analysis and writing – original draft. Laura Batterink: Conceptualization, methodology, formal analysis and writing – original draft. Yiwel Li Hegner: Methodology and formal analysis. Franziska Schleger: Conceptualization, methodology and writing – review & editing. Christoph Braun: Methodology and supervision. Ken Paller: Conceptualization and writing – review & editing. Hubert Preissl: Supervision, conceptualization and writing – review & editing. All authors read and approved the final version of the manuscript.
6. Acknowledgements
We want to thank Jürgen Dax for his support with the technical setup. This work was partially funded by the FET Open Luminous project (H2020 FETOPEN-2014-2015-RIA under agreement No. 686764) as part of the European Union’s Horizon 2020 research and training program 2014–2018 and the German Federal Ministry of Education and Research (BMBF) to the German Center for Diabetes Research (DZD e.V. 01GI0925).
8. References
