Abstract
Examining how young infants respond to unexpected events, is key to our understanding of their emerging concepts about the world around them. Infants reliably show increased attention towards unexpected (i.e., unpredicted) events, which allows them to refine predictive models about their environment. Yet, the neural processing of prediction errors in the infant brain is not well understood. Here, we presented 9-month-olds (N = 36) a series of physical and social events with unexpected versus expected outcomes, while recording their electroencephalogram. We found a pronounced 4 – 5 Hz theta response for the processing of unexpected (in contrast to expected) events, for a prolonged time window (2 s) and across all scalp-recorded electrodes. These findings constitute critical evidence that the theta rhythm is involved in the processing of prediction errors from very early in human brain development, supporting infants’ refinement of basic concepts about the physical and social environment.
From early on, human infants develop basic concepts about their physical and social environment (Spelke and Kinzler, 2007). This includes a basic understanding of numbers (Wynn, 1992), the properties of objects (Baillargeon et al., 1985; Spelke et al., 1992), and others’ actions (Gergely et al., 2002; Reid et al., 2009).
Our understanding of infants’ early concepts of their environment is based on violation of expectation (VOE) paradigms. In VOE paradigms infants are shown unexpected events, which violate their basic concepts, in contrast to expected events. For example, infants are shown a change in the number of objects behind an occluder (Wynn, 1992), a ball falling through a table (Spelke et al., 1992), or an unusual human action (Reid et al., 2009). These unexpected events (in contrast to expected events) commonly increase infants’ attention, indicated by longer looking times, and motivate infants to learn about their environment, indexed by an increased exploration and hypothesis testing of objects that behaved unexpectedly (Stahl and Feigenson, 2015).
Recently, infants’ enhanced attention for unexpected events and their subsequently piqued curiosity and exploration behavior in VOE paradigms (Stahl and Feigenson, 2015), have been interpreted in light of a predictive processing perspective on infants’ brain development and learning (Köster et al., 2020). By this token, events that violate basic expectations elicit a prediction error and require infants to refine their specific predictions about this physical and social event. In the present study, we applied our understanding of infants’ basic concepts, as assessed in VOE paradigms, to test the neural brain dynamics involved in prediction error processing in the infant brain.
Infants’ neural processing of unexpected events has been investigated in terms of time-locked neural responses (i.e., event-related potentials; ERPs) in the scalp-recorded electroencephalogram (EEG). This research has centered around the negative central (NC), a negative component that emerges around 400-600 ms after stimulus onset at central recording sites, and which has been associated with attention processes (for a review, see Reynolds, 2015). A first study looked at the spectral properties of the NC component in the ERP and linked this component to a broad increase in the 1 – 10 Hz frequency range in infants and adults (Berger et al., 2006). Interestingly, unexpected events have been associated with an increased NC (Kayhan et al., 2019; Langeloh et al., 2020; Reynolds and Richards, 2005; Webb et al., 2005) as well as a reduced NC (Kaduk et al., 2016; Reid et al., 2009), when contrasted to the brain activity elicited by expected events. Consequently, the mechanisms reflected in the NC, involved in the processing of unexpected versus expected events, are not entirely understood.
In a recent study, infants’ neural oscillatory dynamics were rhythmically entrained at 4 Hz or 6 Hz, and the presentation of unexpected events led to a specific increase in the entrained 4 Hz but not in the 6 Hz activity (Köster et al., 2019). Critically, 4 Hz oscillatory activity corresponds to the neural theta rhythm, a frequency which plays an essential role in prediction error processing in adults (Cavanagh and Frank, 2014) as well as learning processes in adults (Friese et al., 2013; Köster et al., 2018), children (Köster et al., 2017), and infants (Begus et al., 2015; Begus and Bonawitz, 2020). However, it has not been investigated how the ongoing oscillatory activity (i.e., not entrained or tightly locked to the stimulus onset) responds to unexpected events in the infant brain and, specifically, whether the ongoing theta rhythm marks infants’ processing of prediction errors.
Here, we tested infants’ neural processing of prediction errors, by presenting them a series of different physical and social events with expected versus unexpected outcomes for various domains, from physics about objects to numbers and actions (see Figure 1), while recording their EEG. Based on previous ERP studies in infants, we expected a differential NC response (400 – 600 ms, at central electrodes) for expected versus unexpected events.
Examples for the violation of expectation events presented to the participants. Infants saw the events of four domains of basic knowledge (action, solidity, cohesion, and number). In each trial, the first two pictures initiated an event (prior; for 1 s each) and the third picture showed the outcome (for 2 s), which could be expected or unexpected. (Faces are obscured for the upload to bioRvix.)
Furthermore, because of its pivotal role in prediction error processing in adults, we expected higher 4 Hz theta activity for unexpected versus expected events.
Results
Infants’ event-related responses upon the onset of the outcome picture revealed a clear NC component between 400 – 600 ms over central electrodes. The NC was more pronounced for expected in contrast to unexpected events, t(35) = −2.62, p = .013 (Figure 2).
The topography and time course of the NC for the outcome pictures. (A) The difference between unexpected and expected events for 400 – 600 ms, in contrast to a −100 – 0 ms baseline. (B) The corresponding time course at central electrodes (Cz, C3, C4), with a significant difference between 400 – 600 ms, p = .013.
Furthermore, across all scalp recorded electrodes and the whole 0 – 2000 s time window, we observed an increase in neural oscillatory activity in the 4 – 6 Hz range for unexpected events and an increase at 6 Hz for expected events (Figure 3), t(35) = 4.77, p < .001, and, t(35) = 4.01, p < .001, with regard to the baseline. This resulted in higher 4 – 5 Hz activity for unexpected compared to expected events across all scalp-recorded electrodes and throughout the whole 0 – 2000 s time-window, t(35) = −2.33, p = .025.
The grand mean spectral characteristics for unexpected versus expected events for the outcome picture. (A) The left panels show the time-frequency response across all scalp-recorded electrodes for unexpected and expected events and the difference (unexpected – expected), with regard to a −100 – 0 ms baseline. The right panels show the frequency response between 2 and 15 Hz, averaged over time. The dotted line highlights the activity at 4 Hz, which was expected to increase for unexpected events (Köster et al., 2019). (B) The topography shows the unexpected - expected difference in 4 – 5 Hz activity across the whole time window of analysis (0 – 2000 ms, baseline: −100 – 0 ms). (C) The corresponding time course for the 4 – 5 Hz response across all scalp-electrodes and the whole 0 – 2000 ms time window shows a significant difference between unexpected versus expected events, p = .025.
To get an impression about the consistency of the differences in the central Nc and the 4 – 5 Hz activity across knowledge domains, we also plotted the data split by domains (action, solidity, continuity, number). The ERP effect was somewhat consistent across conditions, but the effect was mainly driven by the differences between expected and unexpected events in the action and the number domain (Figure S1). The results were more consistent across domains for the condition difference in the 4 – 5 Hz activity, with a peak in the unexpected – expected difference falling in the 4 – 5 Hz range across all electrodes (Figure S2). We did not test these domain-specific differences statistically due to the low trial numbers within each domain and the main focus of the study being on generalized prediction error processing in the infant brain.
Discussion
Our results show a clear increase in 4 – 5 Hz power in response to unexpected events, in contrast to expected events. This effect was distributed across all scalp-recorded electrodes and for a prolonged time window of 2 s after the onset of unexpected outcome pictures. Thus, the theta rhythm was substantially increased for the processing of prediction errors in the infant brain. Furthermore, in the ERP response we found a stronger NC for expected events, in contrast to unexpected events, at central electrodes.
In the brain of human adults, the theta rhythm has long been associated with the processing of prediction errors (Cavanagh et al., 2010), cognitive conflict (Hanslmayr et al., 2008), and mnemonic control (Friese et al., 2013; Köster et al., 2018). Furthermore, the theta rhythm has been associated with learning processes in children (Köster et al., 2017), and infants (Begus et al., 2015; Begus and Bonawitz, 2020). Our findings highlight that the theta rhythm promotes the processing of novel, unexpected information, in the sense of prediction errors, already in early infancy. This is particularly interesting because the theta rhythm is usually associated with neural processes in prefrontal and medio-temporal structures, which are still immature in the infant brain (Gilmore et al., 2012).
Embedding the role of the theta rhythm in a broader theoretical framework, from animal models we know that the theta rhythm promotes predictive processes (i.e., such as the activation of future locations in a labyrinth; O’Keefe and Recce, 1993) and facilitates Hebbian learning (Tort et al., 2009). Based on these findings, the theta rhythm has been described as a neural code for the sequential representation and the integration of novel information into existing concepts (Lisman and Jensen, 2013). We would like to add to this that the theta rhythm may implement a computational mechanism that compresses real time events onto a faster neural time-scale, to advance with cognitive processes ahead of real time and to facilitate the integration of new events into existing networks. This is critical to predict future events and integrate novel events as they happen in real time. While former studies have demonstrated that this computational mechanism may be phylogenetically preserved in the mammalian linage (Cavanagh and Frank, 2014; Lisman and Jensen, 2013), here we report first evidence that the ongoing theta rhythm supports the processing of unexpected events already from very early in human ontogeny.
We also identified differences between unexpected and expected events in the NC, a classical visual ERP component associated with infants’ processing of unexpected events. However, this difference lasted around 200 ms (400 – 600 ms), was specific to central electrodes (Cz, C3, C4), and pointed in the opposite direction than most (Kayhan et al., 2019; Langeloh et al., 2020; Reynolds and Richards, 2005; Webb et al., 2005), though not all (Kaduk et al., 2016; Reid et al., 2009), effects previously reported in the NC (namely, the more common findings of a higher negativity for unexpected events; Kayhan et al., 2019; Reynolds and Richards, 2005; Webb et al., 2005). It is currently not clear, why unexpected events induce enhanced NC amplitudes in some studies, but a decreased NC compared to expected events in others. Because the amplitude of the NC has been associated with the extent of attentional engagement with a visual stimulus (Reynolds, 2015; Reynolds and Richards, 2005), in our study infants’ initial orienting response may have been more pronounced for the more familiar and expected outcomes. This is in line with previous studies using partly similar stimuli (in particular the action events; Kaduk et al., 2016; Reid et al., 2009) and with the notion that infants show familiarity preferences (i.e., the preference for events consistent with their experience) when they are still in the process of building stable cognitive representations of their environment (Nordt et al., 2016).
To conclude, our findings make a strong case that the theta rhythm is present from very early in ontogeny, associated with the processing of prediction errors and, putatively, the refinement of the emerging concepts of the physical and social environment. This marks an essential step towards a better understanding of the neural oscillatory dynamics that underlie infants’ brain development and their emerging models of the world around them.
Materials and Methods
Participants
The final sample consisted of 36 9-month-old infants (17 girls, M = 9.7 months, SD = 0.5 months). Participants were healthy full-term infants, from Leipzig, Germany. Informed written consent was obtained from each participant’s parent before the experiment and the experimental procedure was approved by the local ethics committee. Thirteen additional infants were tested but excluded from the final sample, due to fussiness (n = 2) or because fewer than 10 artifact-free trials remained in each condition (n = 11). This attrition rate is rather low for visual EEG studies with infants (Stets et al., 2012). We selected this age group, because previous studies indicated VOE responses for the domains tested here by the age of 9 months or even earlier (Reid et al., 2009; Spelke et al., 1992; Wynn, 1992).
Stimuli and Procedure
Stimuli were based on four classical VOE paradigms for the core knowledge domains action, number, solidity, and cohesion in four variations each (Figure 1 and Figure S1, for the complete stimulus set). Each sequence consisted of three static images which depicted a scenario with a clearly expectable outcome.
In a within-subjects design, each of the 16 sequences was presented two times in each condition (expected or unexpected). This resulted in a total of 64 distinct trials, presented in 16 blocks. The order of the core knowledge domains, outcomes and the specific stimulus variations (four in each domain) were counterbalanced between blocks and across infants.
Every trial began with an attention grabber (a yellow duck with a sound, 1 s), followed by a black screen (variable duration of .5 – .7 s) and the three stimulus pictures (4 s). The first two pictures showed the initiation of an event or action (0 – 2 s, 1 s each picture), followed by the picture presenting the expected or the unexpected outcome (see Figure 1). Note that the final picture was actually presented for 5s, for a companion study that tested infants’ gaze behavior (recorded by an eye-tracker). However, for the present study we included all trials in which infants looked to the screen for at least 2 s of the final picture, coded from video (see below). The stimuli showing the outcome, namely the expected or unexpected outcome were counterbalanced in the case of the cohesion and the number stimuli (i.e., in the cohesion sequences outcome stimuli showed connected or unconnected objects and for number sequences the outcome showed one or two objects) and were matched in terms of luminance and contrast in the case of the action and solidity stimuli (all ps > .30). Stimuli were presented via Psychtoolbox (version 0.20170103) in Matlab (version 9.1). The full set of the original stimuli can be downloaded from the supplemental material of (Köster et al., 2019).
Infants sat on their parent’s lap at a viewing distance of about 60 cm from the stimulus monitor. Sequences were presented at the center of a 17-inch CRT screen at a visual angle of approximately 15.0° × 15.0° for the focal event. We presented all 64 trials, but the session ended earlier when the infant no longer attended to the screen. A video-recording of the infant was used to exclude trials in which infants did not watch the first 4 s of a trial. Gaze behavior was coded offline.
Electroencephalogram (EEG)
Apparatus
The EEG was recorded continuously with 30 Ag/AgCl ring electrodes from 30 scalp locations of the 10-20-system in a shielded cabin. Data were recorded with a Twente Medical Systems 32-channel REFA amplifier at a sampling rate of 500 Hz. Horizontal and vertical electrooculograms were recorded bipolarly. The vertex (Cz) served as an online reference. Impedances were controlled at the beginning of the experiment, aiming for impedances below 10 kΩ.
Preprocessing
EEG data were preprocessed and analyzed in MATLAB (Version R2017b). EEG signals were band-pass filtered from 0.2 Hz to 110 Hz and segmented into epochs from −1.5 to 3 s, around to the onset of the outcome picture. Trials in which infants did not watch the complete 4 s sequence (2 s during the initiation of the event and 2 s of the outcome picture) were excluded from the analyses. Furthermore, noisy trials were identified visually and discarded (approx. 10 % of all trials) and up to three noisy electrodes were interpolated based on spherical information. Eye-blinks and muscle artifacts were detected using an independent component procedure (ICA) and removed after visual inspection. To avoid any bias in the ICA removal, the ICAs were determined and removed across the whole data set, including all experimental conditions (both frequencies, both outcome conditions, all stimulus categories). Prior to the analyses, the EEG was re-referenced to the average of the scalp electrodes (Fz, F3, F4, F7, F8, FC5, FC6, Cz, C3, C4, T7, T8, CP5, CP6, Pz, P3, P4, P7, P8, Oz, O1, O2). Infants with a minimum of 10 artifact-free trials in each condition were included in the statistical analyses. Twenty-two to 52 trials (M = 32.2, SD = 7.3) remained for the infants in the final sample, with no significant differences in the number of trials between conditions (expected, unexpected), t(35) = 0.63 p = .530. We also plotted the data split by conditions, on subsamples with at least one trial for both the expected and the unexpected outcome condition. The respective size of subsamples and number of trials were action: n = 35, M = 10.3, SD = 3.2, solidity: n = 35, M = 6.9, SD = 2.7, cohesion: n = 32, M = 6.1, SD = 3.6, and number: n = 36, M = 8.5, SD = 3.0.
ERP Analysis
For the analyses of event-related potentials (ERPs), we averaged the neural activity, separately for the trials of both conditions (expected, unexpected). We focused on the NC as a classical component associated with infants’ processing of expected versus unexpected events (Reynolds, 2015). Specifically, we averaged the ERPs across central electrodes (Cz, C3, C4), and between 400 – 600 ms, with regard to a −100 – 0 ms baseline. The ERP power was averaged for each participant and condition and the power between expected and unexpected trials was then contrasted by means of a dependent t-test. We band-pass filtered the ERPs from 0.2 – 30 Hz for displaying purposes.
Spectral Analysis
To obtain the trial-wise spectral activity elicited by the outcome pictures we subjected each trial to a complex Morlet’s wavelets analysis (Morlet parameter m = 7, at a resolution of 0.5 Hz). We then averaged the spectral power across trials, separately for conditions (expected, unexpected). We focused on the frequencies from 2 to 15 Hz across the whole analyzed time window 0 – 2000 ms, with regard to a −100 – 0 ms baseline, to make the results directly comparable to the ERP analysis in this and former studies. We did not analyze higher frequencies due to muscle and ocular artifacts in the infant EEG (e.g., Köster, 2016).
Because this was the first study to look at the trial-wise neural oscillatory response to a series of unexpected versus expected events (i.e., not tightly locked to the stimulus onset; cf. Berger et al., 2006), in a first step, we looked at the grand mean spectral activity, separated by conditions (unexpected, expected), and the difference between both conditions (unexpected - expected). Conservatively, we analyzed the neural oscillatory activity averaged across the whole time-range of the outcome stimulus (0 – 2000 ms) and all scalp electrodes (Fz, F3, F4, F7, F8, FC5, FC6, Cz, C3, C4, T7, T8, CP5, CP6, Pz, P3, P4, P7, P8, Oz, O1, O2). While our initial proposal was to look at the difference in the 4 Hz theta rhythm between conditions (Köster et al., 2019), we found the strongest difference between 4 – 5 Hz (see lower panel of Figure 3). Therefore, and because this is the first study of this kind, we analyzed this frequency range.
Conflict of interest
There are no conflicts of interest.
Funding Information
This research was supported by a Max Planck Research Group awarded to SH by the Max Planck Society.
Acknowledgements
We would like to thank Carl Bartl und Ulrike Barth for their support with the data assessment and coding.
