Abstract
Examining how young infants respond to unexpected events is key to our understanding of their emerging concepts about the world around them. From a predictive processing perspective, it is intriguing to investigate how the infant brain responds to unexpected events (i.e., prediction errors), because they require infants to refine their predictive models about the environment. Here, to better understand prediction error processes in the infant brain, we presented 9-month-olds (N = 36) a variety of physical and social events with unexpected versus expected outcomes, while recording their electroencephalogram. We found a pronounced response in the ongoing 4 – 5 Hz theta rhythm for the processing of unexpected (in contrast to expected) events, for a prolonged time window (2 s) and across all scalp-recorded electrodes. The condition difference in the theta rhythm was not related to the condition difference in infants’ event-related activity on the negative central (Nc) component (.4 – .6 s), which has been described in former studies. These findings constitute critical evidence that the theta rhythm is involved in the processing of prediction errors from very early in human brain development, which may support infants’ refinement of basic concepts about the physical and social environment.
From early on, human infants develop basic concepts about their physical and social environment (Spelke and Kinzler, 2007). This includes a basic understanding of numbers (Wynn, 1992), the properties of objects (Baillargeon et al., 1985; Spelke et al., 1992), and others’ actions (Gergely et al., 2002; Reid et al., 2009). Taking a predictive processing perspective on infant brain development (Köster et al., 2020), it has been argued that infants develop basic concepts by forming predictive models about their environment and draw inferences about physical and social events. To optimize their predictive models about the environment and to reduce uncertainties, infants are thought to continuously integrate novel and unexpected information in response to prediction errors (Köster et al., 2020; cf. Clark, 2013; Friston, 2011; Schubotz, 2015). Yet, prediction error processes in the infant brain are not fully understood.
Our understanding of infants’ early concepts about their environment is based, to a large extent, on violation of expectation (VOE) paradigms. In VOE paradigms infants are shown unexpected events, which violate their basic concepts, in contrast to expected events. For example, infants are shown a change in the number of objects behind an occluder (Wynn, 1992), a ball falling through a table (Spelke et al., 1992), or an unusual human action (Reid et al., 2009). These unexpected events (in contrast to expected events) commonly increase infants’ attention, indicated by longer looking times, and motivate infants to learn about their environment, indexed by an increased exploration and hypothesis testing of objects that behaved unexpectedly (Stahl and Feigenson, 2015). From a predictive processing point of view, the response to VOE events corresponds to the processing of prediction errors: events that violate basic expectations elicit a prediction error and require infants to refine their predictions (Köster et al., 2020). Therefore, the neural response in VOE paradigms is highly suitable to investigate the neural brain dynamics involved in prediction error processing in the infant brain.
Infants’ neural processing of unexpected events has formerly been investigated in terms of evoked neural responses (i.e., event-related potentials; ERPs) in the scalp-recorded electroencephalogram (EEG). This research has centered around the negative component (Nc), which emerges around 400-600 ms after stimulus onset at central recording sites, and which has been associated with attention processes (for a review, see Reynolds, 2015). However, unexpected events have been associated with an increased Nc (Kayhan et al., 2019; Langeloh et al., 2020; Reynolds and Richards, 2005; Webb et al., 2005) as well as a reduced Nc (Kaduk et al., 2016; Reid et al., 2009), when contrasted to the brain activity elicited by expected events. Therefore, the neural mechanisms reflected in the Nc are not yet entirely understood. Former studies have also investigated the spectral properties of the Nc component and linked this component to an increase in 1 – 10 Hz activity in infants and adults (Berger et al., 2006) or the 4 – 7 Hz activity for toddlers and adults (Conejero et al., 2018).
In a recent study, infants’ neural oscillatory dynamics were rhythmically entrained at 4 Hz or 6 Hz, and the presentation of unexpected events led to a specific increase in the entrained 4 Hz but not in the 6 Hz activity (Köster et al., 2019). Critically, 4 Hz oscillatory activity corresponds to the neural theta rhythm, a frequency which plays an essential role in prediction error processing in adults (Cavanagh and Frank, 2014) as well as learning processes in adults (Friese et al., 2013; Köster et al., 2018), children (Köster et al., 2017), and infants (Begus et al., 2015; Begus and Bonawitz, 2020). However, it has not been investigated how the ongoing oscillatory activity (i.e., not entrained or evoked upon stimulus onset) responds to unexpected events in the infant brain and, specifically, whether the ongoing theta rhythm marks infants’ processing of prediction errors. It is critical to understand the ongoing theta dynamics because they are fundamentally different from evoked oscillatory responses (Tallon-baudry and Bertrand, 1999) and play a critical role in mnemonic processes and the integration of novel information into existing representations in particular (Friese et al., 2013; Hanslmayr et al., 2009; Klimesch et al., 1997; Köster et al., 2018; Osipova et al., 2006).
Here, we tested infants’ neural processing of prediction errors, by presenting them a series of different physical and social events with expected versus unexpected outcomes across various domains, from physics about objects to numbers and actions (see Figure 1), while recording their EEG. In particular, we used four different stimulus categories representing well-established paradigms from the VOE literature (testing infants’ concepts of action, solidity, cohesion, and number; see Figure S1 for the full stimulus set) to obtain a more generalized prediction error response, independent from a specific knowledge domain. Because of its pivotal role in prediction error processing and learning in adults, we expected higher ongoing 4 Hz theta activity for unexpected versus expected events. Furthermore, based on previous ERP studies in infants, we expected a differential Nc response (400 – 600 ms, at central electrodes) for expected versus unexpected events.
Materials and Methods
Participants
The final sample consisted of 36 9-month-old infants (17 girls, M = 9.7 months, SD = 0.5 months). Participants were healthy full-term infants, from Leipzig, Germany. Informed written consent was obtained from each participant’s parent before the experiment and the experimental procedure was approved by the local ethics committee. Thirteen additional infants were tested but excluded from the final sample, due to fussiness (n = 2) or because fewer than 10 artifact-free trials remained in each condition (n = 11). This attrition rate is rather low for visual EEG studies with infants (Stets et al., 2012).
We selected this age group, because previous studies indicated VOE responses for the domains tested here by the age of 9 months or even earlier (Reid et al., 2009; Spelke et al., 1992; Wynn, 1992). The sample size was oriented at a former study with a very similar study design (Köster et al., 2019) and ultimately determined by the number of families with infants in the in the targeted age-range, which were available in the period of the data assessment.
Stimuli and Procedure
Stimuli were based on four classical VOE paradigms for the four core knowledge domains action, number, solidity, and cohesion, with four different stimulus types (variants) each, resulting in 16 different stimuli, which could be presented with an expected or unexpected outcome (Figure 1 and Figure S1, for the complete stimulus set). Each sequence consisted of three static images which, shown in sequence, depicted a scenario with a clearly expectable outcome.
In a within-subjects design, each of the 16 sequences was presented two times in each condition (expected or unexpected). This resulted in a total of 64 distinct trials, presented in 16 blocks. The order of the core knowledge domains, outcomes and the specific stimulus variations (four in each domain) were counterbalanced between blocks and across infants. We decided to present a high diversity of stimulus types from different domains to reduce transfer effects and keep infants’ attention high throughout the experiment. It should be noted that infants may get used to the stimuli and that this may reduce their surprise for unexpected outcomes over time. However, this would reduce, but not increase, the difference in the neural activity between expected and unexpected events.
Every trial began with an attention getter (a yellow duck with a sound, 1 s), followed by a black screen (variable duration of .5 – .7 s) and the three stimulus pictures. The first two pictures showed the initiation of an event or action (0 – 2 s, 1 s each picture), followed by the picture presenting the expected or the unexpected outcome (see Figure 1). The final picture was presented for 5 s, for a companion eye-tracking study. Specifically, while we initially planned to assess and compare both infants’ gaze behavior and EEG response, the concurrent recording (EEG and eye-tracking) only worked for a limited number of infants and trials. Therefore, a match between the two measures was not feasible and we decided to collect more eye-tracking data in an independent sample, as a companion study. For the present study and analyses, we included all trials in which infants looked at the screen for at least 2 s of the final picture, coded from video (see below). The stimuli showing the outcome, namely the expected or unexpected outcome, were counterbalanced in case of the cohesion and the number stimuli (i.e., in the cohesion sequences outcome stimuli showed connected or unconnected objects and for number sequences the outcome showed one or two objects) and were matched in terms of luminance and contrast in case of the action and solidity stimuli (all ps > .30). Stimuli were presented via Psychtoolbox (version 0.20170103) in Matlab (version 9.1). The full set of the original stimuli can be downloaded from the supplemental material of (Köster et al., 2019).
Infants sat on their parent’s lap at a viewing distance of about 60 cm from the stimulus monitor. Sequences were presented at the center of a 17-inch CRT screen at a visual angle of approximately 15.0° × 15.0° for the focal event. We presented all 64 trials, but the session ended earlier when the infant no longer attended to the screen. A video-recording of the infant was used to exclude trials in which infants did not watch the first 4 s of a trial. Gaze behavior was coded offline.
Electroencephalogram (EEG)
Apparatus
The EEG was recorded continuously with 30 Ag/AgCl ring electrodes from 30 scalp locations of the 10-20-system in a shielded cabin. Data were recorded with a Twente Medical Systems 32-channel REFA amplifier at a sampling rate of 500 Hz.
Horizontal and vertical electrooculograms were recorded bipolarly. The vertex (Cz) served as an online reference. Impedances were controlled at the beginning of the experiment, aiming for impedances below 10 kΩ.
Preprocessing
EEG data were preprocessed and analyzed in MATLAB (Version R2017b). EEG signals were band-pass filtered from 0.2 Hz to 110 Hz and segmented into epochs from −1.5 to 3 s, around to the onset of the outcome picture. Trials in which infants did not watch the complete 4 s sequence (2 s during the initiation of the event and 2 s of the outcome picture) were excluded from the analyses. Furthermore, noisy trials were identified visually and discarded (approx. 10 % of all trials) and up to three noisy electrodes were interpolated based on spherical information. Eye-blinks and muscle artifacts were detected using an independent component procedure (ICA) and removed after visual inspection. To avoid any bias in the ICA removal, the ICAs were determined and removed across the whole data set, including all experimental conditions (both frequencies, both outcome conditions, all stimulus categories). Prior to the analyses, the EEG was re-referenced to the average of the scalp electrodes (Fz, F3, F4, F7, F8, FC5, FC6, Cz, C3, C4, T7, T8, CP5, CP6, Pz, P3, P4, P7, P8, Oz, O1, O2). Infants with a minimum of 10 artifact-free trials in each condition were included in the statistical analyses. Twenty-two to 52 trials (M = 32.2, SD = 7.3) remained for the infants in the final sample, with no significant differences in the number of trials between conditions (expected, unexpected), t(35) = 0.63 p = .530. We also plotted the data split by conditions, on subsamples with at least one trial for both the expected and the unexpected outcome condition. The respective size of subsamples and number of trials were action: n = 35, M = 10.3, SD = 3.2, solidity: n = 35, M = 6.9, SD = 2.7, cohesion: n = 32, M = 6.1, SD = 3.6, and number: n = 36, M = 8.5, SD = 3.0.
ERP Analysis
For the analyses of event-related potentials (ERPs), we averaged the neural activity, separately for the trials of both conditions (expected, unexpected). We focused on the NC as a classical component associated with infants’ processing of expected versus unexpected events (Reynolds, 2015). Specifically, we averaged the ERPs across central electrodes (Cz, C3, C4), and between 400 – 600 ms, with regard to a −100 – 0 ms baseline. We chose a baseline just before the onset of the outcome picture. Because it was shown as part of a picture sequence, each picture elicited a neural response, and this response (4 – 5 Hz and ERP) decayed towards the beginning of the next stimulus. The ERP power was averaged for each participant and condition and the power between expected and unexpected trials was then contrasted by means of a dependent t-test. We band-pass filtered the ERPs from 0.2 – 30 Hz for displaying purposes.
Spectral Analysis
To obtain the trial-wise spectral activity elicited by the outcome pictures we subjected each trial to a complex Morlet’s wavelets analysis (Morlet parameter m = 7, at a resolution of 0.5 Hz). We then averaged the spectral power across trials, separately for conditions (expected, unexpected). We focused on the frequencies from 2 to 15 Hz across the whole analyzed time window 0 – 2000 ms, with regard to a −100 – 0 ms baseline, to make the results directly comparable to the ERP analysis in this and former studies. We did not analyze higher frequencies due to muscle and ocular artifacts in the infant EEG (e.g., Köster, 2016).
Because this was the first study to look at the trial-wise neural oscillatory response to a series of unexpected versus expected events (i.e., not tightly locked to the stimulus onset; cf. Berger et al., 2006), in a first step, we looked at the grand mean spectral activity, separated by conditions (unexpected, expected), and the difference between both conditions (unexpected - expected). Conservatively and because we did not have a specific hypothesis about the topography or temporal evolution of the theta rhythm across all domains, we analyzed the neural oscillatory activity averaged across the whole time-range of the outcome stimulus (0 – 2000 ms) and all scalp-recorded electrodes (Fz, F3, F4, F7, F8, FC5, FC6, Cz, C3, C4, T7, T8, CP5, CP6, Pz, P3, P4, P7, P8, Oz, O1, O2). Note that, a multiple comparison correction was not feasible here as we analyzed the whole electrode space and time range,. While our initial proposal was to look at the difference in the 4 Hz theta rhythm between conditions (Köster et al., 2019), we found the strongest difference between 4 – 5 Hz (see lower panel of Figure 3). Because this was very close to our initial hypothesis, in particular for being the first study looking at infants’ ongoing theta activity in a VOE paradigm, we analyzed this frequency range.
Results
Infants’ event-related responses upon the onset of the outcome picture revealed a clear Nc component between 400 – 600 ms over central electrodes. The Nc was more pronounced for expected in contrast to unexpected events, t(35) = −2.62, p = .013 (Figure 2).
Furthermore, across all scalp recorded electrodes and the whole 0 – 2000 s time window, we observed an increase in neural oscillatory activity in the 4 – 6 Hz range for unexpected events and an increase at 6 Hz for expected events, t(35) = 4.77, p < .001, and, t(35) = 4.01, p < .001 (Figure 3). This resulted in higher 4 – 5 Hz activity for unexpected compared to expected events across all scalp-recorded electrodes and throughout the whole 0 – 2000 s time-window, t(35) = −2.33, p = .025.
To investigate the relation between the effects which we found in the ERP and the ongoing 4 – 5 Hz theta activity, we tested the spectral characteristics of the evoked oscillatory activity (i.e., by applying a wavelet transform to the ERP) and its relation to the ongoing oscillatory activity at central electrodes (Cz, C3, C4) between 400 – 600 ms (Figure S2). We did not find a significant condition difference in the evoked activity, t(35) = 1.57, p = .126, nor the ongoing activity, t(35) = −1.26, p = .218, at these electrodes. The condition effects (unexpected - expected) were also not correlated between the evoked and the ongoing response, neither for the difference in the actual ERP, r = −.07, p = .675, nor its spectral characteristics, r = .23, p = .169.
Although the present study was designed to investigate infants’ prediction error processes across domains, to get an impression about the consistency of the differences in the central Nc and the 4 – 5 Hz activity, we plotted the data split by domains (action, solidity, continuity, number). The overall time course of the ERP and the 4 – 5 Hz effect was somewhat consistent across conditions, however, the condition differences (Figure 1 and 2) were driven to a large degree by the stimuli of the action and the number domain (see Figure S3 and S4). Interestingly, the peak in the unexpected – expected difference was in the 4 – 5 Hz range across all four domains (Figure S4 A). Critically, we did not test these domain-specific differences statistically due to the low trial numbers within each domain and the main focus of the study being on prediction error processing in the infant brain more generally, across different domains.
Discussion
Our results show a clear increase in the ongoing 4 – 5 Hz power in response to unexpected events, in contrast to expected events. This effect was distributed across all scalp-recorded electrodes and for a prolonged time window of 2 s after the onset of unexpected outcome pictures. Thus, the theta rhythm was substantially increased for the processing of prediction errors in the infant brain. Furthermore, in the ERP response we found a stronger Nc for expected events, in contrast to unexpected events, at central electrodes.
As revealed by a direct comparison at central electrodes the effects of the ongoing theta response were not related to the ERP nor the spectral characteristics of the ERP (cf. Tallon-baudry and Bertrand, 1999). Thus, the theta response analyzed here reflects a distinct neural signature compared to those reported in former studies, which focused on evoked responses (Berger et al., 2006; Conejero et al., 2018; Kayhan et al., 2019; Langeloh et al., 2020; Reynolds and Richards, 2005; Webb et al., 2005).
The ongoing theta rhythm has been associated with learning processes in human adults (Friese et al., 2013; Hanslmayr et al., 2009; Klimesch et al., 1997; Köster et al., 2018; Osipova et al., 2006), children (Köster et al., 2017), and infants (Begus et al., 2015). Our findings highlight that the theta rhythm promotes the processing of novel, unexpected information, in the sense of prediction errors, already in early infancy. This is particularly interesting because the theta rhythm is usually associated with neural processes in prefrontal and medio-temporal structures, which are still immature in the infant brain (Gilmore et al., 2012). Furthermore, the theta rhythm has long been associated with cognitive control processes in adults (Cavanagh and Frank, 2014; Hanslmayr et al., 2008) and children (Adam et al., 2020), and infants’ ongoing theta oscillations at 6 months were predictive for their cognitive ability at 9 months (Braithwaite et al., 2020).
Embedding the role of the theta rhythm in a broader theoretical framework, from animal models we know that the theta rhythm promotes predictive processes (i.e., such as the activation of future locations in a labyrinth; O’Keefe and Recce, 1993) and facilitates Hebbian learning (Tort et al., 2009). Based on these findings, the theta rhythm has been described as a neural code for the sequential representation and the integration of novel information into existing concepts (Lisman and Jensen, 2013). We would like to add to this that the theta rhythm may implement a computational mechanism that compresses real time events onto a faster neural time-scale, to advance with cognitive processes ahead of real time and to facilitate the integration of new events into existing networks. This is critical to predict future events and integrate novel events as they happen in real time. While former studies have demonstrated that this computational mechanism may be phylogenetically preserved in the mammalian linage (Cavanagh and Frank, 2014; Lisman and Jensen, 2013), here we report first evidence that the ongoing theta rhythm supports the processing of unexpected events already from very early in human ontogeny.
We also identified differences between unexpected and expected events in the Nc, a classical visual ERP component associated with infants’ processing of unexpected events. As expected from former studies, the Nc and the condition difference was pronounced between 400 – 600 ms, and was specific to central electrodes (Cz, C3, C4). However, the condition difference pointed in the opposite direction than most (Kayhan et al., 2019; Langeloh et al., 2020; Reynolds and Richards, 2005; Webb et al., 2005), though not all (Kaduk et al., 2016; Reid et al., 2009), previously reported Nc effects (namely, the more common findings of a higher negativity for unexpected events). It is currently not clear, why unexpected events induce enhanced Nc amplitudes in some studies, but a decreased Nc compared to expected events in others. Because the amplitude of the Nc has been associated with the extent of attentional engagement with a visual stimulus (Reynolds, 2015; Reynolds and Richards, 2005), in our study infants’ initial orienting response may have been more pronounced for the more familiar and expected outcomes. This is in line with previous studies using partly similar stimuli (in particular the action events; Kaduk et al., 2016; Reid et al., 2009) and with the notion that infants show familiarity preferences (i.e., the preference for events consistent with their experience) when they are still in the process of building stable cognitive representations of their environment (Nordt et al., 2016). While we did not have sufficient statistical power (and it was also not the main purpose) in the present study to test the differential neural responses to the events in different domains, this remains an intriguing question for future research.
To conclude, our findings make a strong case that the theta rhythm is present from very early in ontogeny, associated with the processing of prediction errors and, putatively, the refinement of the emerging concepts of the physical and social environment. This marks an essential step towards a better understanding of the neural oscillatory dynamics that underlie infants’ brain development and their emerging models of the world around them.
Conflict of interest
There are no conflicts of interest.
Funding Information:
This research was supported by a Max Planck Research Group awarded to SH by the Max Planck Society.
Acknowledgements
We would like to thank Carl Bartl und Ulrike Barth for their support with the data assessment and coding.
Footnotes
See Response to open peer reviews