Abstract
Translational applications of cognitive science depend on having predictive models at the individual, or idiographic, level. However, idiographic model parameters, such as working memory capacity, often need to be estimated from specific tasks, making them dependent on task-specific assumptions. Here, we explore the possibility that idiographic parameters reflect an individual’s biology and can be identified from task-free neuroimaging measures. To test this hypothesis, we correlated a reliable behavioral trait, the individual rate of forgetting in long-term memory, with a readily available task-free neuroimaging measure, the resting-state EEG spectrum. Using an established, adaptive fact-learning procedure, the rate of forgetting for verbal and visual materials was measured in a sample of 50 undergraduates from whom we also collected eyes-closed resting state EEG data. A statistical analysis revealed that the individual rates of forgetting were significantly correlated across verbal and visual materials, in agreement previous results. Importantly, both rates correlated with power levels in the alpha (8-13 Hz) and low beta (13-15 Hz) frequency bands, with the correlation between verbal rate of forgetting and low beta power over the right parietal site being significant even when accounting for multiple comparisons. The results suggest that computational models could be individually tailored for prediction using idiographic parameter values derived from inexpensive, task-free imaging recordings.
1. Introduction
To provide a complete account of human behavior, psychological theories should explain behaviors at both the nomothetic level (that is, group aggregates and mean tendencies) and the idiographic level (that is, accurate characterizations of each individual participant; (Allport, 1937)), a claim that equally holds for computational cognitive models.
Although the nomothetic approach has been historically dominant and the majority of published computational models are fitted to group averages, the idiographic approach has been frequently advocated (Ritter & Gobet, 2000) and is often essential to translational applications of cognitive research. For instance, in intelligent tutoring systems, idiographic models are critical to provide appropriate adaptive feedback to specific errors and knowledge of individual students (Anderson et al., 1990).
In the idiographic approach, individuals can be characterized at the level of stable characteristics, or traits, or contingent situations, or states. An individual’s working memory capacity, for example, is relatively stable over time. On the other hand, a student’s domain knowledge is continuously expanded during studying, and is thus better characterized as a state. Because they capture stable characteristics, traits are particularly useful to predict individual behaviors across tasks and over extended periods of time. An intuitive conceptualization is to think of idiographic traits as specific values of a model’s parameter (Collins, 2018; Daw, 2011; Lovett et al., 2000; Ritter & Gobet, 2000; Stocco, 2018). Working memory capacity, for example, can be captured by a parameter that represents the number of free slots in a buffer, and idiographic traits by different values of this parameter (Collins, 2018).
Idiographic trait parameters should exhibit at least two characteristics. First, they should have high test-retest reliability. For instance, Sense et al. (2016) have shown that long-term memory decay is stable across sessions (r ∼ 0.8) and across materials (r ∼ 0.5). Second, it should generalize across tasks: once a parameter has been estimated by fitting a model to an individual, the same parameter’s value should predict that individual’s performance in other tasks. For example, Maaß and colleagues (2019) were able to estimate the variability of the internal clock using a simple time production task (Maaß & van Rijn, 2018), and use it to predict performance in a more elaborated temporal reproduction task in a pre-clinical population.
We propose a third essential characteristic: trait parameters should reflect a feature of the individual’s neurobiology. An example of which is the correlation between procedural learning rate and the density of dopamine receptors (Stocco, 2018; Stocco et al., 2017). In this paper, we investigate whether individual variability in long-term memory rate of forgetting is reflected in individual variability in electrophysiological measures of brain activity.
1.1 Resting State Neuroimaging as a Window to Idiographic Parameters
The idea of correlating model parameters with brain activity is hardly new (e.g., reinforcement learning rate correlates with striatal activity during a learning task (Schönberg et al., 2007), loss aversion parameter in prospect theory with striatal activity (Tom et al., 2007), and the level of response caution during anticipation with activation of pre-SMA (Boehm et al., 2014). Most of these attempts, however, rely on task-based activity, and thus suffer from a potential circularity: Because the model is designed to reproduce a specific type of task, the identified neural substrates are a function of the specific task assumptions as much as they are a function of the specific parameter.
This circularity can be circumvented by the use of task-free neuroimaging measures. These measures are made possible by the fact that spontaneous but organized brain activity exists even in the absence of any observable behavior (Fox et al., 2005). Task-free measures are recorded during “resting-state” sessions, in which participants are requested to refrain from doing anything: although they must remain awake, they are typically asked to just fixate a stimulus or close their eyes for a few minutes during which brain activity is recorded.
Although the nature and the functional significance of brain activity at rest are debated (Raichle, 2006; Raichle & Snyder, 2007), it is undeniable that resting-state brain activity carries signatures of measurable individual characteristics. For instance, resting-state fMRI can be used to classify individuals by age (Dosenbach et al., 2010), even among toddlers (Pruett et al., 2015), variations in resting-state correlations between regions have been linked to pathologies such as depression (Greicius et al., 2007) and Alzheimer’s disease (Sorg et al., 2007), and correlations between regions during resting-state can be used to predict behavioral performance during a task (Cole et al., 2016).
Although resting-state research typically focuses on fMRI, other modalities can be used— such as EEG. Instead of using correlations between time series (which are not meaningful for oscillatory signals), resting-state EEG data is first decomposed into different frequency components, and then the amount of variance of the whole signal that is explained by each frequency is estimated. The amount of variance is referred to as the frequency’s power and the power distribution over frequencies as the power spectrum. Power spectra can be used to uniquely identify individuals, much like an electrophysiological fingerprint (Ma et al., 2015; Mohammadi et al., 2006), and different features of the spectrum correlate highly with cognitive traits like intelligence (Doppelmayr et al., 2005) and language aptitude (Prat et al., 2016).
1.2. Idiographic Parameters in a Model of Long-Term Memory Forgetting Rate
Surprisingly, despite previous successes in identifying correlations between cognitive constructs (such as intelligence or aptitude) with resting-state EEG, no study to date has related task-free activity with the specific parameters of a cognitive model. The goal of this study is to demonstrate the possibility of measuring one specific and well-understood idiographic parameter from specific signatures of EEG data. Specifically, we will focus our attention on decay rate in long-term memory, a parameter that has an established tradition of use in cognitive models (Anderson, 1990; Shiffrin & Steyvers, 1997) and whose test-retest and cross-task reliability has already been previously studied (Sense et al., 2016).
The implementation used herein was inspired by Pavlik and Anderson (2008) and rooted in Anderson’s Bayesian account of memory (Anderson, 1990; Anderson & Schooler, 1991). A memory’s availability is proportional to a scalar meta-quantity, the base-level activation, which is the sum of the decaying traces of its previous usage. The rate of decay is determined for each memory trace separately, and is a function of the base-level activation of earlier encoded traces of the same memory when the memory is encoded and a memory specific intercept (see Van Rijn et al., 2009. and Sense et al., 2016, for details). The average of this memory-specific decay rate intercept, α, is highly stable when the same individual is tested at different moments (r ∼ .80) and even across different materials (r ∼ .50), and thus meets the two first criteria for individual trait characteristics.
To measure α, Sense et al. (2016) asked participants to memorize a series of paired associates relating an old, familiar item with an entirely new item using a previously developed, adaptive memory testing procedure (Van Rijn et al., 2009). In one of their studies, the cues were (unfamiliar) Swahili words and their (familiar) Dutch translation.
1.3 Research Question and Considerations About Effect Sizes
In this study, we replicated the experimental setup of Sense et al. (2016) and asked participants to perform an iterative pair-associates test, using both verbal (i.e., Swahili words) and non-verbal materials (i.e., maps of locations in the United States). The estimated mean value of α for each individual was then correlated with features of an individual power spectrum, estimated from a 5-minute resting-state EEG recording collected prior to the behavioral task. We expected to find a correlation between the value of the decay rate α and the spectral power in frequency bands, such as the alpha and beta rhythms, that have been previously correlated with similar memory-related constructs such as language learning rate (Prat et al., 2016).
In designing the experiment, we considered what effect sizes we expect to find. Intuitively, it is easy to believe that neural features, being closer to an individual’s ground-truth biology, should yield high correlations. In this study, however, the correlation is assumed to be between two different measurements of the same underlying biological process (i.e., the reduction in availability of memory traces), one at the behavioral level (forgetting rate) and one at neurophysiological level (spectral power), each with different types of noise and sources of error. When trying to relate one quantity to another, measurement error for both is the product of the error in each individual measurement. Sense et al. (2016) estimated that the test-retest reliability for α is r ∼.80. Unpublished data from our laboratory shows that the test-retest reliability of power spectra, using the specific equipment reported in this study, is approximately r ∼.60 across all channels and frequency bands (Prat et al., in preparation). Thus, the maximum reliability that we can expect from the correlation is .8 × .6 = .48. Given these effect sizes, we estimated that collecting data from N = 50 participants would give us sufficient power to avoid a Type II error with probability β = 0.95 with uncorrected significance level of p < 0.05.
2. Materials & Methods
2.1 Participants
Fifty-three native English speakers (32 females) aged between 18 to 33 years old (mean = 21) were recruited from the undergraduate population at the University of Washington, Seattle. None of the participants reported any familiarity with the Swahili language. Only native English speakers were selected to ensure that their performance on the Swahili vocabulary test (which required memorizing pairs of English-Swahili words) was not confounded by significant differences in English proficiency and English vocabulary exposure. Data from three participants were excluded because of either equipment failure (one male participant) or too few data points (< 75 artifact-free epochs in each channel: two females), leaving 50 subjects’ data (30 females) in the final analysis.
2.2 Materials
2.2.1 Language Background
Participants’ language background was measured using the Language Experience and Proficiency Questionnaire (LEAP-Q: Marian, Blumenfeld, & Kaushanskaya, 2007). The LEAP-Q reports were used to ensure that all participants were native English speakers (Average age of English acquisition: 14 months old; 100% in self-rated English speaking proficiency; 100% in self-rated English understanding proficiency) and none of them had been previously exposed to Swahili.
2.2.2 Swahili vocabulary learning task
Twenty-five Swahili-English word pairs were selected from a previous study (Van den Broek, Segers, Takashima, and Verhoeven, 2014). On the study trial, a Swahili word and its corresponding English equivalent were presented simultaneously on the screen. Participants were asked to respond by typing the corresponding English word (e.g., “fish”) after being presented with a Swahili word as a cue (e.g., “samaki”). On test trials, only the Swahili word was presented on screen and participants were asked to type the English word. The order of repetitions and moment of introduction for each item were determined by the adaptive scheduling algorithm introduced in the introduction, and described in more detail in Van Rijn et al. (2009) and Sense et al. (2015; 2016). Participants received corrective feedback after each response. A running timer and the ongoing response accuracy remained visible on the screen during each session. The task lasted approximately 12 minutes.
2.2.3 Map Learning Task
In the map learning task, an outline of a US map was presented on the screen with black dots that represent different locations. Twenty-five cities, each with a unique name, were selected evenly across the US map. Names were those of real but small cities. On each study trial, participants were shown the map with one location highlighted in red and were instructed to learn the corresponding city name, which was displayed along with the map. On test trials, participants were shown the map with a highlighted location and were asked to type in the city name. All other details were identical to the Swahili vocabulary task.
2.3 Procedures
Participants were asked to complete the LEAP-Q and a demographic survey before a five-minute eyes-closed and one five-minute eyes-open resting EEG were recorded. Then, they completed both learning tasks with the order of learning tasks counterbalanced based on the parity of participant IDs. At the end of the session, participants filled out a survey about perceived difficulty per task, data of which are not analysed in this manuscript.
2.3.1 Acquisition and Preprocessing of EEG data
A continuous recording of 5 minutes of eyes closed, resting state EEG data were collected for each participant using a wireless 16-channel headset (first-generation Emotiv EPOC, Australia) with a sampling rate of 128 Hz. The reference channels were the DMS and CRL electrodes over the parietal lobe. During the eyes-closed resting-state EEG recording, participants were instructed to close their eyes, clear their mind, and relax, all while in a dark room. During the eyes-opened resting-state EEG recording session, the experimenter turned on the light in the room and instructed participants to relax and look at a black fixation point on a white screen for 5 minutes while their EEG data were recorded. The eyes-opened and eyes-closed EEG data were processed separately following the procedure below.
The resting-state EEG data were divided into two-second epochs with a 0.5-second overlap. Epochs containing significant artifacts (e.g., eye blinks, excessive motion, or signal deflections greater than 200 μV) were excluded from the analysis. All remaining 2-second epochs underwent Fast Fourier Transformation (FFT). Because of the segmentation into epochs, the resulting spectrum had a resolution of 0.5 Hz, and extended from 0 to 64 Hz. The spectra of each epoch were then averaged together for each channel. Finally, the spectral powers within each frequency bands were averaged to yield mean power values in the theta (4-8Hz), alpha (8-13Hz), beta (13-30Hz), low beta (13-15Hz), mid beta (15-18Hz), and high beta (18-30Hz) bands. The signal-to-noise ratio afforded by our equipment did not permit to reliably detect signals at higher frequencies, while very low frequencies tend to be contaminated by sporadic and non-physiological sources of noise (Cohen, 2014). Due to these considerations, gamma (> 30 Hz) and delta (0-4 Hz) frequency bands were excluded from the analysis. Any channel with fewer than 75 artifacts-free epochs for each individual was excluded, resulting in the removal of one channel (0.14% of the data) in the final analysis. This preprocessing pipeline was the same used in a previous study on neural markers of language learning (Prat et al., 2016; 2018). The analysis script can be found at http://github.com/UWCCDL/QEEG.
2.3.2 Analysis of Behavioral Data
The main behavioral measure that will be correlated with eyes-closed EEG activity is the average rate of forgetting α, which was estimated separately for each learning session (maps vs. vocabulary) of each participant. Here, we employed the same approach as in Sense et al. (2016): The parameter is estimated for each item separately, and an individual session’s parameter is computed by averaging the final values of all items that have been repeated at least three times. The aggregate value thus reflects the mean individual rate of forgetting and it reflects how quickly, on average, each participant forgot each type of material.
3. Results
3.1 Behavioral Results
Figure 1A shows the distribution of the rates of forgetting for the two types of materials, estimated for each participant and plotted against each other. For the Swahili items, the mean rate of forgetting is α = 0.283 (SD = 0.062) and for the US Maps, the mean rate of forgetting is α = 0.336 (SD = 0.051). The figure shows that a participant’s rate of forgetting for one type of material is highly correlated with the rate of forgetting for the other type of material (r(50) = 0.59; t(48) = 5.08; p < 0.001). This correlation is comparable to the one reported in the study these materials were adapted from (approx. 0.50–0.55, see Table 1 in Sense et al., 2016). The figure also shows that the rates of forgetting tend to fall above the diagonal equality line, indicating that the rates of forgetting were generally higher for the US maps (range: 0.21–0.47) than for the Swahili words (range: 0.12–0.39; paired t(48) = 7.24, p < 0.001).
3.2 EEG Results
The EEG power spectrum was extracted for each channel and visually inspected to ensure its consistency across channels and its conformity to the established distribution of power across frequency bands. Figure 2 depicts the power spectrum over frequencies in the 0-40 Hz range in all 14 channels, with the seven relevant frequency bands marked by different background colors. Although the gamma and delta bands were not included in the analysis, they are depicted in Figure 2 for completeness. As expected, all channels exhibit the same power spectral profile, with the characteristic 1/f power distribution that is typical of electrophysiological signals (Buzsaki, 2006; Cohen, 2014). The peak in the alpha band (8-13 Hz), known as the individual alpha frequency (IAF), identifies the center of the alpha frequency band; although some authors prefer to use a priori defined ranges for each frequency (Prat et al., 2016), others prefer to define the alpha band on the bases of the mean IAF across participants, and offset the other bands in relation to it (Doppelmayr et al., 2002). In our data, the mean IAF was 10.5 Hz in all channels (Figure 2). This value falls exactly at the center of the predefined range, thus making our results compatible with both approaches and implying that the predefined frequency bands are also representative of our sample.
3.3 Relationship Between EEG Power and Forgetting Rate
Next, we examined the correlations between the rates of forgetting and the power across channels in each of the following frequency bands: theta (4-8 Hz), alpha (8-13 Hz), beta (13-30 Hz), low beta (13-15 Hz), mid-beta (15-18 Hz), and high beta (18-30 Hz). In the following sections, we report all the correlations significant at p < .05, corresponding to a Pearson correlation value of r > 0.27. Because of the number of correlations that were computed, we also applied the False Discovery Rate procedure (Benjamini & Hochberg, 1995) to account for multiple comparisons, and highlight results that pass the corrected threshold of q < .05.
3.3.1 Rates of Forgetting for Verbal Materials
The top row of Figure 3 presents the correlations between power at specific channel locations and the rate of forgetting in the Swahili word learning task across frequency bands. Significant positive correlations were found in the beta frequencies (13–30Hz) over the right parietal lobe (P8 in the 10–20 system, r(50) > .24). Specifically, the largest positive correlation (r(50) = .42, p = .002) was observed at the right parietal channel (P8 in the 10–20 system) in the low beta band (13–15 Hz). This particular location and power band were also the only ones to show a significant effect when correcting for multiple comparisons (q = .03; Figure 4). Significant positive correlations that did not survive corrections for multiple comparisons were found for the low beta band at the bilateral frontal regions (AF3, F4, F7, r(50) > .27, q > .15) and right occipital region (O2, r(50) = .34, p = .02, q = .11), and for mid-beta power at the right parietal lobe (P8, r(50) = .34, p = .02, q = .24).
3.3.2 Rates of Forgetting for Visuospatial Materials
Correlations between EEG power and rates of forgetting in the map learning task showed a similar pattern of results (Figure 3, bottom row), albeit with overall smaller correlations. As in the case of the vocabulary task, no negative correlation reached significance at p < .05. Significant positive correlations between the rate of forgetting and power in the beta frequency band were observed at right frontal site (AF4, r(50) = .31, p = .03, q = .19) and, once more, at the right parietal lobe (P8, r(50) = .36, p = .01, q = .11). Power in the low beta band was positively correlated with the rate of forgetting at the right parietal lobe (P8, r(50) = .30, p = .04, q = .32). The significant positive correlation between power in the beta band and rate of forgetting was observed distributing at bilateral frontal (AF3, AF4, F4, r(30) = .29–.34, p < .04, q < .32), right frontal (F8, r(50) = .32, p = .02, q = .09), and right parietal regions (P8, r(50) = .36, p = .009, q = .09). Power in the high beta (18-30Hz) was positively correlated with the rate of forgetting at the right frontal (AF4, r = .28, p = .05, q = .68) and right parietal region. (P8, r(50) = 0.33, p = .02, q = .25). However, no correlation was significant at the corrected value of q < .05.
4. Discussion
In this paper, we have provided evidence that model parameters that reliably characterize an individual’s performance can be observed in that individual’s neurophysiological activity at rest. Importantly, the observed correlations are in the upper range of values that were expected, given the reliability of both behavioral and EEG measures.
It should be noted that brain activity was extracted at rest before participants performed the experimental task. Thus, this finding adds to the growing number of studies showing that task-free resting-state oscillations contain relevant information about an individual’s biology and cognition (Doppelmayr et al., 2002; Doppelmayr et al., 2005; Prat et al., 2016). Specifically, this study is the first not only to link a feature of resting-state brain activity to cognition, but to a specific parameter in a formal and well-established model.
Although many locations on the scalp were found to show mild correlations with our parameter, the strongest results were found for verbal materials (Swahili words) on the right parietal cortex (channel P8) and in a specific frequency band (low beta). When estimating rates of forgetting, verbal materials had shown better reliability than visual materials (Sense et al., 2016). This location and frequency band, however, might be unexpected, given that one would expect the rate of forgetting to be related to the activity of the hippocampus, which is located in the medial temporal lobe, and whose characteristic frequency at rest is the theta band. This discrepancy, however, should not sound surprising. First, the location of an EEG signal over the scalp does not translate directly to an underlying source. Thus, the fact that a signal is recorded over a parietal region does not imply that its source is also parietal. Second, signals from medial regions are notoriously difficult to identify in EEG data, due to their greater distance from the scalp. Thus, it is likely that the signals that mostly correlated with rates of forgetting do not reflect hippocampal activity per se, but possibly the contribution of other circuits that are ancillary to memory encoding and retrieval and might be indirectly affected by the intrinsic activity of the hippocampus. This idea is supported by the fact that both frontoparietal activity and low-beta activity have been reported as related to successful memory encoding and retrieval during memory tasks (Hanslmayr et al., 2011, 2012). Furthermore, the same location and frequency band have been reported to be related to language learning (Prat et al., 2016, 2018), both of which are cognitive functions that depend on the capacity to quickly and efficiently encode and retrieve material. Thus, our pattern of results is broadly consistent with the known neurophysiology of memory encoding and retention.
4.1 Limitations
When interpreting these results, a number of limitations must be considered. Some of these limitations are intrinsic to the use of EEG: as noted above, it is impossible to precisely identify the source of (resting-state) EEG signals. For this reason, we can only speculate about the brain regions whose activity is associated with the decay rate. By contrast, information about the source would be valuable in understanding what type of information we are dealing with. This limitation could be overcome by using different neuroimaging methods, such as MRI-informed MEG.
The EEG-specific limitations are compounded, in our case, by the use of low-grade systems. While they allowed us to record and collect data quickly and they have been used successfully in research (Jiang et al., 2019; Prat et al., 2016, 2018), they are also characterized by poor spatial resolution, poor sampling rate, and low sensitivity. Thus, not only is source identification practically impossible, but significant features that are best observed in midline locations (not covered by our setup), or have low amplitude, or occur at higher frequency rates would likely go undetected in this study. However, the use of this system demonstrates that relevant information on individual learners can be obtained using systems that might see usage in class-room settings, opening the door to neuroscience-informed adaptive learning systems.
We restricted our analyses to bi-variate correlations, adhering to standards set in similar studies with comparable equipment (Prat et al., 2016, 2018). However, it should be noted that more sophisticated methods, including machine learning approaches that take into account multiple features, could be potentially employed. This would likely result in greater precision at predicting specific values of α from EEG data alone.
Here, we interpret the relationship between brain activity and a specific component of a formal model. Although this model has been successfully employed to estimate the rate of forgetting (Sense et al., 2018; Sense et al., 2016, 2019; Van den Broek et al., 2019), the conceptual model of which the current model is an instance also contains other parameters. For example, the probability of retrieving a memory (for example, the association “kitanda”/”couch”) also depends on spreading activation from the cue (for example, the Swahili word “kitanda”). The amount of spreading activation is assumed to be modulated by attention, and the degree of attention allocated to cues has been found to be predictive of working memory capacity (Lovett, 2001). Thus, it is entirely possible that the high correlation found in channel P8 is partially driven by attentional processes that interact with forgetting rate during retrieval. The fact that both the location of the channel (right parietal lobe) and the specific band (low band) have been previously connected to the retrieval of verbal material suggests that this might be the case. Future studies in which both factors are parametrized and estimated for each individual would help more precisely identify the nature of the biological processes involved.
4.2 Implications for Future Research
These limitations notwithstanding, our results do have significant implications for future research. The first and most obvious is that they outline a method to estimate the value of idiographic trait parameters in the absence of task assumptions and with a very reasonable and cost-effective setup. In our case, for example, the collection of EEG recordings took significantly less time than the collection of behavioral data, even accounting for set-up time. This effect is amplified by the use of consumer-grade headsets, which, while resulting in reduced fidelity recordings, also permit quicker and faster data collection from a larger sample than would be allowed by research-grade equipment. Most importantly, while estimating multiple parameters would require a combination of specific tasks, task-free brain imaging data likely contains signals that reflect multiple parameters of interest. Once collected, every additional parameter can be extracted from the data without increasing the number of sessions or their duration.
In addition to its practicality, the use of task-free neuroimaging data to estimate idiographic parameters provides methodological and conceptual advantages. First, it provides a direct connection to the biological interpretation of a parameter. Second, it provides a useful way to constrain the development of computational cognitive models, as solid correlations with well-established biological correlates could become a litmus test of the construct validity of components/conceptual constructs of computational models.
Of course, for these implications to be realistic, the method we have outlined here would need to be expanded upon and tested using other well-established models and parameters. Nonetheless, we believe in the importance of this approach for future research. As noted in the introduction, idiographic models are necessary for successful translational applications of cognitive research in education and clinical settings, and task-free measures of idiographic parameters provide the most straightforward and biologically-grounded method to tailor models to particular individuals.
Footnotes
↵§ Now at Google, Inc, Mountain View, CA