Summary Paragraph
Episodic memories, or consciously accessible memories of unique events, represent a key aspect of human cognition. Evidence from rodent models suggests that the neural representation of these complex memories requires cooperative firing of groups of neurons on short time scales1, organized by gamma oscillations2, 3. These co-firing groups, termed “neuronal assemblies,”4 represent a fundamental neurophysiological unit supporting memory. Using microelectrode data from neurosurgical patients5, we identify neuronal assemblies in the human MTL and show that they exhibit consistent organization in their firing pattern based on gamma phase information. We connect these properties to memory performance across recording sessions. Finally, we describe how human neuronal assemblies flexibly adjust6-9 over longer time scales. Our findings provide key evidence linking assemblies to human episodic memory for the first time.
Cognitive behavior, including memory formation and retrieval, requires cooperative but distributed neuronal firing2, 4, of which the self-organization of hippocampal neurons represents a key example. Data from animal models of hippocampal-dependent memory indicate that organized firing occurs on time scales of roughly 25 ms1, a scale which has several theoretical advantages including facilitating spike timing dependent plasticity10. Subsequent work further demonstrated the importance of neuronal co-activity on these timescales11-13 by identifying6, 11-17 and even incepting14 memories represented by groups of cofiring neurons. Such data support the influential proposal that coordination of firing over longer time scales requires, at its root, the integration of multiple assemblies organized by gamma oscillations2.
Further, groups of neurons that co-fire on a ∼25 ms time scale (“gamma assemblies”) may be specifically important for episodic memory formation related to the representation of context in the mesial temporal lobe (MTL). Recent reports have highlighted slow changes in the neural populations representing time and space (“drift”) over time scales of minutes to days7, 8, 12 and coexistence of static and drifting spatial codes in the hippocampus6. These findings, along with computational modeling, indicate that assembly drift may reflect flexible memory updating, and indeed that flexibility of assembly formation is required for the accurate representation of temporal information9. Yet, evidence supporting the existence of gamma scale assemblies in humans remains limited. Previous work, including both cortical and MTL recordings, has identified neuronal firing sequences related to both item18, 19 and contextual5 information on the order of hundreds of milliseconds19, seconds5, and minutes18 but not on the fundamental gamma time scale proposed as the basic representational unit2, 3. Co-firing patterns on the scale of milliseconds have been reported in the Rolandic cortex, though these were not linked to behavior20.
Despite the importance of assembly formation on gamma time scales to episodic processes, there exists no data from MTL recordings to support their existence or mnemonic relevance in humans, including the mechanisms underlying their organization and the temporal dynamics of their activity. To address these key questions, we utilized a dataset of single unit recordings in human epilepsy patients performing an episodic memory task5. We employ established methods to identify gamma timescale assemblies during episodic memory processing11-13, 21. We then relate the spatiotemporal characteristics of human gamma scale assemblies, including drift in assembly reactivation strength, to memory behavior. Our findings reveal several new insights into the characteristics of these fundamental electrophysiological components of mnemonic processing.
Identification of mnemonic assemblies
Our study included 26 participants undergoing extraoperative seizure mapping coupled with microelectrode recordings from the MTL. Participants completed a total of 38 recording sessions. For each session, patients performed the free recall task, a common assay of episodic memory in which participants study wordlists of non-repeating nouns (“encoding”) and are asked to recall (“retrieval”) as many words as possible from the immediately previous list without cues22 (Fig. 1A). Included participants recalled an average of 21.4 ± 9.1% (mean ± standard deviation) of the studied words (Fig. 1H). During task performance, we obtained microelectrode recordings from mesial temporal regions, including the hippocampus, entorhinal cortex, amygdala, and parahippocampal gyrus (Fig. 1B-D, Supplementary Fig. 1A). Based on extensive findings in rodent models1, 2, 11-13, we focused our initial analysis on co-firing neurons occurring on a timescale of 25 ms, implementing previously published methods21 using decomposition of neuron-by-25 ms time firing rate matrices. This method identifies assemblies as combinations of neurons whose spiking activity is sufficiently correlated within the specified timescale to conclude that the neurons are functionally codependent. Unlike other approaches that group cells on the basis of increased activity in response to the same stimulus15, 17, this framework selects for self-organizing patterns that persist across different items and context1, 2. To achieve this, we first binned spiking data from all encoding events into 25 ms time bins, calculating a normalized firing rate for each neuron in each bin. With a combination of principal and independent component analysis, we identified patterns of neural activity stemming from co-linearity of cell firing. The ultimate result is a vector of assembly “weights” for each gamma assembly, with a single value for each recorded neuron (Fig. 1F). These weight values communicate the degree to which each neuron contributes to the assembly and are used to determine the expression of the assembly within each 25 ms time bin (Fig. 1E).
(A) Localizations of electrodes used to isolate single units used in the study. (B) Percentage of inter-spike intervals (ISI) less than 3 ms across neurons. (C) Distribution of unit yield per channel. (D) Distribution of firing rates of neurons used for assembly identification (n = 307). (E) Average ratio of peak unit spike voltage to signal noise across neurons. (F) Separability of waveforms isolated from the same microwire, for all relevant neurons. (G) Burst index of all neurons used in assembly identification. (H) Identification of a significant fraction of neuronal assemblies after removing leads demonstrating inter-ictal epileptiform discharges at some point during implantation (left, p = 0.0030). (I) Distribution of z-scores representing the anatomical clustering of assemblies (t(44) = 0.2, p = 0.83). (J) Correlation between the number of single units and assemblies identified during a recording session (r = 0.71, p < 0.001). (K) Three additional examples of neuronal assemblies following the convention outlined in Fig. 1. (L) Mean (bar height) ± s.e.m (error bar) of the z-scored firing rates of all assembly member (dark blue) and non-member (blue) neurons demonstrating that the firing rate of members significantly exceeds that of non-members and of baseline. n.s. = not significant. † = p < 0.10. *p < 0.05. **p < 0.01. ***p < 0.001.
(A) Schematic of the free recall task. Each displayed word the participant studies represents an “encoding event.” (B) Unit yield for each brain region included in the study. (C) Example denoised high frequency signals from which we isolated unit activity (top rows) and local field potentials (bottom rows) for each brain region. Coloring follows the convention shown in (B). (D) Example units from each of the high frequency signals displayed in (C). (E) Expression strength of two example assemblies superimposed on the pertinent spike rasters for three example encoding events. Expression strength curves and spike rasters are colored to link them with the assemblies in (F). (F) Schematics of four example neuronal assemblies. The first two correspond to the data shown in (E). Each colored data point represents a neuron from that recording session. The further the data point from the circle’s center, the greater the contribution of that neuron to the assembly, with member neurons falling outside of the dashed-line circle. The color of each data point represents the region of the neuron, as outlined in (B). (G) Comparison of the number of assemblies identified against a null distribution obtained by shuffling the spike trains. (H) Average and individual recall fraction of recording sessions with identified assemblies. ***p < 0.001.
We isolated a total of 45 gamma timescale assemblies in 15 of the recording sessions (Supplementary Table 1, Fig. 1F, Supplementary Fig. 1K). This significantly exceeded the count expected by chance, as determined by comparing the actual count to a null distribution generated by shuffling the spike trains before re-isolating cofiring patterns (p < 0.001 and 3.81 standard deviations above the distribution mean, Fig. 1G, Supplementary Fig. 1H). From the 15 sessions with identified assemblies, we isolated a total of 203 neurons for which we calculated common spike quality metrics which closely match those in previous studies of the human MTL5 (Methods, Supplementary Fig. 1B-G). The number of simultaneously recorded neurons in these sessions ranged from 5 to 34, with an average of 13.5 cells. We defined assembly reactivation events (Methods) as 25 ms time bins during which assembly expression (Fig. 1E) exceeded the 95th percentile of values observed across the entire recording session. We observed more frequent assembly reactivation events following the presentation of recalled memory items (t-test, t(44) = 2.1, p = 0.022, permutation test, p = 0.091, Supplementary Fig. 3A-C, S3E, and Supplementary Fig. 4B). The degree to which assembly activity predicted successful memory encoding was strongly modulated by assembly complexity11, a metric of how many neurons substantially contribute to assembly expression compared to how many neurons are available to contribute (see methods, Spearman rank correlation, r = 0.61, p < 0.001, permutation test, p < 0.001, Supplementary Fig. 3D, F-G and Supplementary Fig. 4C).
(A) Number of assemblies identified at various timescales within gamma and theta frequency bands. (B) Comparison between the number of assemblies identified in the real data (line) and across 1,000 shuffles (distribution) at each investigated timescale, demonstrating a peak in absolute number and significance at 25 ms.
(A) Two example assembly schematics with member neurons enlarged and colored. (B) Reactivation rates of the two assemblies displayed in (A) during unsuccessful (UE, light) and successful (SE, dark) encoding events. Bar height represents the mean reactivation rate across events and errors bars the s.e.m. (C) Distribution of z-scores obtained by comparing the SE and UE reactivation rate distributions for each assembly. This is referred to as a “subsequent memory effect,” or SME. (D) Positive Spearman correlation between the complexity of the assembly, a metric of how many neurons meaningfully influence assembly activity, and the predictive accuracy of increased assembly reactivation rate. (E) Comparison of the observed t-statistic obtained by compare the distribution shown in (C) to zero to 1,000 t-statistics obtained after shuffling SE and UE event labels. (F) Comparison of the observed Spearman correlation coefficient to a null distribution obtained by shuffling UE and SE event labels. (G) Three example assemblies illustrating the relationship between assembly complexity and memory sensitivity (SME). Schematics are as described in (A). Bar graphs are as described in (B). The SME z-score and complexity values for each example displayed are given by the correspondingly colored data point in (D). n.s. = not significant. † = p < 0.10. *p < 0.05. ***p < 0.001.
(A) Comparison of the average complexity value across assemblies to a null distribution obtained by shuffling the session-wide spike trains of all recorded neurons before re-isolating assemblies and calculating. (B) Distribution of SME z-scores using different thresholds for assembly reactivation (top/teal: 97.5th percentile of observations, bottom/cyan: 99th percentile of observation). (C) Relationship between assembly memory sensitivity and complexity using assembly reactivation criteria outlined in (B).
These data represent a subset of the participant population from a previous study5. We recorded 15 total sessions from these 13 participants, identifying 45 assemblies.
We applied these same methods to detect assembly activation on other time scales as well, reflecting organization at other gamma and theta frequencies. This analysis identified a maximum in the number and significance of assemblies at 25 msec, supporting the inferences drawn from rodent models that motivated our analysis focusing on this time scale1. We note that, as observed in rodent models1, significant co-firing also occurs at other time scales (Supplementary Fig. 2), a point we address below in our Discussion.
Assemblies are comprised of consistent neuronal firing sequences
To characterize the order of firing for assembly neurons within reactivation events, we adapted existing methods to relate consistent firing sequences within an assembly to memory behavior22. First, for each assembly, we isolated all spiking activity of member neurons (n = 84), defined as neurons that increased their firing rate above baseline and significantly more than non-member neurons (Supplementary Fig. 1L), across all reactivation events. We then defined a metric, match index (MI), that communicates how consistently the observed spiking data matches a possible template sequence based on ordered pairs of spikes (Methods). We tested the observed data against all possible orderings of the assembly member neurons and selected the sequence that maximized MI as the expected firing order (Methods, Fig. 2A-C and Supplementary Fig. 5F-H). For example, the bottom two rows of Fig. 2 display data from assemblies with 3 member neurons. There are six orderings of a set of three cells. Within each reactivation event, we generate a list of ordered pairs from the assembly member neurons that fire. A firing sequence of member neuron 3-1-2, generated the ordered pair set of 3-1, 3-2, 1-2. We then calculated the fraction of observed ordered pairs that match each of the possible member neuron orderings to select the expected sequence. We observed that assembly member firing during reactivation events demonstrated significantly higher MI than chance (t-test, t(44) = 7.4, p < 0.001, permutation testing, p < 0.001, Fig. 2D-F and Supplementary Fig. 5I), indicating emergence of consistent temporally ordered sequences in gamma scale assemblies. Further, the MI of assemblies positively correlated with the recall fraction observed during the corresponding session (Spearman rank correlation, r = 0.30, p = 0.042, permutation testing, p = 0.0020, Fig. 2G and 2H), supporting the mnemonic relevance of firing order consistency. This effect remained after including assembly firing rate preferences for successful encoding (Methods, partial Spearman rank correlation, r = 0.30, p = 0.047) and average assembly expression strength across the session (Methods, partial Spearman rank correlation, r = 0.32, p = 0.033), indicating that assembly firing order carries mnemonic information above assembly firing rate. We observed no correlation between assembly member neuron firing order and baseline firing rate (Spearman rank correlation, r = -0.03, p = 1.0, Supplementary Fig. 5J).
(A) Time-frequency plot of the average baseline normalized (local baseline = 1) oscillatory power increases surrounding assembly reactivation events, demonstrating two peaks centered at 40 and 100 Hz. (B) Proportions of assembly member (purple) and non-member (red) neurons significantly phase locked to the underlying 40 Hz oscillation during memory behavior. Assembly members are more frequently phase locked to 40 Hz (χ2(1) = 5.25, p = 0.022). (C) Box plots demonstrating the distribution of z-scores obtained by performing a Rayleigh test on the spike phase distributions of all assembly member (purple) and non-member (red) distribution. Assembly members demonstrate stronger phase locking (z(305) = 1.74, p = 0.041). (D) Analysis controlling for the effect of firing rate on the strength of phase locking. The phase distributions of all neurons were randomly downsampled such that all neurons had the same number of spikes contributing to the Rayleigh test. Positive values indicated that the median Rayleigh z-score of assembly member neurons exceed that of non-members (t(199) = 7.24, p < 0.001). (E) Comparison of the baseline firing rates of assembly member and non-members (z(305) = 0.26, p = 0.80). (F)-(I) Two additional examples of assemblies with member neurons colored according to the observed firing order within reactivation events. (F)-(I) follow the same convention as Fig. 2A-D. (J) Distribution of Spearman correlations between assembly neuron firing rate and assembly neuron firing order demonstrating no relationship. Each data point represents the correlation for one assembly. If an assembly had more than one possible sequence that maximized MI (n = 1), we took the average of correlation across all possible sequences. (K) Additional example of the relationship between firing order and preferred phase of 40 Hz phase locking for assembly member neurons. The plot follows the description for Fig. 3C. n.s. = not significant. *p < 0.05. ***p < 0.001.
(A) Schematics of 3 example assemblies with member neuron colored according to the expected firing order within reactivation events. (B) Cumulative distribution functions of spike times within reactivation events for each assembly member neuron, colored according to the key displayed in (A). (C) Spike rasters of assembly member neurons within 5 example reactivation events demonstrating the expected firing sequence. (D) Comparison of MI for each of the 3 assemblies from (A) to a null distribution obtained by shuffling spike times within reactivation events. (E) Distribution of z-scores obtained by comparing the actual MI to the null distribution for each assembly. (F) Comparison of the average MI across assemblies to that of each shuffle. (G) Positive Spearman correlation between MI and overall recall fraction. (H) Comparison of the observed MI-recall fraction correlation to a null distribution obtained by shuffling spike times before recalculating. *p < 0.05. **p < 0.01. ***p < 0.001.
Assembly firing sequences are organized by underlying gamma oscillations
Motivated by the theoretical1-3 and observed23, 24 importance of gamma oscillations in organizing assemblies and spatial information respectively, we next evaluated the influence of gamma oscillations (40 Hz) on organizing the neuronal spiking into assemblies. During assembly reactivation events, we observed a peak in gamma oscillatory power at 40 Hz, along with a concomitant increase in power at 100 Hz, possibly reflecting ripple-like or multi-unit activity25-27, supporting the presence of gamma oscillations during these events (Supplementary Fig. 5A). The 40 Hz gamma power change reflected a significant increase from baseline (p < 0.05 after Bonferroni correction across time, Fig. 3F). We assessed the influence of this oscillation on assemblies by measuring the phase locking of member neurons relative to this oscillation. Further, 75% of assembly member neurons (63 of 84, binomial test, p < 0.001) showed significant gamma phase locking (Rayleigh test, p < 0.05, Fig. 3A-B). Assembly member neurons were both more frequently (χ2(1) = 5.25, p = 0.022, Supplementary Fig. 5B) and strongly (z(305) = 1.74, p = 0.041, Supplementary Fig. 5C-E) phase-locked to 40 Hz than non-member neurons. Member neurons within the same assembly tended to lock to different phases of the underlying oscillation, with neurons firing later in the observed sequences (as defined by the firing order, see above) spiking later within the gamma cycle compared to neurons firing earlier in the observed sequences (Fig. 3C and Supplementary Fig. 5K). This is illustrated in Fig 3E, which shows that the median phase difference between all pairs of gamma phased-locked member neurons was non-uniform with a positive (non-zero) phase offset (median phase difference = 47°, Rayleigh test, z(29) = 4.7, p = 0.0082, Fig. 3D-E).
(A) LFP (top), gamma frequency band activity (band-passed signal at 30-50 Hz, for display only), and spike raster for two example assembly member neurons demonstrating significant phase locking to the underlying gamma oscillation at 40 Hz. (B) Circular histograms demonstrating the probability density within each of the displayed phase bins for 2 example 40 Hz phase-locked assembly member neurons. Colors indicate that the neurons were isolated from the same recordings as the signal samples displayed in (A). The darkened line is at an angle equal to the median phase across all spikes and the length is equal to the mean resultant vector length. (C) Three examples of the relationship between firing order and preferred phase of spiking along the underlying 40 Hz oscillation. Phase histograms for each gamma phase-locked assembly member neuron for each assembly are superimposed. Median phase and mean resultant vector length are displayed for each neuron as in (B). Neurons firing in later in the observed sequences (o2, o3) fire at gamma phases increasingly advanced from the phase of the primary neuron (o1) in the sequence. (D) Median pairwise phase offset from all neuron pairs of each assembly displayed in (C). Each median pairwise phase offset is colored according to the corresponding assembly from (C). (E) Demonstration of the non-uniformity of pairwise phase offsets across assemblies. Median pairwise phase offset and mean resultant vector length are displayed as in (B). (F) Significant gamma power occurs at the time of reactivation of gamma timescale assemblies. Black dots above a given time point reflects significant increases in 40 Hz power at that time after Bonferroni correction across time with an alpha of 0.05. The central line represents the mean z-scored power across assemblies and the surrounding shaded region represents the 95% confidence interval. *p < 0.05. **p < 0.01. ***p < 0.001.
The flexible adjustment of neuron contribution to assemblies supports memory formation
Recent theoretical work posits that flexible assembly participation may be critical for episodic representation, especially “drift” of assembly activation over scales of minutes to hours9. This idea is also supported by experiments demonstrating dynamically changing neuronal codes for place7 and time8 in the hippocampus. Building on these ideas, we defined a metric, drift fraction (DF), that encapsulates the degree to which member neurons drift out of, and non-member neurons drift into, our observed assemblies. We calculated DF by looking for significant correlations between reactivation event number across the recording session and the firing rate of each neuron within those reactivation events (Methods). For example, if an assembly member neuron demonstrated significantly decreasing firing rate across reactivation events (over the approximately 30-minute recording session, top two rows of Fig. 4A), it was said to be drifting out of the assembly. The higher the DF, the greater the turnover in assembly member neurons (Fig. 4A and 4H and Supplementary Fig. 6A). We observed an overall DF of 21% across assemblies, higher than that observed in shuffle-generated null distributions (p < 0.001, Fig. 4B and Supplementary Fig. 6B). Interestingly, rather than these assembly alterations disrupting memory activity, assemblies with higher drift fraction correlated with greater memory recall performance (Spearman rank correlation, r = 0.47, p = 0.0011, permutation test, p < 0.001, Fig. 4C-D). To ensure this result did not stem from non-stationarity in our recordings (Methods), we compared the stability of assembly member and non-member neurons based on both spike waveform (z(305) = -0.06, p = 0.95) and firing rate (z(305) = 1.69, p = 0.091), observing no difference. Critically, assembly drift fraction did not correlate with the proportion of non-stationary neurons recorded in the corresponding session (r = -0.06, p = 0.69, Supplementary Fig. 6C-F). Further, using sessions with sampling from both the anterior and posterior hippocampus, we observed clear differences in drift fraction along the hippocampal longitudinal axis, with greater turnover in the posterior hippocampus (anterior DF = 26%, posterior DF = 38%, t(20) = -2.4, p = 0.012, permutation tests following shuffle of reactivation bin order, p < 0.001, permutation test following shuffle of region label, p = 0.0070, Fig. 4E-H).
(A) An additional example of assembly drift, following the same convention as described in Fig. 4A. (B) Histogram of DFs observed across assemblies. The line represents the overall drift fraction across all assemblies. (C) Comparison of the fractions of assembly member (blue) and non-member (red) neurons that are significantly stationary by the augmented Dickey-Fuller test. The populations did not significantly differ in the proportion of stable units by spike waveform (χ2(1) = 0.02, p = 0.88) or firing rate (χ2(1) = 0.34, p = 0.56) criteria. (D) Cumulative distribution functions for the spiking activity of all assembly member (blue, top) and non-member (red, bottom) neurons across the recording session. (E) Demonstration of the lack of a correlation between assembly drift fraction and the number of non-stationary units by firing rate criteria isolated in the corresponding session (r = - 0.06, p = 0.69). Thus, non-stationarity in neuron firing rate is unlikely to explain the drift in neuronal contribution to assemblies across the experiment. (F) Histograms of augmented Dickey-Fuller statistics for assembly member (blue) and non-member (red) neurons using firing rate (top) and spike waveform criteria. The critical value above which the unit is not considered significantly stationary is displayed as a vertical black line. The distributions of test statistics do not differ for waveform (z(305) = -0.06, p = 0.95) or firing rate (z(305) = 1.69, p = 0.091) criteria. Thus, recording instability is unlikely to explain assembly identification. n.s. = not significant.
(A) Top, schematic of the anterior (AH) and posterior (PH) hippocampus, entorhinal cortex (EC), amygdala (AMY), and parahippocampal gyrus (PHG) with all recorded neurons from an example session superimposed. Exact neuron location is for illustration only. Both member (M) and non-member (NM) neurons are displayed for an example assembly and neurons demonstrating drift across reactivation events (+d) are noted. The drift fraction is displayed above the schematic. Bottom, firing rate of 4 example neurons from the recording session across all assembly reactivation events, showing examples of neurons that decrease (red) and increase (blue) their contribution and that remain stable (gray) across the session. For display purposes only, the firing rate curve is smoothed with a 10 s window and normalized and set the value at time zero to 0. The anatomical region and category of neuron corresponding to the displayed data is shown to the left of the plots. (B) Comparison of the observed drift fraction (DF) across all assemblies to a null distribution obtained by shuffling the reactivation bin order for each neuron before re-classifying neurons. (C) Positive correlation between the drift fraction and percentage of words recalled across the experiment. (D) Comparison of the observed DF-memory correlation to a null distribution obtained by shuffling reactivation bin order within each cell. (E) Pairwise comparison of DF between the AH and PH with the overall DF for each region displayed as the bar height and each line representing a single assembly. (F) Comparison of the effect size of the observed pairwise DF difference along the hippocampal longitudinal axis to a null distribution generated by shuffling anatomical labels. (G) The same as (F) but with the null distribution created by shuffling reactivation bins. (H) An example of asymmetric drift between the AH and PH, following the same convention as outlined in (A). However, AH and PH DFs are displayed separately above the schematic.
Discussion
Neuronal assemblies have been extensively studied in rodents1, 6, 11-17 as a critical functional unit of brain activity2, 4. We identify such assemblies and correlate their activity with human episodic memory behavior, connecting assemblies with key aspects of human cognition. While we observe the preeminence of the 25 ms timescale among those we tested, as expected1, 2, we also identified significant assembly activity occurring at other time scales (Supplementary Fig. 2). Further work is needed to elucidate how different gamma timescale assemblies interact with each other with longer lasting neuronal co-activity, such as that defined by the period of a theta oscillation, as well as preferred information types of different assembly time scales (contrasting spatial and episodic information, for example)2, 3. We demonstrate that 25 ms assemblies reflect consistent sequences of single neuron activity occurring within gamma oscillations (Fig. 2). The majority of assembly member neurons demonstrate gamma phase locking at 40 Hz, but to different phases (Fig. 3E), possibly giving rise to the observed firing sequences (Fig. 3C). Interestingly, both the reliability of these sequences across task performance and reactivation rate of assemblies positively correlate with successful memory (Fig. 2G and Supplementary Fig. 2C-D). Adopting the analogy employed by Buzsáki in describing the importance of gamma assemblies, our findings suggest that if assemblies can be thought of as the words the brain uses to build neural sentences that represent complex thought2, then both the volume (reactivation) and spelling (activation order) of each individual word matters. We note that co-activity within gamma oscillations is not equivalent to strengthened physical connections by increased spine density thought to underlie assemblies28, though the former may induce the latter via spike timing dependent plasticity10. Testing this in humans may be facilitated by ex vivo recordings of human tissue or pathological analysis of surgically resected tissue recorded from in vivo29. Further characterization of the temporal dynamics of assembly formation can build on our findings related to assembly drift. We found evidence for both stable and drifting assemblies (Supplementary Fig. 5B). However, participants with drifting assemblies demonstrated superior memory performance, supporting recent suggestions that dynamic assemblies may facilitate memory by allowing for flexible updating of experiential detail9. The co-existence of both drifting and non-drifting assemblies is supported by theoretical9 and empirical6 work. Memory testing over long time scales (several days) could begin to probe how these classes of assemblies cooperate during human memory behavior. Interestingly, our finding that drift occurs more frequently in the posterior, compared to the anterior, hippocampus (Fig. 4E) offers a potential connection between theories of hippocampal longitudinal specialization related to “gist” versus “detailed” representations30 and a proposed role for assembly drift to support memory updating9. It is possible that the anterior hippocampus maintains foundational features of an experience while the posterior hippocampus updates the immediately-relevant representation as new events occur. This finding may have eventual implications for diseases that demonstrate selective abnormalities along the anterior-posterior hippocampal axis such as post-traumatic stress disorder30.
Given the experimental paradigm we used to identify assemblies (“single shot” encoding of non-repeated items), a natural question to ask is: what type of information do these assemblies represent2? Assembly activity occurs flexibly across the recording session, so they are not directly analogous to putative “engrams”19, the activity of which can decode individual and specific memory items. Comparison of assemblies defined on gamma time scales and these broader representations may be facilitated by paradigms with a mix of repeated and non-repeated memory items. The assemblies we report exhibit clear associations with memory performance, and they demonstrate similarities to place cell and time cell population activity in terms of assembly drift. Therefore, it is possible that these assemblies represent the mnemonically relevant contextual features of the overall experimental experience, much like place and time cell ensembles. While additional experimentation will be required to differentiate item and context representations at the assembly level, the fact that morphological (Supplementary Fig. 2D), anatomical (Fig. 4E), sequential (Fig. 2G), and temporal (Fig. 4C) features of the assemblies are all implicated in memory performance suggests assembly identification on gamma time scales represents an effective way to understand MTL activity during mnemonic representations.
The identification of mnemonically relevant human MTL assemblies harbors implications for neuromodulation strategies. Hampson et al.31 describe the use of multi-input, multi-output state space models to provide a template for artificial recreation of spiking activity using intracranial electrodes. Our findings establish principles for identifying possible targets for neuromodulation using these approaches, as well the favorable time scales for co-firing and updating/modifying assemblies over the time scale of minutes to hours. Further, the time scale for assembly organization suggests an effective strategy for neuromodulation of memory may include the union of microstimulation using assembly organizational templates with macrostimulation designed to alter hippocampal gamma activity32. However, the importance of assembly identification may extend beyond mnemonic processing, including strategies for the amelioration of psychiatric illness33.
Materials and Methods
Subjects and electrophysiology
We enrolled 26 human epilepsy patients who completed a total of 38 recording sessions for the study. We have previously described this population, and this work follows the same procedures5. All participants were consented to a protocol approved by the institutional review boards at Thomas Jefferson University Hospital (TJUH) and University of Texas Southwestern Medical Center (UTSW). Of the 26 human participants enrolled, 13, completing a total of 15 recording sessions, had adequate single unit yield to identify assemblies and formed the focus of the study. Participants were implanted with Behnke-Fried electrodes allowing for identification of both local field potential (LFP) activity and single units, as previously described5. Neuroradiologists confirmed anatomical locations after co-registration of the pre-operative MRI with the post-operative CT5. We used BrainNet Viewer34 to visualize electrode localizations.
Data pre-processing35 and spike sorting36 followed the same process as our previous report5. We only considered neurons with a baseline firing rate of at least 0.5 Hz. This resulted in a total of 307 neurons (hippocampus = 270, entorhinal cortex = 22, amygdala = 7, and parahippocampal gyrus = 8), 203 of which coming from sessions in which we isolated assemblies. For each neuron, we calculated spike quality metrics including percentage of action potentials violating a 3 ms inter-spike interval (ISI), ratio of the spike peak voltage to signal noise, number of units isolated per channel, isolation distances of units extracted from the same channel, average firing rate and burst index across recording5 (Supplementary Fig. 1B-G).
For LFP analyses, we low-passed the signal at 300 Hz. We subsequently applied a notch filter at 60 Hz and down sampled by a factor of 30 (UTSW) or 32 (TJUH) to an approximate sampling rate of 1,000 Hz. For gamma phase and power analyses, we used Morlet wavelet convolution at 40 Hz as its oscillation period matched our selected assembly timescale (25 ms)1 and we observed a peak in gamma power at this frequency coincident with assembly activity (Supplementary Fig. 5A). To remove the influence of spiking activity, we excised each spike event and bridged the two segments of data with linear interpolation.
We did not include any sessions with observable seizure activity or aura. Further, the clinical team flagged excessively noisy or inter-ictal activity laden channels5. These channels were discarded. Channels exhibiting inter-ictal activity at any point during implantation were also flagged. Though we did not exclude these channels from analysis, we ensured our ability to identify assemblies following their exclusion (Supplementary Fig. 1H).
Task
The task design is the same as in previous work5. In brief, participants studied up to 25 wordlists comprised of 12 (UTSW) or 15 (TJUH) monosyllabic nouns on a laptop computer at bedside. Word stimuli were not repeated. Each word remained on screen for 1.6 s, followed by a jittered inter-stimulus period of 0.8-1.2 s. Patients then completed two-step arithmetic problems of the form A + B + C = ?, where A, B, and C were single digit integers, for at least 20 seconds. Finally, they verbally recalled as many words as possible from the immediately preceding wordlist without receiving any cues. They had 30 (UTSW) or 45 (TJUH) seconds for recall.
Assembly identification
We isolated assemblies using a framework21 previously utilized in rodent studies11-13. For each recording session, we calculated the firing rate of each neuron within time bins spanning the experiment. We tested various timescales corresponding to the periods of previously investigated gamma and theta frequency bands. We selected 25 ms as it is both the theoretical2 and demonstrated1 (Supplementary Fig. 2) optimal timescale of assembly organization. We then normalized (z-scored) the binned firing rates for each neuron to avoid biasing the result towards neurons with higher baseline firing rates. After normalizing, we constructed a neuron-by-time bin matrix. To isolate assemblies, we used all time bins falling within all encoding events across the recording session.
Next, we performed principal component analysis (PCA) on the normalized firing rate matrix, generating an eigenvalue associated with each identified principal component. According to the Marchenko-Pastur law, eigenvalues outside of the bounds:
where n = the number of neurons and b = the number of time bins, stem from correlation between the variables (neurons) within the matrix exceeding that expected if the variables were independent. Therefore, any isolated pattern whose associated eigenvalue exceeds this upper bound is the result of significant neuronal co-firing. While it is possible to define cell assemblies on the basis of PCA patterns, previous work has demonstrated that stopping there leads to mixing of the true cell assemblies21. Therefore, following the aforementioned PCA analysis, we performed independent component analysis. To do so, we first projected the firing rate matrix onto the PCA patterns with eigenvalues exceeding the upper limit delineated by the Marchenko-Pastur law:
where Rp is the firing rate matrix projected onto the significant principal components, 
is the transpose of the matrix with the coefficients for each neuron across all significant principal components, and R is the original normalized firing rate matrix. This, in effect, forces the ICA to return only as many assemblies as there were eigenvalues exceeding the upper bound expected for independently firing neurons. We utilized the MATLAB implementation of the fastICA algorithm37 to solve for the un-mixing matrix, U, using Rp. We then used the unmixing matrix to rotate the PCA-derived assemblies, given by pco:
where W is a neuron-by-assembly matrix with each column containing the weight vector for the index assembly. The weight vector communicates the degree to which each neuron contributes to each assembly, with arbitrary scale and direction. We then transformed each weight vector into a unit vector, setting the sign such that the greatest assembly weight was positive as previously described11.
While each neuron contributes to each assembly to some degree, we defined “assembly member neurons” as those with weight vector values exceeding the mean by at least one standard deviation. We then independently confirmed that this definition isolated neurons that were both significantly more active than baseline and more active than non-member neurons during reactivation events (Supplementary Fig. 1L). We required an assembly have at least two member neurons to avoid assemblies dominated by a single recorded neuron.
We additionally calculated the complexity, referred to as sparsity in previous work11:
where wi is the weight vector value of the ith neuron, and n is the length of the weight vector (equal to the number of recorded neurons in a given session). High complexity communicates that the assembly has a high proportion of recorded neurons contributing to the assembly.
Assembly reactivation
We calculated the “expression strength,” the degree to which an assembly is active, with the following equation:
where E(b) is the expression of assembly i in the time bin of interest, b, R(b) is the firing rate matrix column corresponding to time bin b, and oi is the outer product of the weight vector of assembly i. For each assembly, we calculated the expression strength within all time bins across recording. We defined assembly “reactivation events” as time bins during which the expression strength exceeded the 95th percentile of all observed expression strengths for that assembly. Note that reactivation does not refer to memory retrieval.
Assembly identification controls
The PCA/ICA assembly identification framework depends on the assumption that a matrix storing the firing rates across time of a group of independently firing neurons will not have an eigenvalue exceeding the Marchenko-Pastur limits. However, matrices storing values of a parameter whose distribution is heavily skewed may result in eigenvalues that spuriously exceed these limits38, 39. This is a theoretical risk in firing rate data as they are classically Poisson distributed. However, the risk of tempered by the size of the matrix used for PCA38. Further, eigenvalues beyond the expected bounds in these cases result from one or more instances of extreme values within the matrix. To reduce the risk of extreme values, we limited our analysis to neurons with a baseline firing rate of at least 0.5 Hz.
However, to confirm that the identified assemblies could not be explained by statistical error, we implemented a shuffle control. For this, we randomly shuffled the spike trains of all neurons before re-isolating assemblies. We then compared the actual number of assemblies to this null distribution, noting that the observed count exceeded the shuffle counts in every case (Fig. 1G).
We calculated a spatial distribution score to capture the degree to which assemblies clustered spatially (Supplementary Fig. 1I). To do so, for each assembly, we noted the channel, region, and hemisphere of each assembly member neuron. We then took all pairs of assembly member neurons. If the neuron pair shared the same channel, it received a score of 1. If it shared the same location (for example, both in the entorhinal cortex), it received a score of 2. If it shared the same hemisphere, it received a score of 3. If the neurons were from different hemispheres, it received a score of 4. We summed across all neuron pairs to obtain a single distribution score for each assembly, with lower scores representing more localized assemblies. We then randomly selected neurons from the same recording session, irrespective of assembly membership, to generate a null distribution of distribution scores. If an assembly had four neurons, we randomly selected four neurons for each iteration. We then compared the actual distribution score to the null distribution, generating a z-score for each assembly. We tested across assemblies against zero with a two-sided t-test to assess if observed assemblies were more localized or distributed than expected by chance. That assemblies were not more localized than expected by chance decreases the likelihood that assemblies arose from noisy recordings from a given channel or microwire.
Intra-assembly neuronal sequences
To identify firing sequences within assembly reactivation events, we adapted a pair matching framework previously employed in the rodent place cell sequence replay literature22. We only used assembly member neurons for this analysis. In contrast to place cell replay where the expected firing sequence is known (the sequence observed during behavior), we did not have an expected firing sequence before searching for sequence development. Therefore, for each assembly, we generated a list of all possible orderings of neuron firing. For example, if an assembly had only two member neurons, the sequence set was simply [1-2, 2-1]. We then used each possible sequence as a template, measuring how well the observed firing sequences matched each template.
We looked within all reactivation events to compile a list of all observed ordered pairs of assembly neuron firing. For example, an assembly with three neurons which all fired in a reactivation event in the observed order 2-1-3 generated the ordered pair set of [2-1, 2-3, 1-3]. We calculated the fraction of ordered pairs that matched each template sequence (defined as “matching index,” MI), selecting the template sequence with the expected sequence for that assembly. Rare instances of simultaneous assembly member spiking were not considered.
Due to this procedure, the minimum MI for an identified expected sequence was 0.5. Therefore, we performed permutation testing to define the statistical significance of an observed MI. We did this by repeating the aforementioned procedure 1,000 times after shuffling the spike times within reactivation events each time. We then compared the actual MI to the null distribution of MI values to generate a z-score as follows:
where MIs is the distribution of shuffled MI values and MIr is the MI value obtained from unshuffled data. We used the equation within parenthesis to generate p-values from shuffled data throughout the manuscript. We compared the distribution of z-scores, one for each assembly, to zero with a one-sided t-test (Fig. 2E). We also compared the average MI across assemblies to the distribution of those seen in this null distribution (Fig. 2F).
We calculated the Spearman rank correlation between assembly MI values and the recall fraction from the corresponding session, the ratio of words recalled to words studied across experimentation. We compared the observed correlation coefficient to a distribution of coefficients obtained following shuffling of the spike times within reactivation events, as described above. As a further control, we calculated a partial Spearman rank correlation between assembly MI values and the recall fraction, using each assembly’s rate-coded subsequent memory effect (see below) and session-wide average expression strength, markers of assembly activity during successful encoding events and across the session, respectively, as the controlling variable.
Assembly reactivation-triggered gamma power
We extracted the power at each of 75 linearly spaced frequencies ranging from 2 to 150 Hz for the 500 ms (−250 ms to +250 ms) surrounding each reactivation event, utilizing Morlet wavelets. After extracting raw power, we log-transformed and z-scored the values to allow for averaging across channels. For each assembly, we extracted z-scored power for each channel recording at least one assembly neuron. We then averaged across channels and reactivation events to yield a single z-scored power time series for each assembly. We then performed a t-test across assemblies, comparing the averaged z-scored power across assemblies to zero (session-wide baseline power) for all time-frequency points. This procedure at 40 Hz generated the plot seen in Fig. 3, where we Bonferroni-corrected across time to identify significant deviations from baseline. To identify which frequencies were maximally coincident with assembly activity, we transformed the p-value at each time-frequency point obtained by the t-test and normalized the power values observed in the 100 ms surrounding assembly reactivation (−50 to +50 ms) by the average power values observed at each frequency in the 400 ms surrounding this epoch (−250 ms to +250 ms). This procedure identified two peaks in baseline-normalized assembly reactivation-triggered power: 40 Hz and 100 Hz. We selected the former for further analysis due to its theoretical role in organizing 25 ms assemblies1, 2.
Gamma phase-locking neurons
As we identified co-firing activity on the timescale of 25 ms, the period of a 40 Hz gamma oscillation, and noted a peak in the assembly reactivation-triggered power at 40 Hz (Supplementary Fig. 5A), we used Morlet wavelets at 40 Hz to extract gamma phase and power. Preprocessing of the LFP is described above. We extracted the gamma phase at each spike time for each assembly member neuron. Then, for each neuron, we tested the phase distributions for non-uniformity with a Rayleigh test. We called a member neuron gamma phase-locked if the p-value from the Rayleigh test was below 0.05. We tested for an above chance proportion of gamma phase-locked neurons with a one-sided binomial test with a chance rate of 0.05.
Next, for each assembly, we identified all member neurons that demonstrated gamma phase locking. Then, for each neuron, we calculated the median phase of all spike phases observed during memory behavior. We then calculated the median pairwise phase difference between all member neurons for each assembly. We calculated the pairwise phase difference in such a way that positive differences communicated that neurons later in the expected firing sequence fired at a later phase. We tested the distribution of median pairwise phase differences for non-uniformity with a Rayleigh test (Fig. 3E).
We compared both the proportion and strength of gamma phase locking between assembly member and non-member populations. We compared the proportion of gamma phase locking neurons between groups with a chi-square test. We compared the strength of phase locking by assessing for a difference in the Rayleigh Z distributions of the two groups with a rank sum test. Additionally, we controlled for differences in firing rate between the two groups by first testing for a difference in baseline firing rate between them with a rank sum test. Next, we randomly downsampled the phases contributing to the Rayleigh test for each neuron, such that all assembly members and non-members had numbers of contributing spikes. We then took the median of all obtained Rayleigh Z values for both groups and compared them. We repeated this process 200 times and assessed for a deviation in groupwise difference with a one-sample t-test against zero (Supplementary Fig. 5D).
Rate-coded subsequent memory effect
We calculated the reactivation rate during each encoding event by counting the number of reactivation events and dividing by the duration of the encoding event, 1.6 s. We then assessed for a subsequent memory effect (SME) for each assembly by comparing the reactivation rates during successful encoding events (SE) to those observed during unsuccessful encoding events (UE) with a one-sided rank sum test. This generated a z-score communicating the degree to which assemblies increased their firing rate during SE compared to UE events. We tested for an effect at the assembly level with a one-sided t-test against zero. We additionally performed a Spearman rank correlation between the SME z-scores and the complexity scores for each assembly.
We controlled the observed SME effect and SME-complexity correlation by recalculating SME z-scores 1,000 times, shuffling the event labels each time. The shuffle procedure preserved the number of SE and UE events within the experiment but randomly reassigned the encoding events to each group.
Assembly drift
We defined a metric, drift fraction (DF), to capture the degree to which the neurons contributing to assembly activation changed across recording. For each assembly, we extracted the z-scored firing rates for all recorded neurons during each assembly reactivation event. We performed Spearman rank correlations between the z-scored firing rate and reactivation bin number for each neuron. Members with significant correlations (p < 0.05) and negative correlation coefficients were said to be “drifting out” of the assembly. Non-member neurons with significant positive correlations (p < 0.05, r > 0) were said to be drifting into the assembly. We defined the DF as the number of neurons drifting in and out of the assembly divided by the total number of neurons recorded.
We calculated the overall drift fraction by summing the numerators and denominators from all individual assembly DF calculations. We compared this value to a shuffle-generated null distribution to ameliorate concern of bias in our correlation procedure. We generated this distribution by randomly shuffling the reactivation bin order before re-calculating the z-scored firing rate-reactivation bin correlation for each neuron (Fig. 4B). We next calculated the Spearman rank correlation between assembly DF and recall fraction. We compared the actual value to a distribution of correlation coefficients obtained by randomly shuffling the reactivation bins before repeating the procedure.
We investigated the possible influence of recording drift on our assembly identification and drift analysis. For this, we performed an augmented Dickey-Fuller test on spike waveform5 and firing rate data for each assembly member and non-member neuron. For the spike waveform analysis, we first averaged the first 5 percent of all spikes observed during the experiment. We then calculated the Euclidian distance between this average waveform to all spikes observed throughout the remainder of the experiment. We used the vector of distance values for the augmented Dickey-Fuller test. For the firing rate analysis, we binned the spiking activity across the experiment into 100 equal duration time bins, calculating the firing rate for each bin. We then used this vector of firing rate values for the statistical procedure. We used both metrics to compare the stationarity of assembly member and non-member neurons with a rank sum test. Further, we correlated the fraction of units in a session potentially non-stationary by firing rate criteria with the observed drift fractions of assemblies isolated from the corresponding sessions.
Longitudinal differences in assembly drift
Following previous observation that the posterior hippocampus (PH) is more detail-oriented than the anterior hippocampus30 (AH) and that drift may reflect memory updating9, we compared the DF observed within the AH and PH. We did this by identifying all assemblies with sampling from both the AH and PH (n = 21 of 45). We then calculated the DF for the AH and PH separately. Next, we performed a paired one-sided t-test (Fig. 4E). We controlled this result by generating two shuffled-derived null distributions. For the first (Fig. 4F), with each session, we randomly shuffled the anatomical labels of neurons before re-calculating the AH and PH drift fractions. For the second (Fig. 4G), we randomly shuffled the reactivation event order for each assembly before re-calculating AH and PH drift fractions.
Statistical Testing
Unless otherwise specified, we used the Anderson-Darling test to assess for normality before using parametric tests. Otherwise, we used non-parametric statistics. We used an alpha of 0.05 unless otherwise noted. All other relevant statistical information can be found located under the appropriate heading.
Author contribution
G.U. and B.L. conceptualized the study. G.U., J.J., B.E.P., and B.L. developed the methodology. R.T. and B.L. performed the investigation. G.U. generated data visualization. J.J., B.E.P, and B.L. acquired funding. B.L. administered and supervised the project. G.U. and B.L. wrote the original draft. G.U., J.J., B.E.P., and B.L. reviewed and edited the draft.
Competing interests
The authors declare no competing interests.
Supplementary Information
contains 5 figures and 1 table.
Acknowledgments
We thank Michael D. Rugg for the helpful remarks on the manuscript. We are thankful for each patient who participated in the study. The project was supported by NIH R01NS107357 to B.L. and R01-MH104606 to J.J. as well as the Southwestern Medical Foundation (B.E.P.) and the THR Clinical Scholars Program (B.L.).
