Locally distributed abstraction of temporal distance in human parietal cortex

An enduring puzzle in the neuroscience of memory is how the brain parsimoniously situates past events by their order in relation to time. By combining functional MRI, and representational similarity analysis, we reveal a multivoxel representation of time intervals separating pairs of episodic event-moments in the posterior medial memory system, especially when the events were experienced within a similar temporal context. We further show such multivoxel representations to be vulnerable to disruption through targeted repetitive transcranial magnetic stimulation and that perturbation to the mnemonic abstraction alters the neural—behavior relationship across the wider parietal memory network. Our findings establish a mnemonic “pattern-based” code of temporal distances in the human brain, a fundamental neural mechanism for supporting the temporal structure of past events, assigning the precuneus as a locus of flexibly effecting the manipulation of physical time during episodic memory retrieval.

We further show such multivoxel representations to be vulnerable to disruption through 23 targeted repetitive transcranial magnetic stimulation and that perturbation to the 24 mnemonic abstraction alters the neural--behavior relationship across the wider parietal 25 memory network. Our findings establish a mnemonic "pattern-based" code of temporal 26 distances in the human brain, a fundamental neural mechanism for supporting the 27 temporal structure of past events, assigning the precuneus as a locus of flexibly 28 effecting the manipulation of physical time during episodic memory retrieval.  Representations of brief elapsed time can be inferred from single neuron activities 41 in the primate brain (4-6). Time-registering neurons are found to code time with high 42 precision in the cortico-basal ganglia circuits (5), inferior parietal cortex (4) and medial 43 temporal lobe (6) across short timescales. Recent work in rats has provided evidence 44 that temporal information is encoded across time scales from seconds to hours within 45 the overall population state of the lateral entorhinal cortex (7). In contrast, when 46 complex, coherent experiences become consolidated into long-term memories (8), the 47 neural circuits that build time representations as an infrastructure for episodic retrieval 48 are theorized to be distinct from those implicated in hippocampal-dependent encoding 49 (9, 10) and retrieval (11,12), and from those during transient temporal processing (4-50 6). For the recollection of long-term autobiographical memories or episodic events, the 51 posterior medial (PM) memory system, including hippocampus, precuneus and angular 52 gyrus, plays an instrumental role (13). Event representations in these regions 53 generalized across modalities (e.g., EEG and MRI) and domains (e.g., perception and 54 memory) (14-16). Temporal representation is intertwined with the construct of context. A prominent 56 memory model posits that item representations are linked to a changing "context" at 57 encoding, such that a common retrieved context is triggered during recall for items that 58 were experienced within a similar temporal context (17). However, the critical issue of 59 how elapsed time between pairs of long-term episodic events -and its interplay with 60 the encoding context -is represented by the PM system has yet to be addressed. Here 61 we investigated the abstraction, at a macro-anatomical level, of temporal distances that 62 were encoded more than 24 hours previously (18,19), and determined how several 63 members of this large cortical system are differentially implicated in this putative 64 mnemonic function (20). 65 Combining functional magnetic resonance imaging (fMRI) with an interactive-66 video memory paradigm and a temporal order judgement task (TOJ; Fig 1A)-a 67 validated paradigm to study neural correlates underpinning temporal distances between 68 units of memory traces (10, 18, 19)-we adopted a two-forked protocol to ascertain 69 how temporal distances separating pairs of past moments-in-time are represented in the 70 human neocortex. On the one hand, we identified a locally distributed neural 71 representation characterizing the neural patterns of retrieving temporal distances using 72 a multivariate searchlight representational similarity analysis (RSA) (21). We 73 parametrized a large set of pairs of event-moments geometrically separated by varying 74 temporal intervals and applied RSA to compare neural representational dissimilarity 75 matrices (RDM) with a number of parametric, condition-rich hypothetical/candidate 76 models. Applied across the entire brain, the searchlight approach identifies local multi-voxel patterns driven by structured co-activation at a voxel level within the size of the 78 9-mm radius spherical searchlight, thereby giving us a snapshot of the locally 79 distributed neural architecture supporting temporal order judgements. On the other 80 hand, to enhance the causal strength of the anatomical associations thereby revealed, 81 we focally disrupted the identified critical region with repetitive transcranial magnetic 82 stimulation (rTMS, Fig 1F), seeking to confirm its functional necessity for mediating 83 the distributed representation of temporal distances. The spatial scale of rTMS-induced 84 disruption is comparable to that of our chosen searchlight, rendering it an optimal tool 85 for targeted, reversible disruption of the distributed representation of interest.

86
For memory encoding, participants played an interactive video game containing 87 seven distinct yet related chapters, each in the range of tens of minutes on day 1 (S1 88 Fig, S1 Table). By the nature of the video game, within chapter segments contained 89 more coherent narrative strands than those across chapters, yet all chapters were 90 connected by a common plot. After a 24-hour retention period (day 2), on each trial, 91 participants judged the temporal order of two images (extracted from their individually-92 played video game, Fig 1B), depicting two time-points in their encoded memory, while 93 their blood-oxygen-level-dependent (BOLD) activity was measured (TOJ task, Fig 1C). 94 Assuming a scale-free temporal memory representation (22), we manipulated the 95 between-images temporal distances (TD) for all pairs of images so that the TD 96 distribution adhered to a power function permitting scale-invariance across subjects (23) 97 (60 levels of TD, Fig 1D). To test the interaction effect between TD and its encoding 98 context, we manipulated the factor "context" by controlling whether the paired images 99 Precuneus and temporal context memory 6 presented at TOJ task were extracted from the same chapters or two adjacent chapters 100 of the video game while keeping the 60 TDs fully matched between the two conditions 101 (Within-chapter vs. Across-chapter, Fig 1E). game containing seven related chapters with a first-person perspective for encoding, and 24 106 hours later, received 20 min of repetitive transcranial magnetic stimulation (rTMS) to either 107 one of two cortical sites before performing a temporal order judgement task during fMRI. Order 108 of TMS sites (within-subjects) and choices of video game chapters were counterbalanced across 109 subjects (S1 Table). The two experimental sessions were conducted on different days to 110 minimize rTMS carry-over effects (mean separation = 8 days). Participants underwent 111 structural MRI scans and familiarized themselves with the gameplay using a console prior to (Within-chapter) or two adjacent chapters (Across-chapter). The 60 levels of TD were fully 120 matched within-subjects for these two conditions. Note that scenes depicted in Within-chapter 121 tended to be more contextually similar than those depicted in Across-chapter. (F) TMS 122 stimulation sites, superimposed onto one subject's MRI-reconstructed skull, are marked by a 123 green pointer. The MNI coordinates for precuneus stimulation: x, y, z = 6, -70, 44. 124 125

126
Behavioral results 127 We first looked into the interaction effect between TD and encoding context and 128 the TMS effect on memory retrieval. We collapsed the 60 TD conditions into two levels (short vs. long) for each subject and analyzed the behavioral performance of TOJ 130 (dependent variable: accuracy or reaction times or confidence level) as a function of 131 TMS stimulation site, Context and TD. We ran a three-way repeated-measures ANOVA 132 (TD: Short/Long × Context: Within-chapter/Across-chapter × TMS: TMS-133 vertex/TMS-precuneus) on task accuracy, and obtained a significant main effect of TD 134 (F(1, 16) = 25.53, P < 0.001, η 2 = 10.02%) and a significant two-way interaction effect 135 between TD and Context (F(1, 16) = 5.97, P = 0.026, η 2 = 2.71%). Such interaction effect 136 was driven by a significant difference in accuracy between short and long TD in Within-137 chapter condition (t(33) = 5.94, P < 0.001), but not in Across-chapter condition (t(33) = 138 1.61, P = 0.117) (Fig 2A). 139 A similar two-way interaction effect was found in reaction times (F(1, 16) = 24.21, 140 P < 0.001, η 2 = 1.16%), with longer RT in short than in long TD in Within-chapter 141 condition (t(33) = -3.33, P = 0.002) but longer RT in long TD condition in Across-chapter 142 condition (t(33) = 2.83, P = 0.008) (Fig 2B). No three-way interaction effects were found  order of a pair of memories separated with a given temporal distance is more similar to 166 other temporal order judgements which also enclosed temporal distances of a comparable scale. These voxels were in the posteromedial parietal areas, bilateral 168 angular gyri, and middle frontal gyri (Fig 3B, S2 Table). 169 The temporal-distance memory representation could be confounded by perceptual 170 similarity in each pair of images. To address this concern, we conducted six separate 171 RSAs, in which we indexed perceptual similarity between the image-pairs by six 172 different metrics, some drawn from the visual categorization models referred by Greene    Having identified a multivariate pattern underlying the temporal memory 196 abstraction in the posterior medial parietal cortex, we asked further whether there were 197 voxels whose activities change monotonically as a function of temporal distance using 198 a standard univariate approach, irrespective of its multivoxel characteristics. A whole-199 brain parametric modulation analysis revealed TD-specific BOLD signals in a cluster 200 within the posteromedial region, including the precuneus (Fig 3C, S3 Table). This the precuneus jointly confirms the critical involvement of this region. We accordingly 205 created a conjunction map (Fig 3D), so that both the multivariate and univariate results 206 underlying the temporal distance abstraction would be available for the next analysis. 207 We then performed inference tests to statistically assess whether the TD RDM is RDMs and two behavioral RDMs (Fig 3E-F). These results showed that the neural 215 signals coded in the posteromedial region as revealed by these searchlight analyses 216 were most attributable to the mnemonic representation of temporal distances. Within-chapter and Across-chapter conditions (Fig 4B). The TD model accounted for 264 the variances far better than the other candidate RDMs in the Within-chapter condition (Fig 4B, left). By contrast, the TD RDM (and all other candidate RDMs) failed to 266 explain the neural reference RDM in the Across-chapter condition (Fig 4B, right).

267
This context-dependent difference was also found in a voxel-wise univariate 268 analysis. The beta-estimates (β) from a pmod analysis using TD as a regressor were 269 significantly higher in the Within-chapter condition compared to the Across-chapter 270 condition (Fig 4C). These results were consistent in a control analysis while RT were temporally-and semantically-defined factors observed during memory encoding (9) 275 and retrieval (12).  Table). We found the widespread 293 representation of TD disappeared following TMS on the precuneus, either considering 294 the Within-chapter condition (Fig 5A) or collapsing Within-chapter and Across-chapter 295 conditions. Model comparison results showed that the correlation values between all 296 the candidate RDMs and neural reference RDM now failed to reach statistical 297 significance even when only considering Within-chapter trials alone (Fig 5B). In 298 contrast, the activation-based Pmod analyses showed that TMS to the precuneus did not    showed that BOLD intensity of posteromedial cortex, especially the precuneus, varied 343 as a function of temporal distance. We showed that such information carried by these 344 regions is more pronounced within a similar context and such representations reduced 345 significantly after the precuneus was perturbed. We further revealed that the precuneal 346 representation of TD is associated with subjects' memory performance, especially 347 when two images for temporal order judgment were extracted from the same context.     (power = 1.5), at the same time ensuring the shortest TD to be longer than 30 frames. 465 We generated 60 progressive levels of TD among these pairs (each level repeated twice).

466
In sum, three within-subjects factors regarding the TOJ retrieval task were manipulated:

467
(1) 60 TD levels permitting scale-invariance across subjects between two images (see at the left hemisphere. The TMS coil was systematically moved until the optimal 504 cortical site was located to induce the largest and most reliable motor response; this 505 stimulus output was then recorded. The TMS intensity was then calibrated at 110% of 506 individual active motor threshold (stimulator output: 75.2 ± 6.9%, mean ± se, range 507 from 63% to 88%, S1 Table). In Experimental Session 1 and 2, the TMS was applied 508 at a low-frequency rate of 1 Hz with an uninterrupted duration of 20 min.  interest were also included: 6 head movement regressors and 1 missing trial regressor 556 (i.e., no-response trials; number of missing trials of Across-chapter condition: 5.65 ± 6.96, of Within-chapter condition: 5.29 ± 6.8; n = 17, mean ± sd) and the run mean.

558
The purpose of this analysis was to test for any linear TD-dependent modulation of 559 signal intensity in the brain between the TD between the two images at encoding and 560 the brain activity during TOJ retrieval of the same events. For the TD + RT pmod, we 561 aimed to identify the voxels whose activities changed as a function of TD after the 562 removal of the influence of reaction times. Each subjects' RTs corresponding to each 563 TD level were entered as the modulatory parameter, together with the regressors of no 564 interest as above.

565
For the Across-chapter vs. Within-chapter comparison, we also performed two 566 pmod analyses with identical sets of regressors as described above (namely, TD; TD + 567 RT). We looked for changes in brain responses as a linear function of the regressor of candidate RDMs (see Fig 3F), reflecting different predictions of the information carried 593 by similarity structure of neural signal responses and generated correlational maps (r-594 maps). Finally, these r-maps were converted to z-maps using Fisher transformation. All 595 the z-maps were then submitted to a group-level one-sample t-test to identify voxels in 596 which the similarity between the predicted RDM and observed neural RDM was greater 597 than zero. This allowed us to identify voxels in which information of TD at retrieval 598 might be represented (see S3 Fig). The statistical threshold was set as identical to those 599 employed in the univariate analysis, which was at p < 0.05 (FWE corrected) at cluster 600 level and p < 0.001 at an uncorrected peak level.
Leave-one-subject-out approach (LOSO), functional and anatomical ROIs. We 602 applied a LOSO approach to create functional ROIs to avoid statistical bias (52). For 603 instance, in order to identify an ROI (i.e., conjunction mask in Fig 3D) for Subj01, we 604 estimated the contrast using a one-sample t-test on the whole-brain searchlight z-maps 605 obtained from Subj02 to Subj17. Likewise, we also estimated the contrast using a one-606 sample t-test on the contrast maps obtained from the Pmod analysis of Subj02 to Subj17. 607 We set the same threshold reported above to extract clusters from these two statistical   Note. † denotes a negative relationship between TD and brain activity. 806 807 808 S1 Video. An excerpt of the gameplay video.