Abstract
The brain encodes the statistical regularities of the environment in a task-specific yet flexible and generalizable format. How it does so remains poorly understood. Here, we seek to understand this by converging two parallel lines of research, one centered on striatal-dependent sensorimotor timing, and the other on hippocampal-dependent cognitive mapping. We combined functional magnetic resonance imaging (fMRI) with a visual-tracking and time-to-contact (TTC) estimation task, revealing the widespread brain network supporting sensorimotor learning in real-time. Hippocampal and caudate activity signaled the behavioral feedback within trials and the improvements in performance across trials, suggesting that both structures encode behavior-dependent information rapidly. Critically, hippocampal learning signals generalized across tested intervals, while striatal ones did not, and together they explained both the trial-wise performance and the regression-to-the-mean biases in TTC estimation. Our results suggest that a fundamental function of hippocampal-striatal interactions may be to solve a trade-off between specificity vs. generalization, enabling the flexible and domain-general expression of human timing behavior broadly.
Introduction
When someone throws us a ball, we can anticipate its future trajectory, its speed and the time it will reach us. Our expectations then inform our motor system to plan an appropriate action to catch it. Generating expectations and planning behavior accordingly builds on our ability to learn from past experiences and to encode the statistical regularities of the tasks we perform. At the core of this ability lies a continuous perception-action loop, initially proposed for sensorimotor systems (e.g. (Wolpert et al., 2011)), which is now at the heart of many leading theories of brain function including active inference (Friston et al., 2016), predictive coding (Huang & Rao, 2011) and reinforcement learning (Daw & Dayan, 2014).
Critically, to guide behavior accurately in a dynamically changing environment, the brain needs to balance at least three objectives. First, it needs to capture the specific aspects of the task that inform the relevant behavior (e.g. the remaining time to catch the ball). Second, it needs to generalize from a limited set of examples to novel and noisy situations (e.g. by inferring how fast previous balls flew on average). Third, the sensorimotor representations that guide the behavior need to be updated flexibly whenever feedback about our actions becomes available (e.g. when we catch or miss the ball), or when the task demand changes (e.g. when someone throws us a frisbee disc instead). Herein, we refer to these objectives as specificity, generalization and flexibility. While these are all fundamental principles underlying human cognition broadly, how we learn to balance these three objectives during ongoing behavior remains unclear.
An optimal behavioral domain to study these processes is sensorimotor timing (Gershman et al., 2014; Petter et al., 2018). This is because prior work in humans and non-human primates suggested that timing estimates indeed rely on prior experiences, based on which the temporal regularities and kinematic information of the task are inferred (Wolpert et al., 2011; Jazayeri & Shadlen, 2010; Acerbi et al., 2012; Chang & Jazayeri, 2018). This is reflected for example in the temporal and spatial tuning properties of neurons in the caudate, a part of the striatum also implicated in associative learning, action coordination and feedback processing (Grahn et al., 2008; Foerde & Shohamy, 2011). It is the union of these functions that make the caudate a prime candidate to support specificity and flexibility in the context of timing behavior.
Crucially, timing estimates are not always accurate. Instead, they reflect a trade-off between specificity and generalization, which is expressed in systematic behavioral biases. Estimated intervals regress towards the mean of the distribution of tested intervals (Jazayeri & Shadlen, 2010), a wellknown effect that we will refer to as the regression effect (Petzschner et al., 2015). It suggests that the brain encodes a probability distribution of possible intervals rather than the exact information obtained in each trial (Wolpert et al., 2011). Timing estimates thus depend not only on the interval tested in a given trial, but also on the temporal context (i.e., the intervals tested in all other trials). This likely helps to generalize from our current experience to possible future scenarios (Jazayeri & Shadlen, 2010; Acerbi et al., 2012).
Important evidence for the learning of task regularities for generalization comes from a parallel line of research centered on relational memory and cognitive mapping (Behrens et al., 2018; Momennejad, 2020). In particular, the hippocampus has been implicated in generalizing the structure of a task away from the individual features that were tested (Kumaran, 2012; Schlichting & Preston, 2015; Schapiro et al., 2017; Wikenheiser et al., 2017; Behrens et al., 2018; Schuck & Niv, 2019; Whittington et al., 2020; Peer et al., 2021), providing a unified account for its many proposed roles in domains such as navigation (Burgess et al., 2002), memory (Schiller et al., 2015) and decision making (Kaplan et al., 2017; Vikbladh et al., 2019). Moreover, the hippocampus has been shown to process behavioral feedback in decision-making tasks (Shohamy & Wagner, 2008), pointing to a potential role in feedback learning akin to the striatum. Intriguingly, this dovetails with computational theories (Chersi & Burgess, 2015; Geerts et al., 2020) and empirical work on memory-guided navigation (Doeller et al., 2008; Hartley et al., 2003), which suggested a division of labor between the striatum and the hippocampus. While the former may encode specific environmental details such as landmarks, the latter may support the encoding of the general layout and geometry of the environment (Hartley et al., 2003; Doeller et al., 2008; Goodroe et al., 2018; Gahnstrom & Spiers, 2020).
Here, we seek convergence of these ideas and research fields by testing if the same cognitive-mapping principles that govern learning in the spatial domain also apply to the time domain. Specifically, we investigate the relationship between hippocampal and striatal learning signals with behavioral performance in a fast-paced timing task, revealing how the brain flexibly updates task-relevant sensorimotor representations in real time. We explicitly focus on how the hippocampus and the striatum encode the details and the structure of a task in parallel, thus serving specificity and generalization. To do so, we used functional magnetic resonance imaging (fMRI) to monitor brain activity in participants estimating the time-to-contact (TTC) between a moving fixation target and a visual boundary. We tested how brain activity reflected the ongoing task performance and the behavioral feedback received. Moreover, we characterized in detail the relationship between brain activity and the improvements in task performance over time, indeed revealing distinct roles of the hippocampus and striatum in encoding different task regularities in parallel. While the caudate encoded task-relevant information that were specific to each TTC interval tested, the hippocampus generalized across intervals, signaling learning independent of TTC. Intriguingly, because learning occurred in real time when behavioral feedback is received, the corresponding activity we observed in the hippocampus goes beyond its well-known role in (long-term) episodic memory (Schiller et al., 2015). We conclude by proposing that the fundamental and domain-general function of striatal-hippocampal interactions may be finding the trade-off between specificity and generalization to guide human behavior broadly.
Results
In the following, we will present our experiment and results in four steps. First, we introduce the TTC-task as well as the behavioral and fMRI measurements we acquired. Second, we show that the activity in a large network of regions reflects the behavioral feedback participants received in the current and in the previous trial, and we demonstrate that this network centers on the hippocampus and the caudate as predicted. These results provide evidence for the proposed role of these structures in rapid feedback learning. Third, we show that the feedback modulation in both structures reflects improvements in behavioral performance over trials and thus that the learning was effective and rapid. Critically, while the caudate signaled behavioral improvements specific to the TTC interval tested in a given trial, the hippocampus generalized across TTC’s, signaling behavioral improvements independent of which TTC interval was tested. Voxel-wise analyses further revealed a striking distinction between striatal sub-regions and across a larger brain network. Finally, we show that caudate activity scaled with trial-wise behavioral performance, whereas hippocampal activity scaled with both the performance and the regression effect observed in behavior. Together, these results suggest that the hippocampus and the striatum update distinct information about the task in parallel, supporting the flexible, specific and generalizable expression of sensorimotor timing behavior in humans.
Time-to-contact (TTC) estimation task
We monitored whole-brain activity using fMRI with concurrent eye tracking in 34 participants performing a TTC task. This task offered a rich behavioral read-out and required sustained attention in every single trial. During scanning, participants visually tracked a fixation target, which moved on linear trajectories within a circular boundary. The target moved at one of four possible speed levels and in one of 24 possible directions (Fig. 1A, similar to Nau et al. (2018a)). The sequence of tested speeds was counterbalanced across trials. Whenever the target stopped moving, participants estimated when the target would have hit the boundary if it had continued moving. They did so while maintaining fixation, and they indicated the estimated TTC by pressing a button. Feedback about their performance was provided foveally and instantly with a colored cue. The received feedback depended on the timing error, i.e. the difference between objectively true and estimated TTC (Fig. 1B, Fig. S1), and it comprised 3 levels reflecting high, middle and low accuracy (Fig. 1C). Because timing judgements typically follow the Weber-Fechner law (Rakitin et al., 1998), the feedback levels were scaled relative to the ground-truth TTC of each trial. This ensured that participants were exposed to approximately the same distribution of feedback at all intervals tested (Fig. 1C, Fig. S1B). After a jittered inter-trial interval (ITI), the next trial began and the target moved into another direction at a given speed. The tested speeds of the fixation target were counterbalanced across trials to ensure a balanced sampling within each scanning run. Because the target always stopped moving at the same distance to the boundary, matching the boundary’s retinal eccentricity across trials, the different speeds led to four different TTCs: 0.55, 0.65, 0.86 and 1.2 seconds. Each participant performed a total of 768 trials. Please see Methods for more details.
Visual tracking and Time-To-Contact (TTC) estimation task. A) Task design. In each trial during fMRI scanning, participants fixated a target (phase 1), which started moving at one of 4 possible speeds and in one of 24 possible directions for 10° visual angle (phase 2). After the target stopped moving, participants kept fixating and estimated when the fixation target would have hit a boundary 5° visual angle apart (phase 3). After pressing a button at the estimated TTC, participants received feedback (phase 4) according to their performance. Feed-back was scaled relative to target TTC. B) Task performance. True and estimated TTC were correlated, showing that participants performed the task well. However, they overestimated short TTCs and underestimated long TTCs. Their estimates regressed towards the grand-mean of the TTC distribution (horizontal dashed line), away from the line of equality (diagonal dashed line). C) Feedback. On average, participants received high-accuracy feedback on half of the trials (also see Fig. S1B). BC) We plot the mean and SEM (black dots and lines) as well as single-participant data as dots. Feedback levels are color coded.
Analyzing the behavioral responses revealed that participants performed the task well and that the estimated and true TTCs were tightly correlated (Fig. 1B, Spearman’s rho = 0.91, p = 2.2x10− 16). However, participants’ responses were also systematically biased towards the grand mean of the TTC distribution (0.82 seconds), indicating that shorter durations tended to be overestimated and longer durations tended to be underestimated. This regression effect has been argued to show that timing estimates indeed rely on the statistical task regularities that our brain has encoded (e.g. Jazayeri & Shadlen (2010)). The regression effect may thus reflect a key behavioral adaptation which helps to generalize from current experiences to future scenarios. Visualizing the timing error over trials and scanning runs showed that participants’ task performance improved over time (Fig. S1C; linear mixed-effects model with run as fixed effect and participants as the error term, F(3) = 3.2944, p = 0.024, ϵ2 = 0.06, CI : [0.00, 0.13]), indicating learning over the course of the experiment.
Behavioral feedback predicts hippocampal and striatal activity in subsequent trial
Learning is expected to occur right after the value of the performed action became apparent, which is when participants received feedback. As a first proxy for learning, we thus analyzed how the activity in each voxel reflected the feedback participants received in each trial. Using a mass-univariate general linear model (GLM), we modeled the three feedback levels with one regressor each (high, medium, low), with additional nuisance regressors reflecting the 6 realignment parameters, the inter-trial-interval (ITI), button presses, and periods of rest in the middle as well as at the end of each run. We then contrasted the beta weights estimated for high-accuracy and low-accuracy feedback and examined the effects averaged across runs on the group-level using two-tailed one-sample t-tests.
Intriguingly, a voxel-wise analysis revealed that activity in the thalamus, striatum and the hippocampus could be predicted by the feedback participants received just before the trial had started (Fig. 2A). Higher-accuracy feedback led to overall stronger activity in these regions. Regions-of-interest analyses further localized this feedback-dependent activity to the caudate and the anterior section of the hippocampus (Fig. 2B, Fig. S2; two-tailed one-sample t tests: anterior HPC, t(33) = − 3.80, p = 5.9x10− 4, pf we = 0.002, d = − 0.65, CI : [− 1.03, − 0.28]; posterior HPC, t(33) = − 1.60, p = 0.119, pf we = 0.357, d = − 0.27, CI : [− 0.62, 0.07]; caudate, t(33) = − 5.85, p = 1.5x10− 6, pf we = 4.5x10− 6, d = − 1.00, CI : [− 1.43, − 0.59]). Note that there was no systematic and predictable relationship between subsequent trials on a behavioral level (Fig. S1A; t(33) = 1.03, p = 0.312, d = 0.18, CI : [− 0.17, 0.52]) and that the direction of the effects differed across regions (Fig 2A), speaking against a feedback-dependent bias in attention.
Feedback on the previous trial (n-1) modulates network-wide activity and hippocampal connectivity in subsequent trials (n). A) Voxel-wise analysis. Activity in each trial was modeled with a separate regressor as a function of feedback received in the previous trial. Insert zooming in on hippocampus added. B) Independent regions-of-interest analysis for the anterior (ant. HPC) and posterior (post. HPC) hippocampus as well as the caudate nucleus (CN). We plot the beta estimates obtained for the parametric modulator modeling trial-wise activity as a function of feedback in previous trial. Negative values indicate a negative relationship between feedback valence and brain activity. Depicted are the mean and SEM across participants (black dot and line) overlaid on single participant data (coloured dots). Activity in the anterior hippocampus and in the caudate is modulated by feedback received in previous trial. Statistics reflect p<0.05 at Bonferroni-corrected levels (*) obtained using a group-level two-tailed one-sample t-test against zero. C) Feedback-dependent hippocampal connectivity. We plot results of a psychophysiological interactions (PPI) analysis conducted using the hippocampal peak effects in (A) as a seed. AC) We plot thresholded t-test results at 1mm resolution overlaid on a structural template brain. MNI coordinates added. Hippocampal activity and connectivity is modulated by feedback received in the previous trial.
Feedback-dependent hippocampal functional connectivity
Having established that both the caudate and the hippocampus reflected feedback in the TTC task, we reasoned that the two structures may show systematic co-fluctuations in activity as well. To test this, we estimated the functional connectivity of a 4 mm radius sphere centered on the hippocampal peak main effect (x=-32, y=-14, z=-14) using a seed-based psychophysiological interaction (PPI) analysis. The PPI main regressor reflected the element-by-element product of the task time course and the hippocampal peak voxel time course used as a seed. Separate nuisance regressors modelled the main effect of previous feedback and the physiological signal correlations between the seed region and all other voxels.
We reasoned that larger timing errors and thus low-accuracy feedback would result in stronger learning compared to smaller timing errors and high-accuracy feedback, a relationship that should also be reflected in the functional connectivity between the hippocampus and other regions. We specifically tested this using the PPI analysis by contrasting trials in which participants performed poorly compared to those trials in which they performed well.
We found that hippocampal activity indeed co-fluctuated with activity in the caudate in a feedback-dependent manner (two-tailed one-sample t test: t(33) = − 5.85, p = 4.7x10− 4, d = 0.67, CI : [0.29, 1.05]). These co-fluctuations were stronger when participants received low-accuracy feedback compared to when they received high-accuracy feedback. Interestingly, however, we also observed such cofluctuations between the hippocampus and other regions that were likely task-relevant. These regions included the primary motor cortex, the parahippocampus and medial parietal lobe as well as the cerebellum (Fig. 2C).
Widespread brain activity reflects behavioral feedback in current trial
The results presented so far indicate that hippocampal functional connectivity, as well as the activity in the caudate and the hippocampus reflect feedback received in the previous trial. To test if the activity in these regions also predicted the performance in the current trial, we next conducted a GLM analysis in which we parametrically modeled the time course of each voxel and trial as a function of the feedback received at the end of the trial. Nuisance variance was accounted for using the same nuisance regressors as before.
A voxel-wise group-analysis for our regressors-of-interest showed that indeed a large network of regions signaled the performance in the current trial, which included the striatum, thalamus, cerebellum, lateral occipital cortex, motor cortex, insula, frontal eye fields as well as the hippocampus (Fig. 3A). We again confirmed that these effects were present in the hippocampus and the caudate using an independent regions-of-interest analysis (Fig. 3B, Fig. S2; two-tailed one-sample t tests: anterior HPC, t(33) = − 5.92, p = 1.2x10− 6, pf we = 3.7x10− 6, d = − 1.02, CI : [− 1.45, − 0.60]; posterior HPC, t(33) = − 4.07, p = 2.7x10− 4, pf we = 8.2x10− 4, d = − 0.70, CI : [− 1.09, − 0.32]; caudate, t(33) = − 7.56, p = 1.1x10− 8, pf we = 3.2x10− 8, d = − 1.30, CI : [− 1.78, − 0.85]).
Brain regions signalling TTC-task performance. Activity in each trial was modeled parametrically as a function of the feedback received at the end of the trial. A) Voxel-wise analysis. We plot thresholded t-test results at 1 mm resolution overlaid on a structural template brain. MNI coordinates and insert zooming in on the hippocampus added. A large network of regions signalling TTC performance included the hippocampus, striatum and cerebellum. B) Independent regions-of-interest analysis for the anterior (ant. HPC) and posterior (post. HPC) hippocampus as well as the caudate nucleus (CN). We plot the beta estimate obtained for the parametric modulator modeling trial-wise activity as a function of task performance. Negative values indicate a negative relationship between feedback valence and brain activity. Depicted are the means and SEM across participants (black dot and line) overlaid on single participant data (coloured dots). Statistics reflect p < 0.05 at Bonferroni-corrected levels (*) obtained using a group-level two-tailed one-sample t-test against zero.
Because each trial comprised multiple distinct phases, ranging from tracking the moving target over estimating the TTC to receiving feedback, the underlying processes might not only be distributed across the cortex, but also across time within the trial. To characterize the potentially dynamic relationship between activity and TTC-task performance in detail, we repeated the voxel-wise analysis for each trial phase separately (Fig. S3, similar to prior work (Wimmer et al., 2012)). We modelled each phase with a distinct regressor in a new GLM, finding strong differences between the trial phases in most of the observed areas. The hippocampus and caudate were again most strongly modulated when participants received feedback (Fig. S3). While the results obtained for the three phases are not independent due to the inherent temporal-order effects within each trial (Fig. 1A), they nevertheless suggest that the relationship between activity in each area and the behavioral outcome in the TTC-task is dynamic, and that the BOLD signal in different regions peaks at different times. Moreover, the fact that the hippocampus and the caudate were most strongly modulated in the feedback phase is again consistent with a role in rapid sensorimotor learning.
In sum, these results show that activity in various regions including the hippocampus and the caudate reflects the feedback received in the previous trial, but also the one received in the current trial. Critically, this is the case even though the actual feedback that was received was independent across trials (Fig. S1A; t(33) = 1.03, p = 0.312, d = 0.18, CI : [− 0.17, 0.52]), suggesting that these effects rest at least partially on independent variance in the fMRI signal.
Timing specificity and generalization in the hippocampus and in the striatum
Two critical open questions remained. First, did the observed feedback modulation actually reflect effective learning and thus behavioral improvements over trials? Second, was the information that was learned specific to the interval that was tested in a given trial, thus likely serving TTC specificity, or was independent of the tested interval, potentially serving TTC generalization? To answer these questions in one analysis, we conducted a GLM analysis in which we modeled activity not as a function of feedback received in the previous (Fig. 2) or current trial (Fig. 3), but as a function of the difference in feedback between trials (Fig. 4). Specifically, we modeled with two separate parametric regressors the improvements in TTC task performance across subsequent trials (regressor 1: TTC-generalized learning) as well as the improvements over subsequent trials in which the same TTC interval was tested (regressor 2: TTC-specific learning). We again accounted for nuisance variance as before, and contrasted trials in which participants had improved versus the ones in which they had not improved or got worse.
Distinct cortical and subcortical networks signal learning of TTC-specific and TTC-generalized task information. A) Left panel: Visual depiction of parametric modulator design. Two regressors per run modeled the improvement in behavioral performance since the last trial independent of the tested TTC (Regressor 1: TTC-generalized) or the improvement since the last trial when the same target TTC was tested (Regressor 2: TTC-specific). Right panel: Voxel-wise analysis results for TTC-specific and TTC-generalized regressors. We plot thresholded t-test results at 1mm resolution overlaid on a structural template brain. MNI coordinates added. Note that we depict the results at uncorrected levels for visualization, but the striatal and hippocampal effects survive p<0.05 whole-brain FWE-correction (Fig. S4A). B) Independent regions-of-interest analysis for the anterior (ant. HPC) and posterior (post. HPC) hippocampus as well as the caudate nucleus (CN). We plot the beta estimates obtained for TTC-generalized (orange dots) and TTC-specific (blue dots) regressors. Depicted are the mean and SEM across participants (black dot and line) overlaid on single participant data. Statistics reflect p<0.05 at Bonferroni-corrected levels (*) obtained using a group-level one-tailed one-sample t-test against zero.
Strikingly, our voxel-wise analysis revealed both TTC-specific and TTC-generalized learning activity throughout cortical and subcortical regions, with distinct areas engaging in either one or in both of these processes (Fig. 4A). Most prominently, we observed a salient distinction in the striatum (Fig. 4A), and we found that hippocampal activity signaled behavioral improvements independent of the TTC intervals tested. An independent ROI analysis confirmed that this TTC-generalized main effect was localized to the posterior section of the hippocampus (Fig. 4B; one-tailed one-sample t tests; TTC-specific: anterior HPC, t(33) = 0.57, p = 0.285, pf we = 1, d = 0.10, CI : [− 0.24, 0.44], posterior HPC, t(33) = 1.29, p = 0.103, pf we = 0.619, d = 0.22, CI : [− 0.12, 0.57]; TTC-generalized: anterior HPC, t(33) = 0.36, p = 0.360, pf we = 1, d = 0.06, CI : [− 0.28, 0.40], posterior HPC, t(33) = 2.81, p = 0.004, pf we = 0.025, d = 0.48, CI : [0.12, 0.85]). In stark contrast, the caudate signaled improvements in behavioral performance only relative to previous trials in which the same TTC interval was tested as in the current trial (Fig. 4B; one-tailed one-sample t tests; TTC-specific: t(33) = 5.95, p = 5.6x10− 7, pf we = 3.4x10− 6, d = 1.02, CI : [0.61, 1.45]; TTC-generalized: t(33) = − 0.67, p = 0.746, pf we = 1, d = − 0.11, CI : [− 0.46, 0.23]). It thus likely engaged in the updating of TTC-specific information, unlike the hippocampus, with both regions nevertheless reflecting the behavioral improvements over time. Finally, we again estimated the functional connectivity profile of the hippocampal main effect as before (sphere with 4mm radius centered on the peak voxel at x=-30, y=-24, z=-18), revealing behavioral-improvement-dependent co-fluctuations in multiple regions including the putamen and the thalamus (Fig. S5).
These results suggest that the caudate may serve specificity by updating information specific to the target TTC, whereas the hippocampus may serve generalization by updating information that is independent of the target TTC. In this task, an efficient way of generalizing across TTCs is to bias one’s responses towards the mean of the TTC distribution, which effectively corresponds to the regression effect that we observed on a behavioral level (Fig. 1B). Given the feedback modulation and learning effects we reported above, we thus hypothesized that hippocampal activity should also reflect the magnitude of the regression effect in behavior. Likewise, caudate activity should reflect how accurate participants were independent of the regression effect. To test this in a final analysis, we modeled the activity in each trial parametrically either as a function of performance (i.e. the difference between estimated and true TTC) or as a function of the strength of the regression effect in each trial (i.e. the difference between the estimated TTC and the mean of the tested intervals). Voxel-wise weights for these two regressors were estimated in two independent GLMs in which nuisance variance was again accounted for as before. Both voxel-wise and ROI-based analyses showed that that caudate activity indeed reflected how accurate participants were in each trial (Fig. 5A, B; two-tailed one-sample t test; t(33) = − 5.62, p = 2.9x10− 6, pf we = 8.7x10− 6, d = − 0.96, CI : [− 1.39, − 0.56]), but not the regression effect (two-tailed one-sample t test; t(33) = 1.08, p = 0.287, pf we = 0.859, d = 0.19, CI : [− 0.16, 0.53]), along with other regions. Hippocampal activity reflected the accuracy in each trial (two-tailed one-sample t tests; anterior HPC, t(33) = − 4.85, p = 2.9x10− 5, pf we = 8.7x10− 5, d = − 0.83, CI : [− 1.24, − 0.44]; posterior HPC, t(33) = − 2.88, p = 0.007, pf we = 0.021, d = − 0.49, CI : [− 0.86, − 0.14]), consistent with the previously reported feedback modulation (Fig. 3), but in addition it indeed reflected how strongly participants’ TTC estimates regressed towards their mean (Fig. 5A, B; two-tailed one-sample t tests; anterior HPC, t(33) = − 5.55, p = 3.6x10− 6, pf we = 1.1x10− 5, d = − 0.95, CI : [− 1.37, − 0.55]; posterior HPC, t(33) = − 1.06, p = 0.295, pf we = 0.886, d = − 0.18, CI : [− 0.53, 0.16]). Notably, similar effects were observed in prefrontal and posterior cingulate areas (Fig. 5A).
TTC-task performance vs. behavioral regression effect. A) Voxel-wise analysis. We plot thresholded F-test results for the task-performance regressor and the regression-to-the-mean regressor at 1 mm resolution overlaid on a structural template brain. MNI coordinates added. Distinct networks reflect task performance and the regression effect. B) Independent regions-of-interest analysis for the anterior (ant. HPC) and posterior (post. HPC) hippocampus as well as the caudate nucleus (CN). We plot the beta estimates obtained for each participant for each of the two regressors. Depicted are the mean and SEM across participants (black dot and line) overlaid on single participant data (blue and orange dots). Statistics reflect p<0.05 at Bonferroni-corrected levels (*) obtained using a group-level two-tailed one-sample t-test against zero.
Eye tracking: no biases in viewing behavior across comparisons
To ensure that our results could not be attributed to systematic error patterns in viewing behavior, we analyzed the co-recorded eye tracking data of our participants in detail. After preprocessing (see methods), we used Kruskal-Wallis tests to test for differences in fixation accuracy across speed levels (Fig. S6A; χ(2) = 0.61, p = 0.895, ϵ2 = 0.005, CI : [0.00, 0.06]) and received-feedback levels (Fig. S6B; χ(2) = 0.190, p = 0.909, ϵ2 = 0.002, CI : [0.00, 0.10]). Moreover, we examined the relationship of the fixation error with TTC-task performance (Fig. S6C; Spearman’s rho = 0.17, p = 0.344) as well as with the behavioral regression effect (Fig. 1B, Fig. S6C; Spearman’s rho = 0.26, p = 0.131). None of these control analyses suggested that biased patterns in viewing behavior could hinder the interpretation of our results.
Discussion
This study investigated how the brain extracts the statistical regularities of a sensorimotor timing task in the service of immediate actions and how the formed sensorimotor representations are continuously updated in a feedback-dependent manner. We focused on how the hippocampus and the caudate support behavioral flexibility, specificity and generalization due to their known coding schemes for space, time and reward. We monitored human brain activity with fMRI while participants performed a time-to-contact (TTC) estimation task, allowing us to analyze brain activity as a function of behavioral feedback and task performance as well as of the improvements in task performance over time using a series of general linear models. We found that the feedback participants received was reflected in the activity of a large network of brain regions, which prominently included the hippocampus and the striatum. Moreover, feedback also modulated hippocampal functional connectivity with other task-relevant regions in the subsequent trial. By showing that striatal and hippocampal activity directly followed the improvements in behavioral performance over time, we demonstrated a link between brain activity and effective learning. Strikingly, the two structures engaged in the learning of different types of information in parallel: caudate activity reflected behavioral improvements specific to the tested time intervals, whereas hippocampus generalized across time intervals. These results provide empirical evidence for distinct but complementary roles of the hippocampus and the striatum in sensorimotor learning, supporting specificity and generalization, and they show that the underlying processes directly translate into the immediate behavior participants express. In what follows, we discuss our results in the context of prior work on timing behavior and on spatiotemporal coding in the hippocampus and striatum. Moreover, we elaborate on the domain-general nature of hippocampal-striatal interactions and of the flexible learning mechanisms that potentially underlie the effects observed in this study.
Spatiotemporal coding in the hippocampus and striatum
Both the striatum and the hippocampus encompass neurons sensitive to the time that has passed since a certain task has begun (Paton & Buonomano, 2018; Eichenbaum, 2014; Umbach et al., 2020). These cells might play an important role in guiding timing behavior (Nobre & van Ede, 2018), which potentially explains why damage or inactivation of either structure in rodents (Meck et al., 1984; Gouvêa et al., 2015), non-human primates (Wang et al., 2018) or humans (Richards, 1973) impairs the ability to estimate durations. Our results are in line with these reports, showing that fMRI activity in the hippocampus and striatum also reflects participants’ TTC estimation ability (Fig. 3). They are also in line with other human neuroimaging studies suggesting that the hippocampus bridges temporal gaps between two stimuli during trace eyeblink conditioning (Cheng et al., 2008), and that it represents duration within event sequences (Barnett et al., 2014; Thavabalasingam et al., 2018, 2019). Our results support prior work showing that striatal activity prominently encodes temporal information (Bakhurin et al., 2017; Mello et al., 2015), reflects interval learning (Dallérac et al., 2017) and predicts duration judgements in rats (Mello et al., 2015; Gouvêa et al., 2015), non-human primates (Wang et al., 2018) and humans (Hinton & Meck, 2004). Such striatal representations of time were shown to control the timing of actions in concert with neurons in the medial frontal cortex in monkeys (Wang et al., 2018).
Our results speak to the above-mentioned reports by revealing the widespread brain activity contributing to effective sensorimotor learning of intervals in humans (Fig. 2,3,4,S3,S4,S5). Moreover, they demonstrate a direct link between hippocampal and striatal activity, the feedback participants received and the behavioral improvements expressed over time (Fig. 4). Critically, this underlying learning process must occur in real-time when feedback is presented, suggesting that it plays out on short-term time scales. Notably, the hippocampus is neither typically linked to sensorimotor timing tasks such as ours, nor is it considered to reflect temporal relationships on such short time scales in humans. Instead, human hippocampal activity is often studied in the context of much longer time scales, which showed that it may encode the succession of experiences and events into long-term episodic memories (Deuker et al., 2016; Montchal et al., 2019) or contribute to the establishment of chronological relations between events in memory (Gauthier et al., 2019, 2020). Intriguingly, however, the mechanisms at play may build on similar temporal coding principles as those discussed for motor timing (Yin & Troger, 2011; Eichenbaum, 2014; Howard, 2017; Palombo & Verfaellie, 2017; Nobre & van Ede, 2018; Paton & Buonomano, 2018; J. L. Bellmund et al., 2020; J. L. S. Bellmund et al., 2021; Shikano et al., 2021; Shimbo et al., 2021).
Importantly, our task can be solved by estimating temporal intervals directly, but also by extrapolating the movement of the fixation target over time, shifting the locus of attention towards the target boundary (Fig. 1). The brain may thus likely monitor and learn about the temporal and spatial task regularities in parallel. Participants’ TTC estimates were further informed exclusively by the speed of the target, which inherently builds on tracking kinematic information over time, which may explain why TTC tasks also engage visual motion regions in humans (de Azevedo Neto & Amaro Júnior, 2018). While future studies could tease apart spatial and temporal factors explicitly, our results are in line with both accounts. The hippocampus and surrounding structures for example represent maps of visual space in primates (Nau et al., 2018), which potentially mediate a coordinate system for behavioral planning, for integrating visual information with existing knowledge and to compute vectors in space. These visuospatial representations are perfectly suited to guide attention and thus also the relevant behaviors in our task (Aly & Turk-Browne, 2017), which could be tested in the future akin to prior work using a similar paradigm (Nau et al., 2018a).
The role of feedback in timed motor action
Critically, our results do not imply that the hippocampus acts as an “internal clock” in our task, nor do we think of it as representing action sequences or coordinating motor commands directly. Rather, its activity may indicate the feedback-dependent updating of encoded information generally independent of the specific intervals or even the task that were used. The hippocampus has been proposed as a domain-general model-based learning system, which encodes the task structure into an (allocentric) cognitive-map-like format (Kumaran, 2012; Schlichting & Preston, 2015; Chersi & Burgess, 2015; Schapiro et al., 2017; Wikenheiser et al., 2017; Behrens et al., 2018; Vikbladh et al., 2019; Geerts et al., 2020; Momennejad, 2020). Consequently, it may help to encode the structure of a task abstracted away from our immediate experience. In contrast, the striatum was proposed to encode sensory states or actions, thus supporting the encoding of task-specific (ego-centric) information (Chersi & Burgess, 2015; Geerts et al., 2020). Together, the two regions may thus together play an important role in decision making beyond timing.
Consistent with these ideas, we observed that striatal and hippocampal activity was modulated by feedback in a behavior-dependent manner (Fig. 2,3). Similar feedback signals have previously been linked to learning (Schönberg et al., 2007; Cohen & Ranganath, 2007; Shohamy & Wagner, 2008; Foerde & Shohamy, 2011; Wimmer et al., 2012) and to the successful formation of hippocampus-dependent long-term memories in humans (Wittmann et al., 2005). Moreover, hippocampal activity is known to signal learning in other tasks (Foerde & Shohamy, 2011; Dickerson & Delgado, 2015; Wirth et al., 2009; Schapiro et al., 2017; Kragel et al., 2021). Our results are also in line with prior work on the putative role of the caudate in updating temporal priors in non-human primates (Wang et al., 2018; Suzuki & Tanaka, 2019) and with reports showing that disrupting striatal activity leads to decreases in timing-task performance specifically when the tested time intervals change (Mello et al., 2015; Wang et al., 2018). Here, we show a direct relationship between such effective-learning signals and timing behavior, and we show that feedback modulates widespread brain activity (Fig. 2, 3, potentially reflecting the involvement of these areas in the coordination of reward behavior observed earlier (LeGates et al., 2018). These regions includes those serving sensory and motor functions, but also those encoding the structure of a task or the necessary value functions associated with specific actions (Lee et al., 2012).
The present study further demonstrates that activity in the hippocampus co-fluctuates with activity in the striatum in a task-dependent manner. Similar co-fluctuations with hippocampal activity were observed in the motor cortex, typically involved in action planning and execution, the parahip-pocampus and medial parietal lobe, often associated with visual-scene analysis (Epstein & Baker, 2019), as well as the cerebellum (Fig. 2C), which is tightly coupled with the basal ganglia for coordinating actions (Bostan & Strick, 2018). This may indicate that behavioral feedback also affects the functional connectivity profile of the hippocampus with those domain-selective regions that are currently engaged in the ongoing task.
What might be the neural mechanism underlying sensorimotor learning in our study? Prior work has shown that frontal, striatal and hippocampal temporal receptive fields scale relative to the tested intervals, and that they re-scale dynamically when those tested intervals change (MacDon-ald et al., 2011; Gouvêa et al., 2015; Mello et al., 2015; Wang et al., 2018). This may enable the encoding and continuous maintenance of optimal task priors, which keep our actions well-adjusted to our current needs. We speculate that such receptive-field re-scaling also underlies the learning discussed here, which likely builds both on local and network-wide re-weighting of functional connections between neurons and entire regions. Consistent with this idea and the present results, receptive-field re-scaling can occur on a trial-by-trial basis in the striatum (Mello et al., 2015; Gouvêa et al., 2015; Wang et al., 2018) and in the hippocampus (Shikano et al., 2021; Shimbo et al., 2021).
Striatal and hippocampal interactions: A trade-off between specificity and generalization?
So far, we discussed how hippocampal and striatal processes may support the flexible expression of timing behavior, but how does this process strike the balance between specificity and generalization (i.e. how does the learned probability distribution capture the tested intervals optimally without overfitting)? Our results suggest that the trade-off between these two objectives is governed by activity in many regions, which update different types of task information in parallel (Fig. 4A). Hippocampal activity reflected improvements in behavioral performance over trials independent of the tested TTC level, whereas the caudate signaled behavioral improvements specifically over those trials in which the same TTC was tested. This suggests a striking functional distinction between these areas, serving either specificity or generalization, which is complemented by other areas engaging in both processes simultaneously such as the putamen, the pallidum and the thalamus (Fig.S4C).
Interestingly, this putative division of labor between the hippocampus and striatal areas dovetails with a large body of literature on spatial navigation, reporting many similarities but also clear differences in function between these structures (Doeller et al., 2008; Chersi & Burgess, 2015; Goodroe et al., 2018; Gahnstrom & Spiers, 2020; Geerts et al., 2020). Prior work has shown that the striatum supports the reinforcement-dependent encoding of locations relative to landmarks, whereas the hippocampus may help to encode the structure of the environment in a generalizable and map-like format. We propose that this encoding of specific and generalizable information is the fundamental and domain-general function of hippocampal-striatal interactions: they find an optimal trade-off between specificity and generalization. This agrees well with the functional differences observed in the present study, with caudate activity potentially reflecting the encoding of individual details of our task such as the TTC intervals, and the hippocampus generalizing across TTCs to encode the overall task structure. Notably, meeting the two objectives can have antagonistic effects on behavior, and the consequences of the updating sensorimotor representations must likely be communicated to a wider network of task relevant regions. This could explain the activity co-fluctuations we observed in our data (Fig. 2C), which fits to prior observations of the two areas interacting in many tasks including associative learning (Mattfeld & Stark, 2015) and navigation (Brown & Stern, 2014), even showing synchronized neural activity in rats (Berke et al., 2004), and when temporal expectations are violated (van de Ven et al., 2020).
Importantly, we observed that TTC estimates regress towards the mean of the sampled intervals, an effect that is well known in the timing literature (Fig. 1B), Jazayeri & Shadlen (2010)) and in other domains (Petzschner et al., 2015). We hypothesized that this effect is grounded in the activity of the hippocampus for the following reasons. First, biasing time estimates towards their mean may naturally serve generalization, because the mean of the tested intervals will likely also be the mean of possible future intervals. Second, the hippocampus has been suggested to play a central role in generalization in other non-temporal domains (Kumaran, 2012; Schlichting & Preston, 2015; Schapiro et al., 2017; Momennejad, 2020). The picture that emerges from our results is that this function of the hippocampus may also support generalization in the temporal domain, potentially reflecting the temporal-context-dependent learning of the grand mean of the tested intervals (Jazayeri & Shadlen, 2010). This is because the hippocampus signalled TTC-generalized behavioral improvements over trials (Fig. 4) as well as the strength of the TTC regression effect in behavior (Fig. 5). We thus provide evidence for the poorly understood neural underpinnings of the well-known regression effect in behavior (Petzschner et al., 2015), which likely reflects an important behavioral adaptation central to our ability to generalize. Our findings directly predict that participants with stronger hippocampal involvement during encoding will perform better when new intervals are tested compared to controls, a prediction that could be tested in future work.
Conversely, we found that the caudate signalled behavioral improvements that were specific to the TTC that was tested (Fig. 4), and it reflected the accuracy of the TTC estimates independent of the regression effect (Fig. 5). Moreover, the putamen was strongly modulated by both accuracy and the regression effect unlike the caudate (Fig. 4), revealing a striking functional distinction across striatal subregions. These findings are in line with a large body of literature implicating the striatum in coding time and reward (Cox & Witten, 2019; Petter et al., 2018; Paton & Buonomano, 2018) as well as the value of specific actions (Samejima et al., 2005) in a distributed manner. Prior studies have for example shown a functional distinction between sub-units of the striatum (e.g. Hunnicutt et al. (2016); Grahn et al. (2008)), which also agrees with the fact that stable task-structure and flexible events are coded in parallel in the striatum (Kubota et al., 2009). They also agree with reports showing that the spatiotemporal integration principles governing memory formation in the striatum and the hippocampus differ (Ferbinteanu, 2020).
Finally, feedback-dependent hippocampal-striatal interactions may also speak to the behavioral impairments observed in patients with Parkinson’s disease (PD), who often misestimate durations when neurons in the striatum lack dopaminergic inputs. The timing judgements of such patients tend to regress toward the mean more quickly without dopaminergic medication than when these patients are medicated (Malapani et al., 1998, 2002). We propose that this may reflect an impairment in the interval-specific striatal learning mechanism we observed here (Shi et al., 2013). This is also in agreement with findings of impaired learning in PD patients, which has previously been linked to hippocampal and striatal learning signals (Foerde & Shohamy, 2011; Wimmer et al., 2012). Fittingly, such timing impairments are most pronounced specifically when the tested intervals change, and they can be alleviated using dopamine medication, again pointing to a tight link between the effects observed here and reward processing. Notably, a similar dopaminergic control has been shown for hippocampal-dependent stimulus generalization (Kahnt & Tobler, 2016).
Conclusion
In sum, we combined fMRI and a time-to-contact estimation task to show that the human hippocampus and striatum support the effective learning from feedback, likely serving behavioral flexibility, specificity and generalization broadly. Hippocampal and striatal activity directly reflected the behavioral improvements over trials, signalling the encoding of TTC-generalized and TTC-specific information respectively. This strikingly resembled previous observations made in the spatial domain, suggesting that the proposed functions of hippocampal-striatal interactions are domaingeneral. Inspired by two parallel lines of research, one centered on sensorimotor timing and the other one on cognitive mapping and structure learning, we suggest that the fundamental function of hippocampal-striatal interactions may be finding the trade-off between specificity and generalization. Our results demonstrate that the hippocampus and striatum play an important role in feedback learning, and they implicate both structures in guiding flexible, specific and generalizable timing behavior in humans.
Author Contributions
MN, IP and CFD developed the research questions. MN conceived the experimental idea. IP and MN designed the experimental paradigm, visualized the results and embedded them in the literature with help from RK, VW and CFD. IP implemented the experimental code and acquired and analyzed the data with close supervision and help from MN. MN wrote the manuscript with help from IP. CFD secured funding. RK, VW and CFD provided critical feedback and all authors discussed the results and edited the final manuscript.
Declaration of interest
The authors declare no conflicts of interest.
Data and code availability
Source data and analysis scripts will be shared upon publication. Raw data are available from the authors upon request.
Methods
Participants
We recruited 39 participants for this study (16 females, 19-35 years old). Five participants were excluded: one participant did not comply with the task instructions; one was excluded due to a failure of the eye-tracker calibration; three participants were excluded due to technical issues during scanning. A total of 34 participants entered the analysis. The study was approved by the regional committee for medical and health research ethics (project number 2017/969) in Norway and participants gave written consent prior to scanning in accordance with the declaration of Helsinki (2008).
Task
Participants performed two tasks simultaneously: a smooth pursuit visual-tracking task and a time-to-contact estimation task. The visual tracking task entailed fixation at a fixation disc that moved on predefined linear trajectories with one of four speeds: 4.2°/s, 5.8°/s, 7.5°/s and 9.1°/s. Upon reaching the end of such a linear trajectory, the dot stopped moving until the second task was completed. This second task was a time-to-collision (TTC) estimation task in which participants indicated when the fixation target would have hit a circular boundary if it had continued moving. This boundary was a yellow circular line surrounding the target trajectory with 10°radius. Participants gave their response by pressing a button at the anticipated moment of collision. They performed this task while still keeping fixation, and the individual linear trajectories were all of the same length (10°visual angle), leading to four target TTC durations of 1.2s, 0.88s, 0.67s and 0.55s tested in a counter-balanced fashion across trials. After the button press, participants received feedback for 1 second informing them about the accuracy of their response. When participants overestimated the TTC, half of the fixation disc closest to the boundary changed color (orange or red) as a function of response accuracy (medium or low, respectively). When participants underestimated the TTC, half of the fixation disc further away from the boundary changed color. When participants were accurate, two opposing quadrants of the fixation disc would turn green. This allowed us to present feedback at fixation while keeping the number of informative pixels matched across feedback levels. To calibrate performance feedback across different TTC durations, the precise response window widths of each feedback level scaled with the speed of the fixation target. The following formula was used to scale the response window width: d ± ((k * d)/2) where d is the target TTC and k is a constant proportional to 0.3 and 0.15 for high and medium accuracy, respectively. This ensured that participants received approximately the same feedback for tested TTCs despite the known differences in absolute performance between target TTCs due to inherent scalar variability (Gibbon, 1977). When no response was given, participants received low-accuracy feedback (two opposing quadrants of the fixation dot turned red) after a 4 seconds timeout. After the feedback, the disc remained in its last position for a variable inter-trial interval (ITI) sampled randomly from a uniform distribution between 0.5 seconds and 1.5 seconds. Following the end of the ITI, the dot continued moving in a different direction. In the course of 768 trials, each target TTC was sampled 192 times. We sampled eye-movement directions with 15°resolution, leading to an overall trajectory that was star-shaped, similar to earlier reports (Nau et al., 2018a). The full trajectory was never explicitly shown to the participants.
Behavioral analysis
Participants indicated the estimated TTC in each trial via button press. Estimated and true TTC were strongly correlated (Spearman’s rho = 0.91, p = 2.2x10− 16) showing that participants performed the task well. However, in line with previous work (Jazayeri & Shadlen, 2010), participants tended to overestimate shorter durations and underestimate longer durations (Fig. 1B). As a measure of behavioral performance, we computed the absolute TTC-error defined as the absolute difference in estimated and true TTC for each target-TTC level. Participants received feedback after each trial corresponding to the absolute TTC error of that trial. On average, 46.9% (σ = 9.1) of trials were of high accuracy, 31.2% (σ = 3.9) were of medium accuracy and 21.1% (σ = 9.8) were of low accuracy (Fig. 1C). Moreover, we found that this feedback distribution was indeed similar across target-TTC levels as planned (Fig. S1B). To test participants’ performance improvements over time, we used a linear mixed-effects model with run as predictor, absolute TTC-error as the dependent variable and participants as the error term. The results showed a main effect of run (F(3) = 3.2944, p = 0.024, ϵ2 = 0.06, CI : [0.00, 0.13]. Post-hoc tests using Bonferroni correction confirmed a significant decrease in absolute TTC-error between run 1 and 4 (t(104) = 2.86, pf we = 0.031, d = 0.56, CI : [0.17, 0.95]).
Imaging data acquisition & preprocessing
Imaging data were acquired on a Siemens 3T MAGNETOM Skyra located at the St. Olavs Hospital in Trondheim, Norway. A T1-weighted structural scan was acquired with 1mm isotropic voxel size. Following EPI-parameters were used: voxel size=2mm isotropic, TR=1020ms, TE=34.6ms, flip angle=55°, multiband factor=6. Participants performed a total of four scanning runs of 16-18 minutes each including a short break in the middle of each run. Functional images were corrected for head motion and co-registered to each individual’s structural scan using SPM12 (www.fil.ion.ucl.ac.uk/spm/). We used the FSL topup function to correct field distortions based on one image acquired with inverted phase-encoding direction (https://fsl.fmrib.ox.ac.uk/fsl/fslwiki/topup). Functional images were then spatially normalized to the Montreal Neurological Institute (MNI) brain template and smoothed with a Gaussian kernel with full-width-at-half-maximum of 4 mm for regions-of-interest analysis or with 8 mm for whole-brain analysis. Time series were high-pass filtered with a 128 s cut-off period. The results of all voxel-wise analyses were overlaid on a structural T1-template (colin27) of SPM12 for visualization.
Regions of interest definition and analysis
Regions-of-interest masks for different brain areas were generated for each individual participant based on the automatic parcellation derived from FreeSurfer’s structural reconstruction (https://surfer.nmr.mgh.harvard.edu/). The ROIs used in the present study include the caudate nucleus (CN) and the hippocampus (HPC) as main areas of interest (Fig. S2A) as well as the Nucleus Accumbens, Thalamus, Putamen, Amygdala and Globus Pallidum in addition (Fig. S4). The hippocampal ROI was manually segmented following previous reports into its anterior and posterior sections based on the location of the uncal apex in the coronal plane as a bisection point (Poppenk et al., 2013). We did this because prior work suggested functional differences between anterior and posterior hippocampus with respect to their contributions to memory-guided behavior (Poppenk et al., 2013). All individual ROIs were then spatially normalized to the MNI brain template space and re-sliced to the functional imaging resolution using SPM12. All ROI analyses were conducted using 4mm spatial smoothing.
All ROI analyses described in the following were conducted using the following procedure. We extracted beta estimates estimated for the respective regressors of interest for all voxels within a region in both hemispheres, averaged them across voxels within that region and hemispheres and performed one-sample t-tests on group level against zero as implemented in the software R (https://www.R-project.org).
Brain activity as a function of current-trial performance
We used a mass-univariate general linear model to analyze the time courses of all voxels in the brain as a function of feedback received at the end of each trial. The model included one mean-centered parametric modulator per run with three levels reflecting the feedback received in each trial. The feedback itself was a function of TTC error in each trial (high accuracy = 0, medium accuracy = 0.5 and low accuracy = 1). In addition, we added three nuissance regressors per run modeling ITIs, button presses, and periods of rest. These regressors were convolved with the canonical hemodynamic response function of SPM12. Moreover, the model included the six realignment parameters obtained during pre-processing as well as a constant term modeling the mean of the time series. We estimated weights for all regressors and conducted a t-test against zero using SPM12 for our feedback regressors of interest on the group level. Importantly, positive t-scores indicate a positive relationship between fMRI activity and TTC error and hence with poor behavioral performance. Conversely, negative t-scores indicate a negative relation between the two variables and hence better behavioral performance.
In addition to the voxel-wise whole-brain analyses described above, we conducted independent ROI analyses for the anterior and posterior sections of the hippocampus (Fig. S2A, Fig. 3B; two-tailed one-sample t tests: anterior HPC, t(33) = − 5.92, p = 1.2.x10− 6, pf we = 3.7x10− 6, d = − 1.02, CI : [− 1.45, − 0.60]; posterior HPC, t(33) = − 4.07, p = 2.7x10− 4, pf we = 8.2x10− 4, d = − 0.70, CI : [− 1.09, − 0.32]) and for the caudate (Fig. S2B, Fig. 3B; two-tailed one-sample t test: t(33) = − 7.56, p = 1.1x10− 8, pf we = 3.2x10− 8, d = − 1.30, CI : [− 1.78, − 0.85]). Here, we tested the beta estimates obtained in our first-level analysis for the feedback regressor of interest. See section “Regions of interest definition and analysis” for more details.
Brain activity as a function of trial phase
To examine the relation between brain activity and behavioral performance in a trial in more detail, we repeated the univariate analysis explained above for each phase of the trial. Three regressors modelled the main effects of trial phase. Three additional parametric regressors modeled the feed-back effect on the activity during the tracking phase, the TTC estimation phase and the feedback phase in one GLM. In addition, we again added regressors modeling the ITI’s, button presses and periods of rest to the model as well as head-motion regressors and a constant term as before. Each run was modeled separately. On the group-level, we again used SPM12 to perform t-tests against zero using the weights estimated for the feedback regressors of interest for each trial phase.
Brain activity as a function of performance on the previous trial
To examine how feedback modulates activity in the subsequent trial, we used a GLM analysis to model the activity of each voxel and trial as a function of feedback received in the previous trial. The GLM included three regressors modeling the feedback levels, one for ITIs, one for button presses and one for periods of rest, which were all convolved with the canonical hemodynamic response function of SPM12. In addition, the realignment parameters and a constant term were again added. On the group level, we then contrasted the weights obtained for the low error vs. high error regressors and tested for differences using t-tests implemented in SPM12.
Additionally, we again conducted ROI analyses for the anterior and posterior sections of the hippocampus (Fig. S2A, Fig. 2B; Two-tailed t tests: anterior HPC, t(33) = − 3.80, p = 5.9x10− 4, pf we = 0.002, d = − 0.65, CI : [− 1.03, − 0.28]; posterior HPC, t(33) = − 1.60, p = 0.119, pf we = 0.357, d = − 0.27, CI : [− 0.62, 0.07]) as well as for the caudate (Fig. S2B, Fig. 2B; Two-tailed t test: t(33) = − 5.85, p = 1.5x10− 6, pf we = 4.5x10− 6, d = − 1.00, CI : [− 1.43, − 0.59]) following the same procedure as described earlier (section “Regions of interest definition and analysis”). Here, we tested beta estimates obtained in the first-level analysis for the feedback-in-previous-trial regressor of interest.
Hippocampal functional connectivity as a function of previous-trial performance
We conducted a psychophysiological interactions (PPI) analysis to examine whether hippocampal functional connectivity with the rest of the brain depended on the participant’s performance on the previous trial. To do so, we centered a sphere onto the group-level peak effects within the HPC using main-effect GLM described in the previous section. The sphere was 4mm in radius and was centered on following MNI coordinates: x=-32, y=-14, z=-14. The GLM included a PPI regressor, a nuisance regressor accounting for the main effect of past-trial performance, and a nuisance regressor explaining variance due to inherent physiological signal correlations between the HPC and the rest of the brain. The PPI regressor was an interaction term containing the element-by-element product of the task time course (effects due to past-trial performance) and the HPC spherical seed ROI time course. The estimated beta weight corresponding to the interaction term was then tested against zero on the group-level using a t-test implemented in SPM12. This revealed brain areas whose activity was co-varying with the hippocampus seed ROI as a function of past-trial performance (n-1).
Furthermore, we conducted an ROI analysis for the caudate (Fig. S2B; one-tailed t test: t(33) = − 5.85, p = 4.7x10− 4, d = 0.67, CI : [0.29, 1.05]) following the same procedure as described earlier (see section “Regions of interest definition and analysis”). Here, we tested the beta estimates obtained in the first-level analysis for the PPI regressor of interest against zero.
Brain activity as a function of improvements in behavioral performance across trials
We used a GLM to analyze activity changes associated with behavioral improvements across trials. One regressor modelled the main effect of the trial and two parametric regressors modeled the following contrasts: trials in which behavioral performance improved vs. trials in which behavioral performance did not improve or got worse relative to the previous trial. These regressors modeled the behavioral improvements either relative to the previous trial, and thus independently of TTC (likely serving generalization), or relative to the previous trial in which the same target TTC was presented (likely serving specificity). These two regressors reflect the tests for target-TTC-generalized and target-TTC-specific learning, respectively. Improvement in performance was defined as receiving feedback of higher valence than in the corresponding previous trial. The same nuisance regressors were added as in the other GLMs and all regressors except the realignment parameters and the constant term were convolved with the canonical hemodynamic response function of SPM12. On the group level, we tested the two parametric regressors of interest against zero using a t-test implemented in SPM12, effectively contrasting trials in which behavioral performance improved against trials in which behavioral performance did not improve or got worse relative to the respective previous trials. All runs were modeled separately.
Moreover, we again conducted ROI analyses for the anterior and posterior sections of the hippocampus (Fig. S2A) as well as for the caudate (Fig. S2B) following the same procedure as described earlier (see section “Regions of interest definition and analysis”). Here, we tested beta estimates obtained in the first-level analysis for the TTC-specific and TTC-generalized learning regressors using one-tailed one-sample t-tests (TTC-specific: anterior HPC, t(33) = 0.57, p = 0.285, pf we = 1, d = 0.10, CI : [− 0.24, 0.44], posterior HPC, t(33) = 1.29, p = 0.103, pf we = 0.619, d = 0.22, CI : [− 0.12, 0.57], caudate, t(33) = 5.95, p = 5.6x10− 7, pf we = 3.4x10− 6, d = 1.02, CI : [0.61, 1.45]; TTC generalized: anterior HPC, t(33) = 0.36, p = 0.360, pf we = 1, d = 0.06, CI : [− 0.28, 0.40], posterior HPC, t(33) = 2.81, p = 0.004, pf we = 0.025, d = 0.48, CI : [0.12, 0.85], caudate, t(33) = − 0.67, p = 0.746, pf we = 1, d = − 0.11, CI : [− 0.46, 0.23]). In addition, to test which specific subcortical regions were involved in these processes, we conducted post-hoc ROI analyses for subcortical regions after the whole-brain results were known (one-tailed one-sample t tests; TTC-specific: nucleus accumbens: t(33) = 4.41, p = 5.2x10− 5, pf we = 2.6x10− 4, d = 0.76, CI : [0.38, 1.15], globus pallidus: t(33) = 7.05, 2.3x10− 8, pf we = 1.1x10− 7, d = 1.21, CI : [0.77, 1.67], putamen: t(33) = 8.07, p = 1.3x10− 9, pf we = 6.5x10− 9, d = 1.38, CI : [0.92, 1.88], amygdala: t(33 = 1.78, p = 0.042, pf we = 0.212, d = 0.30, CI : [− 0.04, 0.66], thalamus: t(33) = 2.61, p = 0.007, pf we = 0.034, d = 0.45, CI : [0.09, 0.81]; TTC-generalized, nucleus accumbens: t(33) = 1.82, p = 0.039, pf we = 0.196, d = 0.31, CI : [− 0.04, 0.66], globus pallidus: t(33) = 7.06, p = 2.2x10− 8, pf we = 1.1x10− 7, d = 1.21, CI : [0.77, 1.68], putamen: t(33) = 6.21, p = 2.6x10− 7, pf we = 1.3x10− 6, d = 1.06, CI : [0.65, 1.50], amygdala: t(33) = 4.25, p = 8.3x10− 5, pf we = 4.1x10− 4, d = 0.73, CI : [0.35, 1.12], thalamus: t(33) = 4.05, p = 1.5x10− 4, pf we = 7.4x10− 4, d = 0.69, CI : [0.32, 1.08]). The subcortical ROIs (Fig. S2B) were based on the FreeSurfer parcellation as described in the section “Regions of interest definition and analysis”.
Hippocampal functional connectivity as a function of TTC-generalized learning
To examine which brain regions whose activity co-fluctuated with the one of the hippocampus during TTC-generalized learning, we again conducted a PPI analysis similar to the one described earlier. A spherical seed ROI with a radius of 4 mm was centered around the hippocampal group-level peak effect (x=-30, y=-24, z=-18) observed for the TTC-generalized learning regressor described above. To extract the average time course of the seed region, we first conducted a GLM for two regressors of interest: one modeling trials where TTC-generalized learning occurred, and another one modeling trials where TTC-generalized learning did not. We again added all nuisance regressors as described before. We then used an F-contrast to detect voxels with significant main effects within our seed region, and then averaged across these voxels to obtain the final seed time course. In the second GLM, we then added this time course as the PPI regressor and estimated the element-by-element product of the task time course and this seed ROI time course. We also included in the PPI GLM two nuisance regressors accounting for task-related effects from our contrast of interest (Behavioral improvements vs. no behavioral improvements) as well as physiological correlations that could arise due to anatomical connections to the hippocampal seed region or shared subcortical input. On the group-level, we then tested the weights estimated for our PPI regressor of interest against zero using a t-test implemented in SPM12. This revealed areas whose activity co-fluctuated with the one of the hippocampus with as a function TTC-generalized feedback learning.
Moreover, we conducted independent ROI analyses for subcortical regions as described in the section “Regions of interest definition and analysis”. Here, we tested the beta estimates obtained for the hippocampal seed-based PPI regressor of interest (One-tailed one-sample t tests: caudate: t(33) = 1.06, p = 0.149, pf we = 0.894, d = 0.18, CI : [− 0.16, 0.53], putamen: t(33) = 2.79, p = 0.004, pf we = 0.026, d = 0.48, CI : [0.12, 0.84], globus pallidus: t(33) = 2.52, p = 0.008, pf we = 0.050, d = 0.43, CI : [0.08, 0.79], amygdala: t(33) = 2.60, p = 0.007, pf we = 0.041, d = 0.45, CI : [0.09, 0.81], nucleus accumbens: t(33) = − 1.14, p = 0.869, pf we = 1, d = − 0.20, CI : [− 0.54, 0.15], thalamus: t(33) = 2.71, p = 0.005, pf we = 0.032, d = 0.46, CI : [0.11, 0.83]).
Brain activity as a function of behavioral performance and as a function of the behavioral regression effect
To examine the neural underpinnings governing specificity and generalization in timing behavior in detail, we analyzed the trial-wise activity of each voxel as a function performance in the TTC task (i.e. the difference between estimated and true TTC in each trial) and as a function of the regression effect in behavior (i.e. the difference between the estimated TTC and the mean of the sampled intervals, which was 0.82 s). To avoid effects of potential co-linearity between these regressors, we estimated model weights using two independent GLMs, which modeled the time course of each trial with either one of the two regressors. In addition, we again accounted for nuisance variance as described before, and all regressors except the realignment parameters and the constant term were convolved with the canonical HRF of SPM12. After fitting the model, we used the weights estimated for the two regressors to perform voxel-wise F-tests using SPM12, revealing activity that was correlated with these two regressors independent of the sign of the correlation. In addition, we again performed ROI analyses using two-tailed one-sample t-tests for the anterior and posterior hippocampus (S2A; TTC-task performance: anterior HPC, t(33) = 4.85, p = 2.9x10− 5, pf we = 8.7x10− 5, d = − 0.83, CI : [1.24, 0.44], posterior HPC, t(33) = 2.88, p = 0.007, pf we = 0.021, d = 0.49, CI : [0.86, 0.14]; Regression effect: anterior HPC, t(33) = 5.55, p = 3.6x10− 6, pf we = 1.1x10− 5, d = − 0.95, CI : [1.37, 0.55], posterior HPC, t(33) = 1.06, p = 0.295, pf we = 0.886, d = 0.18, CI : [0.53, 0.16]) as well as for the caudate (S2B; TTC-task performance: t(33) = 5.62, p = 2.9x10− 6, pf we = 8.7x10− 6, d = − 0.96, CI : [1.39, 0.56]; Regression effect: t(33) = 1.08, p = 0.287, pf we = 0.859, d = 0.19, CI : [0.16, 0.53]).
Eye tracking: Fixation quality does not affect the interpretation of our results
We used an MR-compatible infrared eye tracker with long-range optics (Eyelink 1000) to monitor gaze position at a rate of 500 hz during the experiment. After blink removal, the eye tracking data was linearly detrended, median centered, downsampled to the screen refresh rate of 120 hz and smoothed with a running-average kernel of 100 ms. There were no systematic biases in fixation error across speeds (Fig. S6A; Kruskal–Wallis test: χ(2) = 0.608, p = 0.895, ϵ2 = 0.005, CI : [0.00, 0.06]) or across feedback levels (Fig. S6B; χ(2) = 0.190, p = 0.909, ϵ2 = 0.0019, CI : [0.00, 0.10]). Moreover, differences in fixation error could neither explain individual differences in absolute TTC error (Fig. S6C; Spearman’s rho = 0.167, p = 0.344), nor individual differences in the regression effect in behavior (Fig. S6C; Spearman’s rho = 0.26, p = 0.131).
Supplementary Material
Behavioral analyses. A) Behavioral autocorrelation analysis. We did not observe a correlation between subsequent trials in the feedback participants received. The feedback and thus behavioral performance in one trial did therefore not predict feedback or performance in the following trial. B) Feedback distributions for all speed levels. Participant’s received approximately the same feedback for all speed levels and thus for all target TTCs. AB) Depicted are the mean and SEM across participants (black dot and line) overlaid on single participant data (dots). C) TTC task performance over time. Left panel: Across-trial-average performance over scanning runs. Right panel: task performance over trials. We plot the mean (black line) and SEM (shaded area) across participants. Run identity color coded. Participants’ task performance improved over time.
Regions of interest (ROIs). A) Anterior and posterior hippocampal ROIs. B) Caudate ROI. AB) ROIs shown for a sample participant superimposed onto the skull-stripped structural T1-scan of that participant. These masks were created using FreeSurfer’s cortical and subcortical parcellation.
Trial-phase specific relationship between brain activity and behavior. We repeated the voxel-wise mass-univariate general linear model analysis for performance in the current trial (Fig. 2) for each of the three trial phases individually. This included the tracking phase (in which the fixation target moved), the TTC-estimation phase (in which the fixation target had stopped and participants estimated the TTC) and the feedback phase (in which participants received feedback about how accurately they had estimated the TTC). We plot thresholded t-test results at 1mm resolution overlaid on a structural template brain. Positive t-scores indicate a positive relationship between brain activity and TTC-error.
Distinct networks support TTC-specific and TTC-generalized feedback learning. A) Voxel-wise mass-univariate GLM results for TTC-generalized and TTC-specific parametric regressors. We plot thresholded t-test results at 1mm resolution at p < 0.05 whole-brain FWE-corrected levels. Activity maps were overlaid on a structural template brain. Positive t-scores indicate a relationship between brain activity and the updating of either TTC-specific or TTC-generalized information respectively. Insert zooming in on hippocampus and MNI coordinates added. B) Subcortical regions-of-interest (ROIs) for the nucleus accumbens, the amygdala, the thalamus, the caudate, the putamen and the pallidum. All ROIs are shown for a sample participant superimposed onto the skull-stripped structural T1-scan of that participant. They were created using FreeSurfer’s cortical and subcortical parcellation. C) ROI-analysis results for subcortical regions for TTC-generalized (orange dots) and TTC-specific regressors (blue dots). Depicted are the mean and SEM across participants (black dot and line) overlaid on single participant data. Statistics reflect p<0.05 at Bonferroni-corrected levels (*) obtained using a group-level one-tailed one-sample t-test against zero.
TTC-generalized hippocampal connectivity. A) Regions of interest analysis for subcortical regions estimated using a Psychophysiological-interactions (PPI) analysis conducted using the hippocampal effect in Fig.4A as a seed. Positive beta estimates indicate that functional connectivity between each ROI and the hippocampal seed depended on how much participants TTC-task performance improved across trials. Depicted are the mean and SEM across participants (black dot and line) overlaid on single participant data for the nucleus accumbens, the amygdala, the caudate, the globus pallidum, the putamen and the thalamus. Statistics reflect p<0.05 at Bonferroni-corrected levels (*) obtained using a group-level one-tailed one-sample t-test against zero. B) Whole-brain voxel-wise t-test results for the TTC-generalized hippocampal connectivity overlaid on a structural template brain at 1mm resolution. MNI coordinates added.
Eye tracking analyses. A) Fixation error over speed. There were no significant differences in fixation error across speed levels and thus across target TTC’s. B) Fixation error over TTC-task performance. There were no significant differences in fixation error across TTC-task performance levels. C) No correlation of TTC-task performance or the behavioral regression-to-the-mean effect with fixation error. Fixation quality does not affect the interpretation of the imaging results presented in this study. Each dot represents a single participant. Regression line (black) and standard error (gray shade). AB) Group-level mean and SEM depicted as black dot and line overlaid on single participant data.
Acknowledgements
We thank Raymundo Machado de Azevedo Neto for helpful comments on an earlier version of this manuscript. CFD’s research is supported by the Max Planck Society, the Kavli Foundation, the European Research Council (ERC-CoG GEOCOG 724836), the Centre of Excellence scheme of the Research Council of Norway – Centre for Neural Computation (223262/F50), The Egil and Pauline Braathen and Fred Kavli Centre for Cortical Microcircuits and the National Infrastructure scheme of the Research Council of Norway – NORBRAIN (197467/F50).
Footnotes
↵* Shared-first authors
References
