Abstract
Moving from association to causal analysis of neuroimaging data is crucial to advance our understanding of brain function. The arrow-of-time (AoT), i.e., the known asymmetric nature of the passage of time, is the bedrock of causal structures shaping physical phenomena. However, almost all current time series metrics do not exploit this asymmetry, probably due to the difficulty to account for it in modelling frameworks. Here, we introduce an AoT-sensitive metric that captures the intensity of causal effects in multivariate time series, and apply it to high-resolution functional neuroimaging data. We find that that causal effects underlying brain function are more clearly localized in space and time than functional activity or connectivity, thereby allowing us to trace neural pathways recruited in different conditions. Overall, we provide a mapping of the causal brain that challenges the association paradigm of brain function.
Introduction
The advent of functional neuroimaging has provided us with unique insight into the complex spatiotemporal structure of brain function1. This organization is classically characterized on the basis of association assessments such as functional connectivity2 that was shown to reflect, e.g., cognitive status3,4 and disease5–7. However, the usefulness of this approach has been increasingly questioned as it bears crucial limits in understanding neural communication and pathways8,9. Therefore, it is crucial to move from association to causal frameworks to improve the interpretation of functional neuroimaging datasets10.
Various approaches have been proposed to extract causal structure from functional imaging time series. They include dynamic causal modelling11,12, multivariate autoregressive modelling13,14, Granger causality15,16, and more application-oriented variants of these17. Most, however, do not directly exploit the known asymmetric nature of the passage of time, also called the arrow-of-time18 (AoT, Fig. 1A). Since the cause and effect pattern fundamentally builds upon the AoT, we hypothesize that defining AoT-sensitive metrics of neuroimaging time series will provide unique insights into the causal structure of brain function.
To test this, we introduce a new AoT-sensitive multivariate metric and apply it to high-resolution functional magnetic resonance imaging (fMRI) time series from the Human Connectome Project19 (HCP). This metric is a multivariate extension of a previously defined measure20, and relies on the comparison of residuals of linear models identified from forward vs backward time series. More precisely, we define τ, the AoT strength, as the difference between non-Gaussianity of the residuals of multivariate autoregressive models of forward time series and backward time series (Fig. 1B & Eq. (2), details in Materials and Methods). These residuals are expected to be less Gaussian when computed from forward time series21, hence we expect τ to be positive. This metric is applied on fMRI data from 100 subjects in the resting state and when performing seven different tasks, thereby providing the AoT strength in each brain region, each condition, and as a function of time during paradigms.
We find that in almost all conditions, the AoT strength averaged over brain regions is positive, i.e., the AoT is detected in fMRI time series and shapes their dynamics. Then, we show that patterns of brain regions acting as causal triggers or targets are more localized in space and time as compared to classical activity or connectivity patterns, complementing the ‘networked-brain’ paradigm that has emerged in recent years22. Finally, the temporal fluctuations of τ during a task paradigm allowed us to identify a causal pathway of neural activations supporting the task. Overall, our results provide unique insight into the causal structure of brain function by leveraging the asymmetric nature of the passage of time to which almost all classical functional neuroimaging metrics are blind23.
Results
The AoT characterizes cognitive status
We first evaluate τ in all conditions as a function of the number of time points used. The AoT strength was computed for each brain region across 100 folds in which subjects were randomly ordered and their time courses were concatenated. The median across folds was taken as an estimate of regional AoT strength, and averaging was then performed across regions to derive a whole-brain AoT heuristic, referred to as . Fig. 2 (top) shows as a function of the total amount of considered samples and for all paradigms. In the resting state case (left panel), progressively increased as more time points were included, and started to plateau from ns = 8000 samples, at . Thus, when sufficient data is available, the AoT is detected in resting-state fMRI time series, confirming the presence of an underlying causal structure.
For task paradigms (middle and right panels), also progressively stabilized as more samples were used, but the asymptotic values differed from case to case: while no sizeable was detected for the gambling (purple) and emotion (yellow) tasks, it was negative for the social task (pink), and positive for the others at varying intensities. The largest AoT was obtained for the motor task, at . Thus, whole-brain AoT strength also varies as a function of the cognitive task being performed. The negative AoT found in the social task is surprising and suggests that a model assumption has been violated, e.g., the presence of an important non-observed variable, or spatial variation in hemodynamic delays.
For subsequent analyses, we focused on the results obtained using samples, as AoT convergence is observed with this amount of data. Fig. 2 (bottom) shows estimated AoT strength τ across regions as a violin plot for each paradigm, as well as when quantified from surrogate data having underwent amplitude-adjusted phase randomization24, i.e., non-causal null data. In the null case, τ was close to zero for all regions, spanning a narrower range of values than for any paradigm. With the exception of the emotion and gambling tasks, while median τ across regions was close to zero, mean τ was not, denoting that the aforementioned whole-brain causal effects are induced by a subset of brain areas.
Mapping the causal brain
To determine which brain regions exhibit a significant AoT, we compared them to their respective non-causal null distributions24. Fig. 3A shows the results at rest (left), and for the motor task when analyzing full recordings (center) or only task epochs (i.e., having excluded baseline periods, right). Fig. 3B summarizes network contributions to causal effects in all paradigms where contributions to positive and negative τ were distinguished. From Eq. (2), it is observed that a positive τ corresponds to the presence of a causal sink, i.e., the variable is the target of the causal effect. By symmetry, we associate negative values of τ to the presence of a causal source, i.e., the variable triggers the causal effect (details on the interpretation of positive and negative AoT values are found in the Materials and Methods).
At rest, 184 regions (43.91%) showed a significant AoT, with a mild right lateralization, and positive-valued τ dominated (130 to 54 negative values). The most influential areas primarily spanned the temporal, prefrontal and parietal cortices, and belonged to the default mode and fronto-parietal control networks. Some canonical hubs of these high-level networks showed little significance, such as the posterior cingulate cortex. During the motor task, 284 regions (67.78%) displayed significant causal effects, with no lateralization, and positive values still dominated (214 to 70 negative values). Contributions from the limbic and somatomotor networks were seen in addition to the default mode and fronto-parietal control ones. When excluding baseline moments, 333 regions (79.47%) became significant, with no evident lateralization, and positive values continued to be more prominent (237 to 96 negative ones). Contributions within the somatomotor cortical stripe became stronger, and some other areas with marked negative values were also newly resolved with regard to the two above cases, such as a low-level visual region (R218, VIS18) and a prefrontal region (R178, PFC13). Overall, these result support the presence of stronger causal mechanisms when a subject engages into the motor task as compared to the resting state.
More broadly across all task paradigms (Fig. 3B), negative-valued τ was consistently primarily observed within the visual network, indicating that it always acts as a causal trigger (note that this effect is not observed at rest). This network was also dominant in terms of positive contributions for the working memory and the relational tasks, indicating that it also acts as a causal target in these tasks.
From causal maps to neural mechanisms
The differences found between full and task-only recordings (Fig. 3A, middle-right) hint at strong temporal fluctuations of the AoT. To ascertain this, we performed a sliding window analysis on the motor task paradigm with a window width of W = 20 time points slid by one sample until a full AoT strength time course is computed for each region, and using data from all 100 subjects (Fig. 4A, top). Obtained results were contrasted to the activity time courses temporally smoothed with a moving average filter of length W, and to dynamic functional connectivity time courses generated using identical window settings and Pearson’s correlation coefficient as functional connectivity measure. In this latter case, we derived a regional measure by summing all functional connections of an area to the rest of the brain within each temporal window.
As expected, clear increases in activity occurred during each of the task epochs in motor regions subserving hand, foot or tongue movement. Connectivity of a given region to the rest of the brain was consistently either positive (denoting a temporally stable regime with more prominent correlation to the rest of the brain), or negative (more prominent anti-correlation). On the whole, activity and connectivity fluctuations were relatively diffuse in time (spanning full task epochs) and in space (involving many different areas). In contrast, causal effect time courses were highly localized in space (typically only applying to individual regions at any given time point), and occurred within shorter time intervals with fast transition from positive (causal target) to negative (causal source) values.
Fig. 4B exemplifies the evolution of causal effects when transiting from baseline to the first tongue movement epoch (see highlighted area in panel A, bottom), for the four left hemispheric brain regions with the largest extent of temporal fluctuations of τ within this interval. Consistent with the paradigm’s demands, these regions were motor (SM12 and SM14, for tongue movement), visual (VIS24, for parsing the provided instructions), and prefrontal (PFC13, to trigger movement execution). When the visual cue is provided to the subjects, VIS24 becomes a causal sink. Shortly afterwards, PFC13 becomes a sink, as visual information is treated frontally to make the decision to move. This information is then transmitted to the rest of the brain, as PFC13 becomes a causal source (see the temporally localized negative values in its time course), while SM14 and, later on, SM12 become sinks. Finally, SM14 further transmits the information and becomes a source to trigger motion. Fig. 4C schematically summarizes these observations.
Discussion
Here, we introduced a new AoT-sensitive metric that captures causal effects in multivariate time series. Applied to fMRI data, we showed that causal effects (i) shape brain function in all conditions, (ii) are highly localized in space and time, and (iii) reflect underlying neural mechanisms. These results are found to be robust to head motion, to the use of a different metric of non-Gaussianity, and to varying processing strategies (see Supplementary Material). While causality has been assessed in neuroscience and neuroimaging using other methods17,26–28, this is to the best of our knowledge the first use of the AoT to interrogate causality in neuroimaging data, thereby providing a new and natural description of the causal brain.
The AoT provides a new perspective into the causal structure of time series
The term ‘arrow-of-time’ has been coined by Sir A. Eddington almost a century ago to express this one-way property of time which has no analogue in space18. Rather surprisingly, identifying the AoT from time series is not trivial and most current AoT detection methods rely on deep learning29–31. Other approaches instead exploit simpler features such as the distribution20 or the independence32 of linear model residuals in forward and backward time series. The latter measures, from which we defined τ in Eq. (2), also come with a natural interpretation in terms of causality as they leverage causal inference theory to detect the AoT21,32. Therefore, the interpretation of τ in terms of causality comes with all causal inference assumptions and guarantees, which is not necessarily the case of other causality detection methods used in neuroimaging studies that encode different forms of causality23,33.
Identifying causal effects rather than association effects in multivariate time series comes with estimation challenges. For example, it is seen from Fig. 2 (see also Supplementary Material for further evidence) that at least ~ 1000 fMRI time points are required to identify stable AoT patterns. In contrast, stable patterns of functional connectivity, i.e., of correlation, can be identified from as little as around 100 fMRI time points34. Exploiting the non-Gaussianity of time series through kurtosis also requires cautious estimation of group effects as this metric relates to outliers in a distribution. For this reason, we took several precautions to maximize the stability of our maps: we evaluated our group (original and null) results from the median over folds (thus accounting for the selection of different subjects and making our results more generalizable), and adopted the most efficient sample selection scheme after evaluating several candidates (see Supplementary Material). Resorting to non-Gaussianity of linear models was important in order to unambiguously identify causal structures; indeed, linear-Gaussian approaches usually only lead to a class of possible models, a.k.a. Markov equivalence class, equivalent in their conditional correlation structure and from which no unique causal structure can be inferred21,35.
The association brain vs the causal brain
The current perception of brain function has been built from association metrics of functional neuroimaging data, thus probing the ‘association brain’. For example, functional connectivity2,36,37 canonical resting-state networks1,25, and most representations of brain dynamics such as (innovation-driven) co-activation patterns38,39, dynamic modes40, or sliding window-based states41–43 are defined from association metrics, e.g., correlation, which are blind to causality. By leveraging advances in causal inference, we defined a simple metric that exploits time series asymmetry induced by causal effects. This shift of the methodological paradigm lays the ground to a shift of canonical representations of brain function and dynamics. Furthermore, a causal representation of brain function also comes with promises for the cognitive and clinical use of neuroimaging data as the causal brain is expected to reflect underlying neural mechanisms9, as illustrated in Fig. 4B/C. Recent neuroimaging endeavours further substantiate this potential: after training a deep learning network to distinguish between temporal segments of forward and backward fMRI time series, Deco et al.31 not only observed a variable AoT strength (inferred from classification accuracy on unseen data) across cognitive states, but also between healthy subjects and patients suffering from bipolar disorder, attention deficit hyperactivity disorder or schizophrenia. In another study leveraging the same framework on electrocorticography data, de la Fuente et al.44 also revealed that deep sleep and ketamine-induced anesthesia lowered the differences between forward time series and their inverted counterparts, i.e., decreased AoT strength.
Our results show that the topology of the the causal brain exhibits strong differences as compared to the association brain. Specifically, the dynamic tracking of the AoT in Fig. 4A revealed how remarkably localized it was with regard to functional activation and connectivity. While these two common measures reflect the overall simultaneity in activation across regions, when information has already arrived and been locally amplified (for instance, somatomotor areas in our motor task example), our AoT metric captures the arrival and departure of information. It thus more finely pinpoints the spatial entry and exit points of neural pathways, as well as their exact temporality. As a consequence, time-averaged representations of the causal brain might be harder to interpret as they destroy the rich temporal structure of causal effects (Fig. 3A). In particular, further work will be required to efficiently characterize the causal brain, e.g., through causal networks accounting for its specificities. Finally, the present association vs causal brain dichotomy differs from the one between functional and effective connectivity2. Indeed, using the current causal inference nomenclature, most functional and effective connectivity measures would be classified as association measures (see, e.g., the discussion on the nature of Granger causality in Pearl et al.23, Chap. I).
Limitations and further considerations
The proposed characterization of causal effects comes with the assumptions and limitations of the modelling framework in Eqs. (1)-(2). In particular, we limit our assessment to linear and non-Gaussian causal effects. This is motivated by the indeterminacy inherent to linear-Gaussian assessments21, but does not mean that causal effects cannot be Gaussian. Future work will explore whether relaxing these assumptions, e.g., using convergent cross mapping45 or other nonlinear approaches46, provides new insights into the causal brain. Robustness to violation of causal sufficiency, i.e., the presence of non-observed variables, would also need to be further assessed47,48, potentially by including additional experimental variables of interest such as a record of the visual cue or electrophysiological variables. Then, comparisons across paradigms must be interpreted with caution as while the total number of samples was the same, the length of the paradigms was different. Thus, a distinct number of subjects contributed to the estimates in each case. This directly relates to the question of individual as opposed to population-wise causal effects, and further work will explore the potential of the causal brain as a subject-level marker49,50. Finally, our framework is directly applicable to other neuroimaging modalities, e.g., electro- or magneto-encephalography, but also outside of neuroimaging to any multivariate time series dataset.
Conclusion
Together, our findings suggest that a causal assessment of neuroimaging data indeed provides new insights into the neural mechanisms underlying brain function. More precisely, our mapping of the causal brain hints at key differences as compared to association paradigms of brain function during rest and task, e.g., in terms of spatial and temporal localization. In light of this, brain imaging studies have an opportunity to move beyond classical association paradigms and unveil information contained in neuroimaging data to which current metrics are blind.
Materials and Methods
Data acquisition and preprocessing
We considered S = 100 unrelated healthy subjects from the Human Connectome Project S900 data release (46 males, 54 females, mean age = 29.1 ± 3.7 years). We used fMRI recordings acquired at rest and during 7 tasks (emotion, gambling, language, motor, relational, social, working memory), for which ethical approval was obtained within the HCP. Our analyses focused on the first of two available resting state sessions, and on each available task session, purely on the left-right phase encoding direction runs. Right-left phase encoding data were examined in supplementary analyses (see Supplementary Material).
To generate regional fMRI time courses, for each run of interest, minimally preprocessed data from the HCP19,51 were taken as input. Nuisance signals were first removed from the voxel-wise fMRI time courses, including linear and quadratic trends, the six motion parameters and their first derivatives, as well as the average white matter and cerebrospinal fluid signals and their first derivatives. In our main analyses, the global signal was also included as a confounding variable. In additional analyses (see Supplementary Material), we contrasted the obtained results to those without global signal regression, and also examined the impacts of performing scrubbing as a final preprocessing step. Voxel-wise time courses were averaged within each region of a parcellation containing 400 cortical52 and 19 subcortical51,53 areas, for a total of R = 419 parcels, and eventually z-scored. To complement these analyses, we also considered cortical atlases containing 200 and 800 regions 52 (see Supplementary Material).
AoT quantification
To quantify AoT strength across brain regions, we extend a univariate metric defined previously20 to the multivariate case. First, we fit a first-order multivariate autoregressive model to concatenated fMRI time series population-wise54, both in the forward and in the backward directions as shown in Eq. (1): where xt is of size R × 1, Af and Ab each have size R × R, and the residuals and are of size R × 1. The model parameters are estimated using ordinary least squares55, and successive samples that originate from separate subjects (owing to the concatenation step) are excluded. Then, the presence of causal effects in different brain regions is assessed by comparing non-Gaussianity of forward and backward residuals. This was motivated by the fact that residuals of linear models of true cause-effect links (in this case, the forward model) are more non-Gaussian than the residuals of the reversed linear models (in this case, the backward model)21. Concretely, with T the total number of time points, we define and as the forward and backward error distributions. Regional AoT strength τ(i) is then estimated as: where K(·) denotes the kurtosis of a distribution, and stands for the standard normal distribution. In the case of a marked AoT, non-Gaussianity of residuals is larger in the forward than in the backward model, and τ(i) is positive. Region i is then a causal sink, primarily receiving information from the rest of the brain. By symmetry, we say that if τ(i) is negative, brain region i is a causal source. Note, however, that a negative value of τ suggests that one model assumption has been violated, e.g., due to the presence of an unobserved variable, or due to different delays in hemodynamic responses, and interpretation of negative values of τ(i) should be cautious. Finally, we also devised an alternative metric relying on the Kullback-Leibler divergence to quantify AoT strength (see Supplementary Material for details).
Regional AoT patterns
Using samples, regional AoT patterns were extracted for each paradigm of interest. For the compatible tasks, the same process was also conducted after the removal of baseline epochs. To do so, individual binarized paradigm time courses (0=rest, 1=task) were convolved with the canonical haemodynamic response function from SPM12, and resulting time points with a value larger/lower than 0.5 were treated as task/rest samples. Of note, since less samples are then available per subject, the obtained AoT estimates gather data from a more extended set of subjects compared to the full recording case.
To study the contribution of separate networks to the AoT patterns, each cortical brain region was assigned to one of seven canonical whole-brain resting state networks25 through a majority voting procedure. Positive- and negative-valued AoT contributions were separately quantified.
Significance assessment
To assess AoT significance, comparison was performed to null data for which causal effects were destroyed. For this purpose, for each paradigm at hand, amplitude-adjusted phase randomization24 was applied to the original time courses to generate nn = 100 null realizations. We considered this surrogate procedure in order to destroy causal effects while preserving the original auto-correlation structure and sampling distribution, including potential non-Gaussian effects. For each set of null data, using samples, AoT strength was calculated across 100 folds, and the median was taken as an estimate of null regional AoT strength. The mean and standard deviation were quantified for each regional null distribution, and τ was deemed significant if it exceeded the Bonferroni-corrected or null percentiles (τ- and τ+ in Fig. 3, respectively).
Software availability
All the scripts used in this work were implemented and tested in MATLAB, versions 2014b, 2020b and 2021a (MathWorks, Natick, MA, USA). They can be freely downloaded from the following link: https://github.com/TiBiUan/AoT_Benchmarking.git. For figure generation, we used the cbrewer and BrainNet Viewer56 (version 1.7) utilities.
Acknowledgements
RL acknowledges support by the Swiss National Centre of Competence in Research - Evolving Language (grant number 51NF40_180888). MGP was supported by the CIBM Center for Biomedical Imaging, a Swiss research center of excellence founded and supported by Lausanne University Hospital (CHUV), University of Lausanne (UNIL), Ecole Polytechnique Fédérale de Lausanne (EPFL), University of Geneva (UNIGE) and Geneva University Hospitals (HUG).
References
- [1].↵
- [2].↵
- [3].↵
- [4].↵
- [5].↵
- [6].
- [7].↵
- [8].↵
- [9].↵
- [10].↵
- [11].↵
- [12].↵
- [13].↵
- [14].↵
- [15].↵
- [16].↵
- [17].↵
- [18].↵
- [19].↵
- [20].↵
- [21].↵
- [22].↵
- [23].↵
- [24].↵
- [25].↵
- [26].↵
- [27].
- [28].↵
- [29].↵
- [30].
- [31].↵
- [32].↵
- [33].↵
- [34].↵
- [35].↵
- [36].↵
- [37].↵
- [38].↵
- [39].↵
- [40].↵
- [41].↵
- [42].
- [43].↵
- [44].↵
- [45].↵
- [46].↵
- [47].↵
- [48].↵
- [49].↵
- [50].↵
- [51].↵
- [52].↵
- [53].↵
- [54].↵
- [55].↵
- [56].↵