ABSTRACT
The human brain consists of functionally specialized areas, which flexibly interact and integrate forming a multitude of complex functional networks. The principles underlying this functional differentiation and integration remain unknown. Here, we demonstrate that a fundamental principle ubiquitous in nature - harmonic modes - explains the orchestration of the brain’s functional organization. Applied to the functional connectivity in resting state averaged across 812 participants, harmonic modes give rise to functional harmonics revealing the communication channels of the human brain. Remarkably, the isolines of the continuous functional harmonic patterns (gradients) overlap with the borders of cortical areas. Furthermore, each associated with a different spatial frequency, the functional harmonics provide the frequency-ordered building blocks to reconstruct any pattern of brain activity. We show that 47 brain activation patterns elicited by 7 different task categories in the Human Connectome Project task battery can be reconstructed from a very small subset of functional harmonics, uncovering a parsimonious description of the previously unknown relationship between task and resting state brain activity. Crucially, functional harmonics outperform other well-known basis functions such as those used in principle component analysis (PCA) or independent component analysis (ICA) in both, reconstructing the task activation maps as well as explaining the emergence of functionally specialized regions. Thus, our findings not only unify two competing views of the brain’s functional organization, i.e. modular vs gradiental perspective, by revealing that the functional specialization of the human cortex occurs in a gradiental manner across multiple dimensions in the functional harmonic basis, but also evidence that this basis underlies task-elicited human brain function.
Introduction
The human brain is topographically organized into functionally specialized brain areas1. Integration of these areas in various different constellations allows for the immense complexity of human brain function2. Despite remarkable progress in mapping the brain into functionally meaningful subdivisions, known as cortical areas3, 4, and in identifying functionally relevant combinations of these areas forming the functional networks of the brain5, the principles governing this functional segregation and integration in the human brain have remained unknown. Here we demonstrate that a fundamental principle ubiquitous in nature, i.e. harmonic modes, when applied to functional connectivity data in humans, reveals both, the brain’s functional networks as well as its topographic organization.
The topographic organization of the brain into functionally specialized areas is one of its fundamental properties, observed in evolution as early as the last common ancestor of vertebrates4, 6. The individuality of each brain area is determined by its functional specification, its microstructure (cyto- and myeloarchitecture)4, and its inter- and intra-area connectivity3. Significant effort in neuroscience has been directed towards subdividing the brain into adjoining parcels, which are assumed to have uniform functional specification and homogeneous connectivity3, 4. A multitude of functionally distinct brain areas coordinate through synchronous fluctuations in their activity7. Coherent oscillations among distinct brain areas have been shown to be another evolutionarily conserved aspect of brain activity8. The overlap of the networks formed through these spontaneous system oscillations, termed the functional connectivity patterns, with the functional networks of the human brain identified by various sensory, motor, and cognitive task paradigms9–12, strongly indicates their relevance for the brain’s functionality.
However, this modular view of brain organization, where separate, adjoining brain areas with uniform functionality and homogeneous structural connectivity integrate into functional networks through coherent oscillations, has been challenged by the presence of gradually varying boundaries between brain areas suggesting a degree of transition instead of sharply separated brain areas13, as well as by the existence of topographic mappings, which characterize the differences within a functionally specific brain area14–16. Topographic mappings including retinotopy14, somatotopy15, tonotopy16, show that representation of our visual field, body and auditory frequency spectrum are spatially continuously represented across the areas of the primary visual, somatomotor and auditory cortices, respectively, challenging the assumption of uniform functionality within the determined brain areas and demonstrating a smoothly varying functionality13. As an alternative, theoretical work17, 18 and recent experimental findings13 suggested a “gradiental perspective”, where the functional organization of the cortex is argued to be continuous, interactive and emergent as opposed to mosaic, modular and prededicated17. Similar to the smoothly varying functionality of primary sensory and motor areas, association cortices functioning as integration centres for more complex or elaborated mental processes are hypothesized to emerge from the convergence of information across sensory modalities18 with increasing spatial distance on the cortex from the highly functionally specialized primary cortices19. Supporting this hypothesis, a principal connectivity gradient of cortical organization in the human connectome has been identified, where the functional networks of the human brain are located according to a functional spectrum from perception and action to more abstract cognitive functions13. Although converging evidence13, 20, 21 supports the continuous and emergent view of cortical organization, the principles underlying the functional organization in the brain remain largely unknown.
Here, we demonstrate that the functional segregation and integration in the brain are governed by the same natural principle of harmonic modes that underlies a multitude of physical and biological phenomena including the emergence of harmonic waves (modes) encountered in acoustics22, optics23, electron orbits24, 25, electro-magnetism26, 27 and morphogenesis28, 29. By solving the time-independent (standing) wave equation30, 31 on the functional connectivity (FC) structure of the human brain, we uncover the spatial shapes of the harmonic modes emerging from synchronous hemodynamic fluctuations in large scale brain activity as measured with functional magnetic brain imaging (fMRI). These harmonic modes decompose the functional connectivity into a hierarchical set of (spatial) frequency-specific communication channels, which naturally emerge from coherent, spontaneous brain activity, and unveil both, the principal connectivity gradient13, as well as cortical parcellations3. Our results indicate that the functional segregation and integration in the brain are governed by a multi-dimensional harmonic representation that we call “functional harmonics”. Finally, the decomposition of the brain activity maps elicited by various cognitive tasks into the set of functional harmonics reveals that each task primarily involves activation of a very small subset of functional harmonics, suggesting that the functional harmonics reveal fundamental building blocks of not only resting state activity, but also various cognitive functions.
Estimation of functional harmonics
Mathematically, the patterns of harmonic modes of a dynamical system are estimated by the eigendecomposition of the Laplace operator, which lies at the heart of theories of heat, light, sound, electricity, magnetism, gravitation and fluid mechanics32. In vibrating systems, eigenfunctions of the Laplacian constitute standing waves, which also have been proposed as the mechanism underlying cortical communication observed in electroencephalogram (EEG) data33. Theoretical studies as well as experimental findings have shown that spherical harmonics, i.e. Laplace eigenfunctions on a sphere, underlie cortical activation patterns in fMRI34. Harmonic modes of the structural connectivity of the human brain, i.e. Laplace eigenfunctions on the human connectome, have been found to predict the collective dynamics of cortical activity at the macroscopic scale, and reveal resting state networks31.
In this work, we hypothesized that the harmonic modes of the brain’s communication structure given by its functional connectivity underlie its functional integration and segregation. There are several crucial properties of harmonic modes that led us to form our hypothesis:
The dense functional connectivity (dense FC) matrix, in our study estimated from the pairwise temporal correlations between all pairs of vertices on the cortical surface (59.412 vertices in total), encodes the communication structure of the human brain. In order to find a multi-dimensional representation that best preserves this functional communication structure, we utilized the discrete counterpart of the harmonic modes defined on a graph, i.e. the eigenvectors of the graph Laplacian, which have been shown to optimally preserve the local graph structure while embedding it into a lower-dimensional space35. Hence, the functional harmonics estimated by the harmonic modes of the dense FC in this work, reveal the optimal multi-dimensional mapping between the communication structure of the brain given by the dense FC and the cortical surface in the sense that the strongest functional relationships given by the largest correlation values are optimally preserved.
Functional harmonics are the smoothest patterns that respect the constraints posed by the functional relationships given by the FC35. This implies that the average difference between neighbouring nodes in a graph representation is minimized. Intriguingly, theoretical work has shown that activation patterns on graphs in which neighbouring nodes co-activate lead to patterns with minimum free energy or entropy36, 37, and that the transition between such patterns requires minimal energy38.
Due to their orthogonality, Laplace eigenfunctions provide a new function basis. When applied to a one-dimensional domain with cyclic boundary conditions, i.e. to a circle, Laplace eigenfunctions constitute the well-known Fourier basis, whereas on a sphere, they yield the spherical harmonics. Each eigenfunction corresponds to a unique eigenvalue related to its spatial frequency, and the set of all eigenfunctions forms a function basis, in which any signal can be represented in the frequency domain. Considering this particular aspect of harmonic patterns, functional harmonics provide a new frequency-specific function basis driven by the brain’s communication structure (dense FC), where each dimension provides a frequency-specific communication channel on the cortex.
The eigenfunctions of the Laplacian explain self-organizing patterns in many dynamical systems24–26, 28, 39, ranging from relatively simple physical phenomena like vibrating strings and metal plates22 to complex biological processes such as biological pattern formation and morphogenesis28, 29.
Considering that functional harmonics provide an optimal, frequency-specific mapping of the brain’s communication structure to the cortex; that they represent the most energy-efficient activation patterns which respect the constraints posed by this communication structure; and given the ubiquity of harmonics in nature, we hypothesized that functional harmonics provide the ideal candidate to explain functional segregation and integration in the brain. In order to test this hypothesis, we used the dense FC computed from resting state fMRI data averaged across 812 subjects, provided by the Human Connectome Project (HCP) 900 subjects data release40–47.We obtained the functional harmonics by estimating the eigenvectors of the graph Laplacian computed on the graph representation of the FC (Figure 1). We compared our results to five alternative function bases. In order to test the effect of each step in our processing pipeline, we compared the performance of the functional harmonics first to that of the eigenvectors of the dense FC matrix (Figure 1c, SI Figure 5); second to the eigenvectors of the adjacency matrix (SI Figure 4), which is obtained after thresholding and binarizing the dense FC matrix, and which encodes the graph of the brain’s communication structure (Figure 1d); and third to a surrogate harmonic basis created by applying spherical rotations to the functional harmonics48(SI Figure 8 for an example). Furthermore, to relate the performance of functional harmonics to other well-known function bases, we also performed comparisons to the basis functions of PCA (SI Figure 6) and ICA (SI Figure 7).
Functional harmonics reveal functionally relevant communication channels
We first investigated whether functional harmonics yield functionally meaningful communication channels, i.e. patterns of correlated activity. Figure 2 shows the first 11 non-constant functional harmonics (referred to as [ψ1, ψ2, ⋯, ψ11]), ordered starting from the lowest eigenvalue, illustrating that each harmonic is a smoothly varying pattern on the cortex between a positive and a negative polarity; i.e., a gradient. There is an intrinsic relation between the Laplace eigenvalues and the spatial frequency/wavelength; namely as the eigenvalue increases, spatial frequency also increases, while the spatial wavelength decreases. Hence with increasing eigenvalue, the functional harmonics become increasingly more complex and segregate the cortex into an increasing number of nodal areas30 (contiguous areas of the cortex with similar colors in the surface plots in Figure 2). This means that functional harmonics yield not only a multi-dimensional, but a multiscale description of the cortex. Note that the ordering by the wavelength/frequency is a property that emerges from the Laplacian and therefore only applies to the functional harmonics themselves and, by definition, their rotations; whereas other function bases used as controls in this study, shown in SI Figures 4-7, are not ordered by wavelength (or equivalently wavenumber) and thus do not implicitly possess this multiscale property.
As shown in Figure 2, functional harmonic resemble known functional systems and brain areas. In order to test the degree of this correspondence, we quantified the overall overlap between functional harmonic patterns and the brain regions (parcels) delineated by the HCP parcellation3. To this end, for each of the functional harmonics shown in Figure 2, we compared the within- and between-area-variability of each cortical region, where a large difference between the within- and between-area variability, indexed by a large silhouette value, indicates that that particular region is well-separated from the rest of the cortex49. We compared the resulting values to those obtained from spherical rotations of the functional harmonics, in which we rotated the functional harmonic patterns on a spherical version of the cortical surface48 (see SI Figure 8 for an example). This control still yields smooth, symmetrical harmonic patterns on the cortex, but they do not emerge from the communication structure (FC matrix) of the brain and are not necessarily orthonormal. Furthermore, we repeated this analysis for the other four function bases (FC eigenvectors, adjacency eigenvectors, PCA and ICA), using spherical rotations of these basis functions. As shown in Figure 3a, we found an alignment between the isolines of the functional harmonics and parcel borders for each of the first 11 functional harmonics, as verified by significantly larger silhouette values for functional harmonics compared to the rotated harmonic basis (pcorr < 0.05 after Bonferroni correction, Monte Carlo tests; see Online Methods for details). The only exception to this alignment was functional harmonic 4 (ψ4), which captures the retinotopic organization of early visual regions (see below for a discussion of retinotopic organization of functional harmonics). Importantly, this was not the case for any of the control function bases, where in each case at least some of the first 11 basis functions and their rotations performed equally well (Figure 3b-e). For qualitative evaluation, the overlap between parcels and functional harmonics as well as other bases is shown in SI Figures 3-7.
In the following, we provide some insight into the functional significance of each of the functional harmonics shown in Figure 2. Functional harmonics 1 (ψ1) and 2 (ψ2) correspond to previously identified large-scale gradients13 that delineate the separation between the major sensory and the uni-vs. multimodal cortices in the brain, respectively (see SI Figure 1a). Figure 2a and b demonstrate the overlap between the visual and sensorimotor networks as defined in Yeo et al. (2011)50 and the gradiental patterns of the first and second functional harmonics. We observed that functional harmonic 3 (ψ3) reveals a finer subdivision of the somatosensory/motor system51–53. The overlay of the borders of the five somatotopic areas defined by the HCP3, 54 on the third functional harmonic are shown in Figure 2c. Similarly, in functional harmonic 4 (ψ4), we found a finer segregation of the visual system, following a retinotopic eccentricity gradient (for further details on retinotopic mapping see the following section)55. The overlay of the borders of early visual areas (V1-V4) on functional harmonic 4 (ψ4) are shown in Figure 2d.
The regions found in the positive polarity of functional harmonic 5 (ψ5) (borders shown on the left 3 panels in Figure 2e) closely resemble the sensory-motor pathway50 (Figure 2e), and are known to be modulated by visuospatial attention56. In the negative polarity, we found the auditory cortex and parts of the somatosensory/motor network (Figure 2e). In contrast, in functional harmonic 6 (ψ6), auditory and visual areas were both localized in the positive polarity, forming a network related to audiovisual object (including faces) recognition57–59, i.e. recognition of the “outer world”. The negative polarity of functional harmonic 6 (ψ6) segregates the somatotopic face area as well as parts of the default mode network (DMN), a network of regions whose activity has been related to self-referential tasks60. Thus, the negative polarity of functional harmonic 6 (ψ6) forms a self-referential processing stream60–62. Functional harmonic 7 (ψ7) provides a further somatotopic gradient, including a higher hand area, 24dd, in the medial cortex51 (see Figure 2g and annotations in Figure 2c). Functional harmonics 8 to 10 (ψ8, ψ9, ψ10) correspond to different subdivisions of higher order networks such as the frontoparietal network and DMN (see SI Figure 2). In particular, the DMN63 is delineated in the positive polarity of functional harmonic 9 (ψ9) (borders of the DMN as defined by Yeo et al. (2011)50 are overlaid on functional harmonic 9 (ψ9) in Figure 2i). Functional harmonic 11 (ψ11), the first asymmetric harmonic between the two hemispheres, yields the separation between the right and left somatotopic hand areas64. Overall, these results demonstrate that functional harmonics provide a multitude of functionally relevant communication channels, each associated with a unique spatial frequency, and enable a set of parallel processing streams in the human brain.
Functional harmonics reveal brain areas and topographic mappings
The fact that functional harmonics display both, well-delineated specialized regions; e.g. in functional harmonics 3 (ψ3), 7 (ψ7), and 11 (ψ11), also evident by the large silhouette values mentioned in the previous section, as well as gradients that integrate brain regions from different functional systems; e.g. in functional harmonics 5 (ψ5) and 6 (ψ6), led us to hypothesize that functional harmonics provide a unifying explanation for two seemingly opposing perspectives of cortical organization: the gradiental perspective arguing that cortical organization is continuous on the one hand and the modular perspective stating that brain function emerges from modular organization of specialized brain regions on the other. We therefore explicitly tested whether functional harmonics fulfill the constraints posed by both of these views.
In addition to the parcels delineated in the HCP parcellation3, we investigated whether functional harmonics also capture somatotopy15 and retinotopy14, two major topographic mappings found in the brain. Topographic mappings represent sensory input on the cortical surface such that the relative positions of the receptors, which receive these inputs, are preserved. Five somatotopic sub-areas (in each hemisphere) as defined by the HCP3 form a topographic map of the surface of the body on the cortex, i.e., the face, hands, eyes, feet, and trunk.
To quantify the degree to which each somatotopic sub-area is delineated within functional harmonics 3 (ψ3), 7 (ψ7), and 11 (ψ11), we again utilized the within- and between-area-variability as above, but applied this measure specifically to the 10 somatotopic sub-areas (see SI Figure 9). We measured their separation both from the rest of the brain as well as from other somatotopic areas. We found that for each of the tested functional harmonics, at least one somatotopic region is significantly separated (pcorr < 0.05 after Bonferroni correction, Monte Carlo tests with 300 permutations). This finding indicates that functional harmonics capture somatotopic organization in the cortex. Figure 4a illustrates the two-dimensional subspace formed by functional harmonics 3 harmonics 3 (ψ3), and 11 (ψ11), which strikingly accounts for the precise mapping of the human body onto the somatotopic regions of the cortex (see SI Figure 1b-d for further examples).
We next investigated the presence of retinotopic mapping of early visual regions (V1-V4), where cortical representations of the visual field reflect the positions of the receptors such that each vertex within the patterns of functional harmonics is assigned an eccentricity (distance from the fovea) and an angle (top, bottom, left, right).55 To investigate the degree of agreement between the functional harmonics and the retinotopic mappings, we measured the correlation between eccentricity as well as polar angle maps and the functional harmonic patterns in V1-V4. We found significant correlations (pcorr < 0.05 after Bonferroni correction) between the retinotopic eccentricity map and all harmonics except functional harmonic 9 (ψ9); and between the retinotopic angular map and harmonics 1-4 (ψ1,⋯, ψ4), 7-9 (ψ7,⋯, ψ9), and 11 (ψ11). Examples of polar plots of the retinotopic gradients are shown in Figure 4c, d (all polar plots are shown in SI Figure 10).
Besides the two major topographic mappings of the cortex, we observed that functional harmonic 10 (ψ10) captures the hierarchical organization of the auditory system. To quantify this agreement, we measured the correlation between the spatial pattern of functional harmonic 10 (ψ10) and the extent to which each area is associated with the auditory network in the resting state (degree of auditory involvement)3. We found a significant correlation (r = −0.63, p =4·10-21) between functional harmonic 10 (ψ10) and the degree of auditory involvement of the functional areas (Figure 4b).
Functional harmonics are basis functions of human cognition
Considering the parallel between functional harmonics and the well-known Fourier basis, i.e. the fact that they both are defined as Laplacian eigenfunctions, the former applied to a one-dimensional domain with cyclic boundary conditions (a circle) and the latter to the communication structure of the human brain (dense FC matrix), functional harmonics provide an extension of the Fourier basis to the communication structure of the human brain. As such, they provide per definition a frequency-specific function basis, in which any pattern of brain activity can be represented as a weighted combination of functional harmonics. Given the experimental evidence showing that resting state functional connectivity reflects connectivity during task9–12, we hypothesized that functional harmonics provide building blocks of task activity measured on the cortex. In order to test this hypothesis, we reconstructed 47 group-level task maps provided by the HCP54 from the functional harmonics (see Online Methods). The 47 maps consist of activation maps as well as contrasts derived from 7 groups of tasks (working memory, motor, gambling, language, social, emotional, relational - see Online Methods for summaries). The functional harmonic reconstruction yields a coefficient (weight) for each functional harmonic, quantifying how much it contributes to a certain task map. The set of all coefficients forms a spectrum equivalent to the power spectrum obtained from a Fourier transform, in this case the power spectrum of the functional harmonic basis.
We first tested whether it is possible to approximate task maps as superpositions of subsets of functional harmonics, linearly combining them in the order of their eigenvalues. We quantified the goodness of fit by measuring the distance between the original and the reconstructed task maps. Figure 5a-g shows the average normalized reconstruction errors for all groups of tasks and for all function bases: for the functional harmonic basis (red line), the error drops from about 1.00 to about 0.5 when using only the first 11 functional harmonics shown in Figure 2, corresponding to 0.02% of the total functional harmonic spectrum. This corresponds to a level of correlation of around 0.7 between the original and reconstructed task maps (see SI Figure 11b). Figure 5h illustrates the reconstruction procedure for one specific task (working memory: body; see also SI Figures 16-22).
We compared the performance of the first 11 non-constant functional harmonics in reconstructing task activation maps to that of the control function bases (rotations of functional harmonics, eigenvectors of the adjacency matrix, eigenvectors of the FC, principal components, and independent components; Figure 5a-g). We found that functional harmonics outperform the rotated harmonic basis (pcorr < 0.005, Monte-Carlo tests with 1000 permutations, Bonferroni corrected for multiple comparisons), adjacency eigenvectors (pcorr < 0.005, Monte-Carlo tests with 1000 permutations, Bonferroni corrected for multiple comparisons), as well as PCA (pcorr < 0.005, Monte-Carlo tests with 1000 permutations, Bonferroni corrected for multiple comparisons) and ICA (pcorr < 0.005, Monte-Carlo tests with 1000 permutations, Bonferroni corrected for multiple comparisons), and did not exhibit any significant difference to the performance of the FC eigenvectors (p > 0.15 before correction for multiple comparisons, not significant (n.s.)).
In order to examine the reconstruction performance of each function basis for different task groups, we applied the same Monte-Carlo analysis to each of the 7 task categories separately. We found that reconstruction errors of functional harmonics were significantly lower than those of their rotations for each of the task groups (all pcorr < 0.035, Monte-Carlo tests with 1000 permutations, Bonferroni corrected for multiple comparisons), and significantly lower than those of the adjacency eigenvectors in six out of seven task groups (all pcorr < 0.035, Monte-Carlo tests with 1000 permutation, Bonferroni corrected for multiple comparisons, except language, where p =0.18, before correction for multiple comparisons, n.s.). In comparison to FC eigenvectors, while there was no significant difference in the reconstruction performance when all tasks were pooled, we found that functional harmonics performed significantly better in the reconstruction of motor tasks (pcorr < 0.035, Monte-Carlo tests with 1000 permutations, Bonferroni corrected for multiple comparisons; see inset in Figure 3c). Compared to PCA and ICA, the reconstruction errors of functional harmonics were significantly lower for motor and working memory task groups (all pcorr < 0.035, Monte-Carlo tests with 1000 permutation, Bonferroni corrected for multiple comparisons), while for all other task groups there were no significant differences (all p > 0.01 before correction for multiple comparisons, n.s.). These results indicate that functional harmonics delineate the functional systems involved in working memory and motor tasks more precisely than other function bases used as control. It is important to note that the number of tasks in the remaining categories is smaller (3 tasks per category) than that of the motor and working memory task groups, and more data may be required to achieve significant differences for these categories. In summary, when all individual task groups as well as the overall performance in reconstructing the complete task pool is considered, the functional harmonics outperform all 5 control function bases using only first 11 non-constant components.
Given that functional harmonics constitute functionally relevant communication channels, we hypothesized that the task activation maps can be characterized by their power spectrum. Figure 6a, d and Figure 6b, e, show two examples of task activation maps and the corresponding normalized power of the first 11 non-constant functional harmonics, respectively, revealing how strongly each of the 11 functional harmonics shown in Figure 2 contributes to these particular task maps. For qualitative evaluation, we display the task activation maps reconstructed by superimposing functional harmonics in the order of their contribution strength for varying numbers of functional harmonics in Figure 6c, f (see also SI Figures 16-22). Across all 47 task maps that were evaluated, the functional harmonic which was the strongest contributor was always either the constant functional harmonic or one of the first 11 non-constant harmonics shown in Figure 2.
In order to evaluate the uniqueness of the functional harmonic power spectrum of each task activation map, we computed the distance between a given reconstructed map and all original task maps, resulting in a confusion matrix for each number of harmonics with maximum contribution. If task maps can indeed be characterized by their functional harmonics power spectra, the error should be minimal between a reconstruction and its corresponding task map compared to the error of the reconstruction of the other 46 task maps. The confusion matrices in Figure 6g-i show the pairs of the original and reconstructed task activation maps with the minimum distance when using 1, 4, and 40 functional harmonics with maximum contribution. Coloured squares mark the 7 task groups as in Figure 5. The proportion of unambiguously identified tasks in relation to the number of functional harmonics is shown in Figure 6j. We found that sparse representations using the 4 functional harmonics with the largest power for each task are sufficient to unambiguously characterize the seven task groups with the exception of one working memory task (Figure 6h), and 70% of all individual tasks. When the 40 functional harmonics with maximum contribution are used, which corresponds to 0.1% of the complete spectrum of functional harmonics, 44 out of 47 task maps are correctly identified from their reconstructions (Figure 6i).
Overall, our results demonstrate that that functional harmonics provide a novel functionally relevant representation, where the brain activity accompanying different tasks can be uniquely identified from the activation profiles of a small range of functional harmonics.
Discussion
We reveal a previously unknown principle of cortical organization by applying a fundamental principle ubiquitous in nature - harmonic modes - to the communication structure of the human brain. The resulting modes termed functional harmonics reveal a data-driven, frequency-specific function basis derived from the human resting state functional connectivity matrix and constitute the optimal mapping of the communication structure encoded in this matrix onto the cortex.
We demonstrate the meaning of the first 11 functional harmonics as functional communication channels in the brain. Functional harmonics estimated as the eigenvectors of the graph Laplacian provide an orthogonal function basis that can reconstruct any pattern of cortical activity. Furthermore, harmonic function bases are unique in that its basis functions exhibit an implicit ordering according to their wavelength (spatial frequency) and hence provide not only a multi-dimensional but also a multiscale representation of brain activity. In this work, we show that when this harmonic basis is estimated from the communication structure of the human brain, each basis function, i.e. each functional harmonic, yields a frequency-specific communication channel, where specific brain regions communicate through their correlated activity. Crucially, our findings using the functional harmonic representation suggest that a brain region is able to fulfill a multitude of functions because of its simultaneous membership in several communication channels, which are orthogonal to each other and separated by spatial frequency.
Moreover, functional harmonics unify the competing views that brain activity arises either from smoothly varying gradients or from the modular and specialized regions. Within the functional harmonic framework, specialized regions emerge from the interaction of functional harmonics across multiple dimensions. Hence our findings provide, to our knowledge, the first principle that unifies the gradiental and modular aspects and reveals the multi-dimensional nature of cortical organization.
Furthermore, by definition, functional harmonics are the extension of the well-known Fourier basis to the functional connectivity of the human brain. As such they provide a function basis to reconstruct any pattern of brain activity as superpositions of these harmonic patterns. We explicitly show that functional harmonics are building blocks of cognitive activity in the brain by characterizing a multitude of task activation maps from their functional harmonic reconstructions. In particular, our results demonstrate that although there is a multitude of function bases one can choose to represent patterns of brain activity such as the well-known principal components or independent components of PCA and ICA, functional harmonics stand out in their ability to capture certain aspects of cortical organization: our findings reveal that out of the 5 function bases used to represent patterns of cortical activity; i.e. (i) eigenvectors of the FC matrix, (ii) eigenvectors of the adjacency matrix, (iii) rotated versions of functional harmonics, (iv) PCA, (v) ICA, only the functional harmonics yield both, a delineation of cortical areas and an efficient reconstruction of task activation maps, and thus provide the strongest candidate to be the basis functions of human cognition.
Considering that the principle of harmonic modes when applied to the structural connectivity of the human brain - the human connectome - have been shown to reveal the functional networks31, our results point to the emergence of the same fundamental principle in multiple aspects of human brain function. Beyond the results presented here, functional harmonics suggest novel ways to understand the dynamics of the human brain in health and in pathology as well to explore individual differences within this multi-dimensional harmonic representation.
Online Methods
Data
The data used in this study was acquired and made publicaly available by the Human Connectome Project, WU-Minn Consortium (Principal Investigators: David Van Essen and Kamil Ugurbil; 1U54MH091657) funded by the 16 NIH Institutes and Centers that support the NIH Blueprint for Neuroscience Research; and by the McDonnell Center for Systems Neuroscience at Washington University. All study protocols were approved by the Washington University institutional review board, and informed consent was obtained in all cases40, 41.
In this study, we used the dense functional connectivity (FC) matrix, which is part of the Human Connectome Project’s 900 subjects data release40–47. It is available under db.humanconnect.ome.org/data/projects/HCP_120065. Clicking on “812 Subjects, recon r227, Dense Connectome” will download the appropriate .zip-archive (user login necessary). The list of names of all the files used in this study is shown in Table 1. Note that in this release, many of the subjects are related to at least one other subject of the group. The group average functional connectivity matrix was obtained by correlating group-PCA eigenmaps from 812 out of the 900 subjects included in this release, which are the subjects that having completed all four sessions of 15-minute resting state fMRI.
For task reconstructions, we used data contained in the S1200 group average data release, which is available on www.humanconnectome.org/study/hcp-young-adult/document/extensively-processed-fmri-data-documentation, as “HCP_S1200_GroupAvg_v1 Dataset”.
For the analyses involving retinotopic maps, we used data available on osf.io/bw9ec/ and described in Benson et al. (2018)55. The relevant file is named “prfresults.mat” and contains a variable “allresults” of dimensionality 91282 (grayordinates) ×6 (quantities) ×184 (181 subjects plus 3 different group averages) ×3 (model fits). We used only the quantities ‘ang’ and ‘ecc’, the first model fit, of the group average across all available subjects, which uses all available time points. See osf.io/bw9ec/wiki/home/ for details.
Data are encoded in CIFTI file format40, which means that coordinates are defined on the cortical surface (“grayordinates”), i.e. using n vertices rather than voxels47. The file was read using connectome workbench functions65 and converted to a single precision vector of length (n·n − n)/2 (due to its symmetry) using Matlab66. We also excluded the medial wall. This reduced the size of the FC matrix in memory from 33 GB to approximately 6 GB, greatly easing subsequent computations. The loss in precision is negligible compared to the accuracy with which pairwise correlation can be estimated from noisy fMRI time courses.
For visualization purposes, we used the surfaces provided with the functional data.
Software
All data analysis was performed using MATLAB 2014b or 2017b, using also scripts and functions from the following freely available software packages:
Fieldtrip version 20180903
Connectome workbench (https://www.humanconnectome.org/software/connectome-workbench)
gifti toolbox (https://www.artefact.tk/software/matlab/gifti/)
Background: Functional Harmonics
The approach presented here relies on representing the human brain’s communication structure (dFC) as a graph and estimating the eigenfunctions of graph Laplacian applied to this structure. The graph representation of the brain’s communication structure is created by representing the vertices sampled form the gray matter cortical surface as the nodes with n being the total number of nodes (n = 59.412 in this study) and by representing the connections between the vertices as the edges , which come from the connections in the dFC matrix. We represent this graph structure by its n × n adjacency matrix A = [aij] that is formed by connecting each node i to its k-nearest neighbours (k =300 in this study) according to its correlations in the dFC matrix, i.e.: where κi is the set of the k largest values in row i in the dFC matrix. In order to ensure A is symmetric, we also set aji = 1, if aij = 1. Defining A as such results in a symmetrical sparse binary matrix.
Then we estimate the graph Laplacian defined as where A is the adjacency matrix as defined above, and D is the degree matrix, which is defined as a diagonal matrix with diagonal elements
As such, the degree matrix D contains each node’s degree in its diagonal. Finally, we estimate the functional harmonics as the eigenfunctions Ψ = {ψ1,ψ2,⋯,ψn} by solving: where ψi are the n × 1 eigenvectors and λi are the corresponding eigenvalues.
Control function bases
Spherical rotations: We performed comparisons against spherical rotations of surface maps. We followed48, adapting freely available code (github.com/spin-test/spin-test) to be used with HCP surfaces. In this approach, surface maps are projected to a spherical surface and then rotated by a random angle. Values are then mapped back to the nearest vertex, and the map is symmetrized in order to preserve this property. Parts of the corpus callosum that are rotated to the cortical surface are labelled as missing data (NaNs) and are ignored in any subsequent calculations (e.g. within- and across area distances, see below). Since we used multi-dimensional function based, we rotated the surface maps corresponding to each dimension by the same angle. Note that, however, the resulting rotated function basis is no longer orthonormal due to the symmetry preserving step.
Principal components (PCs): PCA (principal component analysis) is a popular dimensionality reduction technique which preserves the maximum amount of variance in the data. It consists of taking the eigenvectors of the covariance matrix of the time series. These principal components are provided by the HCP via Connectome DB (see Table 1). The first 20 PCs are shown in SI Figure 6.
Eigenvectors of the dense FC: An intuitive basis is to take the eigenvectors of the dense FC without applying a threshold as done for obtaining the adjacency matrix. These eigenvectors have been shown to contain valuable information about dynamical FC67. The first 20 eigenvectors of the dense FC are shown in SI Figure 5.
Eigenvectors of the adjacency matrix: In order to test the effect of thresholding/binarizing on the one hand and the effect of using the graph Laplacian instead of the adjacency matrix itself on the other, we also compared to the eigenvectors of the adjacency matrix, i.e. the dense FC thresholded such that only the 300 nearest neighbors of each vertex are retained and set to 1. The first 20 eigenvectors of the adjacency are shown in SI Figure 4.
Independent components (ICs): A very popular dimensionality reduction technique in resting state fMRI68, independent component analysis is the foremost method for obtaining resting state networks. It consists of analyzing the time series of the data and finding those spatial patterns that are maximally independent. We tested all sets of ICs that are provided by the HCP (see Table 1, and found that the set with the lowest number of components, i.e. n = 15, performs best. Therefore, we restricted our comparisons to this set of ICs. Note that ICs are not orthonormal and thereform do not form a basis in the strictly mathematical sense. The 15 ICs used in our comparisons are shown in SI Figure 7.
Monte Carlo simulations
We used a Monte-Carlo approach for statistical validation.
For the silhouette values, we followed48, where permutations consist of rotated surface maps (see previous section) of the functional harmonics as well as principal components, independent components, eigenvectors of the dense FC, and eigenvectors of the adjacency matrix. Silhouette values were then computed for the original, non-rotated map as well as for n = 220 rotated maps, and p-values were computed based on the number P of rotations that performed better than the original map:
We performed Bonferroni correction by multiplying the resulting p-value by 11, i.e. the number of dimensions that was tested.
We used the same approach for the somatotopy index, but only applied to the functional harmonics and their rotations. Since in this case, we had five somatotopic areas (we averaged over the two hemispheres) and tested three of the 11 functional harmonics (ψ3, ψ7, and ψ11), we required n =300 rotations in order to achieve a significance level of α = 0.05 with 15 comparisons.
We also applied a Monte-Carlo permutation test to the mean reconstruction errors by permuting the labels of the basis 1000 times for each control basis. Here, we pooled the reconstruction errors over the first 11 non-constant components. For the overall reconstruction performance, we also pooled all 47 task maps; for ad-hoc tests of each task category, we pooled only over the tasks in each category.
Silhouette values
To test whether isolines of the functional harmonics follow the boundaries of the parcels as defined in the HCP parcellation3, we compute the silhouette value49 of each functional harmonic as: where Mbetween (i) is the average Euclidean distance between vertices belonging to a parcel i and vertices belonging to all other parcels, while Mwithin (i) is the average distance between vertices within the parcel i. If all vertices belonging to a parcel i have the same value, and at least some vertices outside the parcel i have different values, then Mbetween (i) > 0, Mwithin (i) = 0 and S(i) = 1. By averaging over the silhouette values of all parcels, one obtains a measure of how well the data fit the parcellation. Note that we replaced the somatosensory/motor core areas 1, 2, 3a, 3b, and 4 with the somatotopic sub-areas given by the HCP3 for a more detailed evaluation.
To evaluate the somatotopic organization of the functional harmonics, we use a measure that was similar to the silhouette value, but adapted to measure the separation from the rest of the cortex and from other somatotopic areas. where Mbetween,som is the average Euclidean distance between vertices belonging to a somatotopic area and all other vertices belonging to all other somatotopic areas. The first term of the equation is between 1 and 2 and is close to 2 if both Mbetween,som and Mbetween are equal. Multiplying by Mbetween,som ensures that Ssom is not large if both Mbetween,som and Mbetween are small.
Task maps
We used group-averaged task activation maps provided with the S1200 group average data release of the HCP (see Table 1, www.humanconnectome.org/study/hcp-young-adult/document/extensively-processed-fmri-data-documentation). Here we provide a summary of the tasks that form part of the HCP task battery54. There are 7 groups of tasks: working memory, motor, gambling, language, social, emotional, relational. Subjects performed all tasks in two separate sessions (working memory, gambling, and motor in the first session, language, social cognition, relational processing, and emotion processing in the second).
Working memory
Four different stimulus types were used, presented in separate blocks: pictures of faces, places, tools and body parts. Two different task types were used: a 2-back working memory task, where subjects had to respond if a stimulus matched that two trials back, and a 0-back working memory task, where subjects had to respond whenever a single stimulus returned that was presented at the beginning of the block. This results in a total of 19 different working memory task maps, consisting of 14 activation maps (such as 0-back, 2-back, face, body, etc.) and 5 contrasts (between the two task types, between each stimulus type and the average across all stimuli, etc.).
Motor
Visual cues indicated whether participants should move their left or right fingers, left or right toes, or move their tongue. The goal was to identify the motor areas that correspond to these five body parts. This results in 26 different task maps (7 activation maps for 5 body parts plus visual cue plus average, and 6 contrast maps).
Gambling.(Incentive processing.) Subjects played a game in which they could win or lose money. The game was to guess whether the number on a “mystery card” that could range between 1 and 9 would be less or more than 5. The numbers were given after subjects made their guess and were chosen according to the trial type: “win” - the number would correspond to their guess and they would win 1$; “neutral” - the number would equal 5 and they would neither win nor lose any money; “loss” - the number would not correspond to the guess and participants would lose $0.50. Separate blocks are used in which trialsare either mostly win or mostly lose, resulting in two conditions, punish and reward. This results in 3 different task maps (2 activation maps, i.e. one for each condition, and 1 contrast).
Language
Two different task types were used, “story” and “math”. “Story” consisted of participants listening to 5-9 sentences of a story, and answering a 2-alternative forced choice question thereafter. “Math” required participants to solve simple addition and subtraction problems. The two task types are similar in terms of auditory input and attentional load, but different in terms of semantic and numerosity related processing. As for gambling, the two task types result in 3 task maps (2 activation, 1 contrast).
Social
(Theory of Mind, TOM.) Subjects viewed videos of objects (squares, circles, triangles) that moved around in one of two ways: “Random” - there was no interaction between the objects, or “TOM” - the objects moved as if they were reacting to the other objects’“thoughts and feelings”. They then had to judge whether the objects were interacting or not, or respond with “not sure”. As with gambling and language, the two task types result in 3 task maps (2 activation, 1 contrast).
Emotional
Subjects viewed one of two types of stimuli, “faces” or “shapes”, and had to decide which of two stimuli presented at the bottom of the screen matched the stimulus at the top of the screen. The faces included emotional stimuli, i.e. angry or fearful expressions. Again, the two task types result in 3 task maps (2 activation, 1 contrast).
Relational
There were two conditions, “match” and “relational”. In all cases, stimuli can have one of six shapes combined with one of six textures. In the “match” condition, which served as a control condition, two shapes were presented at the top and one at the bottom of the screen. A word (“shape” or “texture”) that appears in the middle of the screen instructs subjects to decide whether the bottom stimulus matches either of the top stimuli in the dimension indicated by the word. In the “relational” condition, two stimuli are presented each at the top and at the bottom of the screen, with no word in the middle. Instead, participants have to determine themselves across which dimension the top pair differs, and, subsequently, indicate whether the bottom pair differs over the same dimension. Again, the two task types result in 3 task maps (2 activation, 1 contrast).
Task maps were computed using FSL’s FEAT and FLAME69, 70 and conducting a between-subject (“level 2”) analysis, resulting in effect sizes (Cohen’s d). We used the task maps with minimal smoothing (2mm total smoothing); see 1200 subjects data release reference manual, pp. 45-54 and 100-104.
Reconstructing the task maps from functional harmonics
The spatial pattern of each task map on on the cortex s(v) was decomposed into and reconstructed from the functional harmonics as: where the coefficient αk of each functional harmonic ψk was estimated by projecting the task map ŝ(v) onto that particular harmonic ψk. As such αk are estimated as:
Then, each task map is reconstructed using Eq.8. In this study, we limit our reconstructions to using a maximum of 100 non-constant functional harmonics (n = 101).
For a reconstruction s*(m), where m indicates a binary vector of dimensionality 101 × 1 which contains ones for harmonic basis functions that are used in the reconstruction and zeros otherwise, we then compute the reconstruction error as:
We also computed the Pearson correlations between s and s*(m). For comparing the correlations between task maps and reconstructions obtained from real functional harmonics versus randomized connectivity harmonics, we considered the number of comparisons to be nC = nTasks · nLevels, where the number of tasks equals 47 and the number of levels refers to the different numbers of harmonics used in the reconstructions, i.e. 0, 1, 2, 3, …, 20, 30, 40, …, 100, in 29 levels. From this we obtained a corrected alpha level of αcorr = 0.05/nC, and we computed the critical value as Fisher’s z-transform of the correlation which a sample has to exceed in order to be significantly higher than the random correlation:
We obtain zcrit = 0.44, which corresponds to a minimum required empirical correlation of 0.41, with N1 = N2 = 59.412 (the number of vertices that contribute to the correlation values), zα = 0.438 (the inverse Student’s t distribution with N1 = N2 = 59.412 degrees of freedom evaluated at 1 − αcorr), and zrand = atanh(0.05) (Fisher’s z-transform of the maximal random correlation between any reconstruction - with any number of functional harmonics - and any task).
Visualization
Somatotopic areas
In the visual and somatosensory/motor cortices, functional harmonics are rather determined by retinotopy and somatotopy than by anatomical or microstructural features. For the former, somatotopic areas occupy exactly the same surface area as the sensorimotor core areas, 1, 2, 3a, 3b, and 4. We therefore replaced, where appropriate, the borders of the HCP parcellation by the borders of the five somatotopic regions.
Parcel borders for visualization
In order to discuss the meaning of the functional harmonics, we show borders of certain parcels on the cortical surfaces (Figure 2). We used three different methods to select which borders to show. First, for some functional harmonics, it was feasible to select these areas manually (for example, early visual areas in functional harmonic 4, somatotopic areas in functional harmonics 3 and 4). The anatomical supplementary information from Glasser et al. (2016)3 uses a functional grouping of many regions that we often used as a guideline, for instance to distinguish between early and association auditory cortex. Second, for some functional harmonics (for instance, functional harmonics 1 and 2), we show the borders of parcels that belong to resting state networks as defined by Yeo et al. (2011)50. The 7-network parcellation is provided by the HCP, which does not perfectly overlap with the HCP parcellation. We adjusted the network borders slightly to align the network borders to follow those of the parcels defined in HCP. Thereby we assigned each parcel to the RSN with which it had the most overlap. Third, some functional harmonics are too complex to manually select areas or networks (namely, functional harmonics 5, 6, 8, and 10). Here we employed simple k-means clustering on the functional harmonic, using k=2 (functional harmonics 5, 6, and 8) or k=3 (functional harmonic 10). To obtain meaningful clusters in the somatosensory/motor cortex, we again replaced the sensorimotor core regions 1, 2, 3a, 3b and 4 with the somatotopic areas. For this purpose, we used vertices within the core regions and re-assigned them to the somatotopic areas based on their distances to the sub-area borders.
Author contributions statement
S. A. and K. G. designed the methodology and the analysis. J. P. and G. D. contributed to the design of the study. M. L. K. contributed to design of the methodology and the statistical analysis. M. L. K. and P. H. aided in the interpretation of the results. S. A., K. G. and M. L. K. wrote the manuscript. All authors reviewed the manuscript.
Data availability
All data generated in this study are available from the corresponding author upon reasonable request.
Code availability
All custom scripts used in this study are available from the corresponding author upon reasonable request.
Additional information
Competing financial interests
The authors declare no competing financial interests.
Footnotes
We added comparison of functional harmonics to alternative function bases, including principal and independent components. We also updated the null model to use rotated versions of the functional harmonics surface maps.