Abstract
Understanding processes fundamental to brain functioning will require knowledge of how functionally-relevant localized brain processes are generated. We identified such functionally-relevant brain activations using fMRI data from N=352 human participants, focusing on select visual cortex regions given long-standing observations that they exhibit localized visual category selectivity. First, we systematically tested the hypothesis that local visual category selectivity can be generated by highly distributed, resting-state network interactions. This was accomplished using a recently developed distributed network interaction approach for mapping task-evoked brain activations by explicitly modeling the influence of activity flowing over resting-state brain connections. Next, we tested refinements to our hypothesis based on category selectivity being generated by stimulus-driven network interactions initialized in V1. We found evidence in support of the refined hypothesis that V1-initialized activity flow processes that were further shaped by fully distributed (i.e., whole-cortex), intrinsic network interactions were sufficient to generate visual category selectivity (but not when further shaped by later visual network interactions alone). Further, using null network architectures we found that each region’s unique resting-state “connectivity fingerprint” was key to category selectivity being sufficiently generated by distributed network interactions. These results generalized across regions associated with four visual categories (processing images of bodies, faces, places, and tools), and provides evidence that intrinsic network organization plays a prominent role in the generation of local, functionally-relevant responses.
Introduction
Fundamental progress in cognitive neuroscience will require understanding how locally exhibited, functionally-relevant activations are established in the distributed, networked brain systems within which they are embedded. Historically, competing frameworks have posited that neural operations are determined by a functional organization along a theoretical, local-to-distributed continuum. At one end, specialized local computations organize neurocognitive functioning (Kanwisher 2010; Schalk et al. 2017), and at the other end, functional organization is based upon distributed patterns of activity (Op de Beeck et al. 2008; Mur et al. 2009). Most modern neuroscience assumes (explicitly or implicitly) theory somewhere between these two extremes (Borra et al. 2010), but the extent to which locally exhibited brain activations are generated by distributed processes remains unclear. This outstanding gap in knowledge is fundamental to comprehensively understanding both local and distributed neurocognitive processes, which are co-exhibited by the brain.
There is extensive evidence for cortical regions locally exhibiting responses that are highly selective for visual image categories such as bodies (Downing et al. 2001; Taylor et al. 2007), faces (Kanwisher and Yovel 2006), places (Park and Chun 2009), and tools (Peelen and Downing 2005b). For instance, high selectivity for face images in the fusiform face area (FFA) has been validated with human neuroimaging (Kanwisher 2010), neural recordings in non-human primates (Tsao et al. 2008), human lesion studies (Corrow et al. 2016), and brain stimulation in humans (Schalk et al. 2017). In parallel, an influential study demonstrated that visual categories could be decoded from distributed yet overlapping activity patterns in ventral temporal cortex (VTC) even after highly category selective regions (e.g., FFA) were excluded from the analysis (Haxby et al. 2001). Contemporaneous work proposed that a cortical region’s functioning is largely determined by its connectivity fingerprint (not to be confused with individual difference “connectome fingerprinting”; Finn et al. 2015), or, a region’s pattern of cortico-cortical connections that specify what information that region receives and conveys (Passingham et al. 2002). Thus, connectivity fingerprints may not be just a byproduct of a region’s function, but a key determinant (Mars et al. 2018). Over recent decades, the functional relevance of brain network interactions has been borne out by a growing body of network neuroscience (Sporns 2014) studies assessing structural, functional, and effective connections (Friston 2011; Reid et al. 2019) during rest and task (Cole et al. 2021). Work by Osher et al. (2016) found that structural connectivity fingerprints can predict visual category responses in select VTC regions. However, because statistical associations were used to make those predictions, it remains unknown why connectivity is predictive of visual category selectivity. Moreover, it is unclear whether an intrinsic (i.e., resting-state, Biswal et al. 1995; Fox and Raichle 2007; Power et al. 2011; Ji et al. 2019) functional network architecture maintains the interactions required to support such a mechanism. Based on our recent work demonstrating the importance of activity flow processes (i.e., network interactions shaped by connectivity fingerprints) in generating neurocognitive processes (Cole et al. 2016; Cocuzza et al. 2022), we hypothesized that visual category selectivity exhibited locally by visual cortex regions is sufficiently determined by activity flowing over each region’s connectivity fingerprint. By testing various models (Fig. 1) of such generative network interactions (i.e., extending Betzel and Bassett, 2017 to empirically-based parameters), the present study provides evidence that activity flowing over a distributed, resting-state network architecture is key to accounting for why connectivity can predict local activity.
In all panels, a toy model of face selectivity in the fusiform face area and posterior superior temporal sulcus (FFA/pSTS, Methods) is depicted as a local outcome of network interactions, or, activity flow processes over an intrinsic network architecture. Circles: regions in a network (color legend: bottom, Ji et al. 2019). Gray arrows: activity flow processes (Methods), which are weighted (connectivity fingerprints), but shown as uniform for visualization purposes. (A) This predicts that all cortical network interactions (“fully distributed”) converge to generate local activation patterns required to locally compute face selectivity. We tested this by applying activity flow mapping from Cole et al. 2016 to select held-out “target” regions (including FFA/pSTS). (B) This predicts that only stimulus-driven interactions with V1 generate face selectivity. We tested this by adapting activity flow mapping to use vertex-level data from V1. (C) Extending panel B: stimulus-driven interactions are further shaped by activity flow processes in the visual system(s). We tested this via a multi-step procedure using mapped activations from panel B, weighted by their connectivity patterns (with the held-out region) in a visual-system-only network, to generate selectivity. (D) This predicts that stimulus-driven interactions, further shaped by fully distributed interactions (i.e., beyond just the visual system in panel C) generates selectivity. Using mappings initialized in V1 (as in B-C), we assessed (here, at the region level) how well selectivity was generated when the fully distributed set of network interactions was initialized by stimulus-driven activity flow processes. In the present study, we tested these predictions using a mapping approach that explicitly models the influence of activity flowing over brain connections.
Putting this hypothesis into a theoretical perspective that concurrently takes into account studies reporting the functional importance of local processes and those of distributed processes, we propose that distributed activity, selected by a given region’s connectivity fingerprint, converges on that region, conferring it with locally specialized functioning (e.g., category selectivity). One likely purpose of this localized convergence (e.g., in the FFA) of network interactions is to represent related information (e.g., faces) distributed throughout the brain in a common space for efficient competition between alternative activity patterns. In this account, FFA may facilitate face recognition via implementing a winner-take-all computation (via local inhibition; Pajani et al. 2017) between alternate faces represented in FFA, which is consistent with many computational models of local neural functions (O’Reilly 2001). Importantly, rather than these local (e.g., recurrent activity or competition due to local inhibition) processes, we propose that the selective activity observed in functionally specialized regions is generated primarily by distributed activity flowing over each region’s connectivity fingerprint. Importantly, our hypotheses suggest that connectivity patterns play an active, generative role in localized region-level activity, and predict that estimating activity flow over a region’s unique connectivity fingerprint would account for the majority of that region’s localized task-evoked selectivity.
Importantly, evidence that distributed activity flow processes provide the dominant influence on visual-category-selective responses does not discount the importance of local computations, but rather provides the groundwork for understanding how network interactions can sufficiently generate such local computations. Activity flow mapping (Cole et al. 2016) is well-suited to test these hypotheses because all parameters are explicitly based on empirical quantities, rather than abstractions (Kriegeskorte and Douglas 2019; Ito et al. 2020c). The parameter of brain connectivity (i.e., a region’s unique connectivity fingerprint) is key to characterizing the network-based processes that converge to generate local responses. Moreover, we explicitly tested whether activity flow mapping based on intrinsic connectivity fingerprints could generate local visual category selectivity, which would provide novel evidence for the functional relevance of the resting-state network architecture within the context of canonically localized visual responses.
We tested our hypotheses (Fig. 1) — an initial step toward understanding how local activations emerge from distributed network interactions in the scope of visual category selectivity — using human functional magnetic resonance imaging (fMRI) resting- and task-state data from the Human Connectome Project (HCP) (Van Essen et al. 2013) young adult release. We focused on category-specific representations of bodies, faces, places, and tools embedded in a working memory task. These visual categories correspond to functionally specialized visual cortex regions that are highly replicated in the literature, such as the fusiform face area (FFA) and the parahippocampal place area (PPA). In sets (or complexes) of functional ROIs in the visual cortex, we quantified actual and activity-flow-mapped visual category selectivity and estimated the degree to which fully distributed (i.e., whole-cortex) network interactions (Fig. 1A) contributed to this selectivity. We probed the differential contributions of large-scale functional networks to activity flow processes, and used substitution null models to corroborate that a given ROI’s connectivity fingerprint significantly shaped its visual category selectivity. Given evidence that activity in V1 represents a retinotopic map of visual features (Carandini et al. 2005; Wandell and Winawer 2011), we also tested refined hypotheses (Fig. 1B-D) that specify stimulus-driven activity flow processes as initialized in V1. In adapting the activity flow mapping procedure to address stimulus-driven hypotheses, we were also able to reduce the theoretical impact of causal circularity by constraining empirical parameters to network interactions of V1 only. Systematically testing how well visual category selectivity can be generated via activity flow mapping constitutes an important first step in refining a network-interaction-based framework for how functionally selective responses exhibited by local regions of cortex emerge within a distributed, functionally-organized network architecture.
Materials and methods
Data acquisition and participants
Data used in the present study were collected by the Washington University-Minnesota Consortium of the Human Connectome Project (HCP) (Van Essen et al. 2013) as part of the young adult (S1200) release. This included resting- and task-state functional neuroimaging data (see following Methods sections for more details on each). We obtained the minimally preprocessed (Glasser et al. 2013) neuroimaging data for N=352 healthy young adults. Following the pipeline outlined in Ito et al. (2020a), this N=352 subset of the HCP participants were selected to ensure high quality scan data and no familial relations. All study procedures were approved by the Institutional Review Board of Rutgers University-Newark. Further details on participant recruitment can be found in Van Essen et al. (2013).
We used a split-sample validation approach to minimize false discovery rate (Anderson and Magruder 2017). The cohort of HCP participants in the present study (N=352) were randomly allocated to either a discovery (n=176) or replication (n=176) dataset. Each dataset was analyzed identically but independently. Participants in the discovery dataset (77 men, 99 women) had the following age ranges: 22-25 years (26.14%), 26-30 years (41.48%), 31-35 years (30.68%), and 36+ years (1.7%). Participants in the replication dataset (93 men, 83 women) had the following age ranges: 22-25 years (22.16%), 26-30 years (43.18%), 31-35 years (34.09%), and 36+ years (0.57%). Results presented in figures refer to the discovery dataset, with replication dataset results reported in-text.
Task paradigm
In a subset of analyses, we used data corresponding to seven HCP tasks and 24 task conditions that sampled diverse cognitive domains: emotion, gambling reward, language, motor, relational reasoning, social cognition, and working memory. In all other analyses we focused on the working memory/category-specific representation n-back task (see Barch et al. 2013 for more details on all tasks). Prior studies suggest that the n-back task can be used to test hypotheses regarding the function(s) of specific brain areas (Drobyshevsky et al. 2006). Further, manipulating how far back a participant must remember (0-back versus 2-back) allows for the assessment of working memory maintenance (along with other contrasts, such as stimulus type). While working memory was not the main interest of the present study, specifically manipulating working memory has the added benefit of promoting task engagement.
The cognitive domain of interest in the present study was the processing of visual categories (Haxby et al. 2001; Wierenga et al. 2009; Op de Beeck et al. 2019). Category-specific representations were embedded in the n-back task via blocks of trials that presented images of four reliably studied and distinct visual categories (Peelen and Downing 2005b; Osher et al. 2016): (1) bodies (Downing et al. 2001; Taylor et al. 2007); (2) faces (Kanwisher et al. 1997; Kanwisher and Yovel 2006; Iidaka 2014); (3) places (Park and Chun 2009; Weiner et al. 2018); and (4) tools (Grill-Spector et al. 2001; Wierenga et al. 2009; Matić et al. 2020).
The specifics of the n-back task are detailed in Barch et al. (2013). Briefly, the n-back task included 2 runs with blocks of visual category trials (8 blocks, 10 trials each, 2.5 seconds each) and fixation blocks (4 total, 15 seconds each). Half of the blocks were 0-back and half were 2-back. During 0-back working memory tasks, participants determined via button-press whether presented stimuli (images of either bodies, faces, places, or tools) matched a constant target shown in the initial cue screen. During 2-back working memory tasks, participants were asked to indicate if the present stimulus matched a target from two screens back. Given that working memory load was not the main interest of the present study, results presented here reflect an average across 0-back and 2-back conditions (i.e., fMRI data was averaged after all preprocessing steps), except for in analyses examining “response profiles” (i.e., all 24 HCP conditions were assessed separately).
MRI parameters
All MRI data were collected at Washington University in St. Louis, Missouri, USA in a customized Siemens 3T “Connectome Skyra” scanner. Whole-brain, multiband, and echo-planar images (EPI) were acquired with a 32-channel head coil. The repetition time (TR) was 720 ms; the echo time (TE) was 33.1 ms; the flip angle was 52 degrees; the bandwidth (BW) was 2290 Hz/Px; in-plane field-of-view (FoV) was 208 x 180 mm; 72 slices; 2.0 mm isotropic voxels; and the multiband acceleration factor was 8. Whole-brain and high-resolution T1-weighted and T2-weighted anatomical scans were also acquired, with an isotropic voxel resolution of 0.7 mm. All imaging data were collected over two days for each participant. Each day included two resting-state fMRI scans, each lasting 14.4 minutes (for a total of approximately 29 minutes of rest data per day). During rest, participants were instructed to keep their eyes open and fixated. Task fMRI (30 minutes of task data per day) followed resting-state fMRI scans on each day. Each of the 7 tasks were performed over two consecutive runs. Further details on MRI protocols and parameters can be found in Van Essen et al. (2013).
fMRI preprocessing
We acquired minimally preprocessed data (Glasser et al. 2013) from the HCP database (www.humanconnectome.org). In-brief, this included: anatomic reconstruction and segmentation; motion correction; intensity normalization; and EPI reconstruction to a standard (surface-based) template; MSM-All registration (multimodal surface matching registration based on areal features from multiple sources, including: cortical folding/sulcal depth, myelin maps, resting-state networks, and visuotopic maps; Glasser et al. 2016). The resulting data were in CIFTI grayordinate (i.e., vertex) space. We performed additional preprocessing steps on cortical vertices (59,412 vertices across the two hemispheres). On both resting- and task-state data, we removed the first five frames of each fMRI run, demeaned and detrended the timeseries, and performed nuisance regression. We followed a variant on methods suggested by Ciric et al. (2017) to model and regress out nuisance parameters, including motion and physiological noise. We performed nuisance regression to remove six motion parameters, their derivatives, and quadratics (24 total). Anatomical CompCor (aCompCor; Behzadi et al. 2007) was used on white matter and ventricle timeseries to model physiological noise. The first five principal components from white matter and ventricles were independently extracted. These parameters, their derivatives, and the quadratics of all regressors were removed (40 total). Altogether, 64 nuisance parameters were modeled and associated variance removed from the data.
Global signal was not removed given evidence that it can artificially introduce negative relationships (Murphy et al. 2009; Power et al. 2014) and is non-standard for task activation analyses. However, the nuisance parameters modeled and removed with aCompCor contain components similar to the proposed global signal, with the added benefit of not removing gray matter signals (Power et al. 2018; Ito et al. 2020a).
Task-state activity estimation
Cortical activations related to task-state fMRI conditions (n-back visual categories) were estimated by a standard general linear model (GLM), which convolved task timing (per block-design condition) with the SPM canonical hemodynamic response function (Friston et al. 1994). This was performed per cortical vertex, each belonging to a region of the Glasser et al. (2016) multimodal parcellation (MMP) atlas. Average coefficients of the GLMs were utilized for each MMP region or functional complex (described further below, Identification of functional complexes) as estimates of visual category evoked brain activity.
Identification of functional complexes
To address the question of whether distributed network interactions can map (Cole et al. 2016; Cole et al. 2021) local, category-selective functional responses, we focused on the functional brain regions identified in Osher et al. (2016) given the similarity of the four visual categories in that study to the HCP n-back task studied herein. Additionally, as suggested by Poldrack et al. (2019), variables that will be used in predictive analyses should not be chosen in an empirically-driven manner (e.g., from post hoc cross validation and/or optimization procedures) because this has the potential to artificially inflate accuracy due to circularity. Thus, selecting functional regions of interest a priori has the benefit of controlling false positives in accuracy, particularly with our focus on the well-established visual category literature. Upon further review of the literature as well as the regions belonging to the Glasser et al. (2016) MMP atlas, we observed that these functional regions: (1) contained multiple MMP regions (e.g., the parahippocampal place area or PPA, which contained 6 MMP regions as in Weiner et al. 2018); (2) did not have an exact terminological match in the MMP atlas; and/or (3) included two discrete locations (e.g., the retrosplenial complex, or RSC, often studied in conjunction with the PPA). To address this we first performed a literature search of each of the Osher et al. (2016) functional regions with Google scholar (https://scholar.google.com/) and Neurosynth (https://neurosynth.org/), filtering for relevance, year, and citation count (when possible) (see Table 1 references). We focused further on publications that had corresponding volumetric coordinates (or MMP surface region labels when possible), and generated a list of Montreal Neurological Institute (MNI) x/y/z volumetric coordinates for each functional region (with a minimum of 10 entries per region). We then used the average x/y/z coordinate set as a reference to identify corresponding MMP regions (Table 2).
(Column 1) Visual categories grouping remaining rows into four categories: bodies, faces, places, and tools. (Column 2) The MMP (Glasser et al., 2016) region acronym and corresponding indices (numbers in the MMP atlas) for left and right (L/R) hemispheres. Sup.: superior; post.: posterior; lat.: lateral; med.: medial. MST: med. sup. temporal area; PH: basal parietal area (sup.); V4t: visual area V4t; FST: fundus of the sup. temporal area; TE2p: temporal area 2 (post.); FFC: fusiform face complex (MMP phrasing); STSdp: post. sup. temporal sulcus (dorsal); STSvp: post. sup. temporal sulcus (ventral); PHA1-3: parahippocampal areas 1-3; VMV1-3: ventromedial visual areas 1-3; POS1: parietooccipital sulcus area 1; V4: fourth visual area; V8: eighth visual area; LO1: lat. occipital area 1; LO2: lat. occipital area 2; PIT: post. inferotemporal; V3CD: area V3CD. (Column 3) CAB-NP (Ji et al., 2019) network affiliation (see Results Figs. 3-4) (Column 4) Functional complex each region belongs to, based on prior literature. EBA = extrastriate body area; FBA = fusiform body area; FFA = fusiform face area (we used “FFA” versus “FFC” for consistency with the broader literature); pSTS = posterior superior temporal sulcus; PPA = parahippocampal place area; RSC = retrosplenial complex; LOC = lateral occipital complex. (Column 5) References used to identify volumetric coordinates (Table 2) corresponding to each functional complex and/or references to corroborate MMP regions (identified with asterisks) given their reference in Glasser et al., 2016.
(Column 1) The functional complexes assessed in the present study. Naming conventions are the same as in Table 1. (Columns 2-4) Stereotactic (x/y/z) MNI coordinates for the left hemisphere; mean and standard deviation (SD) of at least 10 publications per complex (see Table 1 for full list of references). Publications that listed coordinates in Talairach space were converted to MNI space following Lacadie et al., (2008). Note that some publications reported multiple functional localizer experiments, which were treated as separate entries in obtaining average coordinates. (Columns 5-7) The same as in columns 2-4, but for the right hemisphere. These coordinate sets were used as references to locate corresponding MMP (Glasser et al., 2016) regions, yielding surface based functional complexes.
To convert from volumetric space to surface space, we employed the following steps (using Connectome workbench, FreeSurfer, and AFNI/SUMA commands, per hemisphere): (1) convert the MMP atlas to FreeSurfer’s fsaverage space; (2) convert fsaverage to the surface-mesh MNI-152 template (which is also included in AFNI); (3) convert surface MNI-152 template to volumetric MNI-template and obtain coordinates. Additional commands were used to ensure that the templates aligned as best as possible, including zero-padding, thicker ribbons (that were later filled), and adding MMP sphere labels. To obtain x/y/z coordinates, we divided the template into equal portions perpendicular to the long axis of a given sphere label, identified the middle division, then calculated the centroid. We then found the MMP regions that were physically closest to the reference coordinate set for each functional region. We then referred to the Glasser et al. (2016) supplemental results (as well as publications they based the MMP atlas upon; see references with asterisks in Table 1) to exclude regions that were physically close but functionally irrelevant (e.g., auditory regions near the pSTS, which was a potential ROI for face processing).
The final set of regions, including their proposed category selectivity, network affiliations, and references are listed in Table 1. The term “functional complex” indicates firstly that more than one MMP region comprised each complex, and secondly serves to distinguish an MMP atlas region from a category-selective (based on prior literature) complex in the present study. Further, in some cases there were two functionally relevant complexes in the literature (e.g., PPA and RSC, both relevant to place processing, as in Osher et al. 2016), therefore both were included. When appropriate, we applied the Cole-Anticevic brain-wide network partition (CAB-NP, Ji et al. 2019, see main text Fig. 3C), which was based on HCP resting-state fMRI data and was generated from a Louvain community detection algorithm that assigned MMP regions to 12 functional networks (labeled via distinct colors in see main text Fig. 3C). Table 1 lists the network assignments of each region, which is further reflected in Results figures by color labeling matching main-text Fig. 3C. The reference sets of volumetric coordinates (x/y/z) are listed in Table 2.
(A) Activity flow mapping toy diagram and formula. Task activity for held-out region j (purple) is mapped as the sum of task activity of all other cortical regions, i (coral) (n = total number of regions), weighted by their connectivity estimates with j (gray). (B) Activity flow simulation results (reproduced with permission from Cole et al. 2016) demonstrating that fully-distributed activity flow mapping (Fig. 1A) is most successful when distributed processing mechanisms are high and local processing mechanisms are low. (C) Example of activity flow mapping (the standard, fully-distributed application) with empirical data (steps 1-6) (reproduced with permission from Cocuzza et al. 2022). For a given target region j, estimates of intrinsic (e.g., resting-state) connectivity between j and all other source regions (step 1) are multiplied by all other regions’ actual task activations (step 2). This yields an activity flows map quantifying the contribution of all other regions’ activity flow upon the held-out region, j, for the task of interest (step 3). These are summed to equal the mapped task activity of j (step 4). This procedure is iterated over all regions to generate activity-flow-mapped task activations across the whole cortex (step 5). This is compared with the actual whole-cortex map of task activations via Pearson’s r, MAE, and R2 to estimate accuracy (step 6). Importantly, this approach is flexible to different estimates of connectivity. Source vertices not included in the analyses (10 mm from the target region j; Methods) are depicted in green.
(A) Causally confounding graph patterns in standard FC estimation methods (adapted with permission from Sanchez-Romero and Cole 2021a). CombinedFC incorporates both bivariate and multiple regression measures such that confounders, chains, and colliders are better accounted for than in each approach alone. (B) The cross-participant (n=176) average resting-state connectivity matrix (estimated via combinedFC) of 360 MMP regions (Glasser et al. 2016), ordered along each axis per the Cole Anticevic brain-wide network partition (CAB-NP; Ji et al. 2019; color-coded on each axis to match panel C). This was the functional network organization utilized in the present study for activity flow mapping. (C) Cortical schematic of the CAB-NP and its 12 functional networks from Ji et al. 2019, reproduced with permission. (D) Response profiles (across all 24 HCP conditions) of four complexes of interest to the present study (indicated along the y-axis; note that “r” stands for right hemisphere); mapped versus actual (left and right respectively; mean across participants depicted in each panel). Black boxes highlight the n-back conditions that maintained visual category embeddings germane to a given functional complex (e.g., 0-back bodies and 2-back bodies for the right EBA and right FBA). Activity-flow-mapped response profiles were highly accurate, suggesting that mapped activation patterns of the functional complexes of interest were reliable across multiple cognitive domains.
Resting-state functional connectivity estimation
Functional connectivity (FC) was modeled as a series of target dependent variables (i.e., of each region), each with multiple source independent variables (i.e., other regions) (consistent with terminology in the following section: Activity flow mapping and corresponding Fig. 2C, step 1). As in Cole et al. (2016), source vertices excluded all vertices within a 10 mm dilated mask of the target region (as well as excluding vertices of the target region itself). This reduced potential circularity related to spatial autocorrelation by avoiding nearby vertices in predicting the activity of the target. Modified source sets of vertex-level timeseries were then averaged together based on their Glasser et al. (2016) MMP region assignment. FC was then estimated with a method proposed by Sanchez-Romero and Cole (2021a) termed combinedFC. CombinedFC integrates the standard methods of bivariate correlation, partial correlation, and multiple regression together to account for causal confounds better than each method alone (as in main-text Results Fig. 3A). Briefly, each participant’s resting-state data (pre-processed) was firstly assessed with partial correlation (amongst each pair of regions, conditioning on all other regions) to find conditional associations, which were examined for statistical significance. This step addressed potential confounders and causal chains (Spirtes et al. 2000), and built an initial connectivity architecture by adding in significantly correlated edges only. Then, bivariate correlation estimates (i.e., Pearson correlation) for each pair of connected regions (from the initial partial correlation step) were used to remove spurious edges resulting from conditioning on colliders (specifically, edges where r was statistically equal to zero). Lastly, a multiple regression procedure was performed to scale the estimated edge weights. For each target region functioning as a response variable, all other connected source regions functioned as regressors to estimate the final combinedFC weight. See Sanchez-Romero and Cole (2021a) for further detail.
Activity flow mapping
Activity flow mapping is an approach that allows us to test whether task-evoked activations in held out brain regions can be generated based upon the inherent link between activation patterns and functional connectivity (Norman-Haignere et al. 2012; Hermundstad et al. 2013; Cole et al. 2014a; Ito et al. 2020c) (i.e., an empirically-parameterized extension of generative network models outlined in Betzel and Bassett 2017). Activity flow was developed by Cole et al. (2016) (see Cocuzza et al. 2022 for further details), and is described by Formula 1 (Fig. 2):
Where j is a held out brain region, and i indexes all other regions in a given parcellation. Note that the standard application of activity flow mapping given by Formula 2 is consistent with the “fully distributed” mapping of Fig. 1A (i.e., mapping that is used to generate visual category selectivity, see Category selectivity below) because source regions (i) refer to all other cortical regions. Refinements to this are outlined in later Methods sections (see Stimulus-driven visual category selectivity and Category selectivity from stimulus-driven network interactions further shaped by fully distributed network interactions). This method uses empirical parameters to test the extent that brain activations are propagated and shaped via distributed network interactions. Applied to research questions on functional processing, activity flow processes generate task-evoked (i.e., functionally relevant) activity in held-out target regions by summing the task-evoked activity in all other source regions scaled by FC estimates between the target region and corresponding source regions (Formula 1). Using a simulated model of neural processing, Cole et al. (2016) found that activity flow processes were most accurate when a global coupling parameter was high and the strength of recurrent (local) connections was low (Fig. 2B). Critically, this suggests that the successful application of fully-distributed activity flow mapping (Fig. 1A) to empirical brain data reflects similar high global coupling properties being preserved in real (rather than simulated) data. Thus, the success of activity flow mapping can be taken as a proxy for how much signals over distributed network interaction contribute to a given activation (see Category selectivity for more details).
To map category-selective responses to bodies, faces, places, and tools we applied the activity flow approach to each of these categories independently. Actual task activations used were beta coefficients estimated by task GLMs (described above in Task-state activity estimation), with a focus on the MMP regions comprising functional complexes as the held-out target regions (described above in Identification of functional complexes). Functional connectivity estimates between targets (functional complex regions) and sources (all other connected regions) were based upon resting-state timeseries and estimated via combinedFC (described above in Resting-state functional connectivity). To assess accuracy, activity-flow-mapped category responses were compared to actual (i.e., empirical) category responses via Pearson r correlation coefficient (NumPy corrcoef function in Python), mean absolute error (MAE), and the coefficient of determination (R2) (sklearn r2_score function in Python).
Each of these indices quantified a different aspect of accuracy, altogether providing a thorough evaluation of activity flow mapping. In brief, Pearson’s r estimated the linear correlation between actual and mapped brain activations, with a potential range from -1 to 1 and the property of scale invariance. MAE measured the arithmetic mean of absolute errors between actual and mapped brain activations, with a sensitivity to scale. The coefficient of determination (R2) estimated the percent of unscaled variance in the actual brain activations that was accounted for in the mapped activations. Note that R2 maintains a range of negative infinity to 1; from 0 to 1 R2 can be interpreted as 0 to 100% variance, but negative values indicate incorrectly scaled predictions. Thus, each index had clear advantages and disadvantages, but altogether provided a complementary, full characterization of mapping accuracy. In all cases, these comparison statistics were taken per participant, then averaged across participants to provide a random effects estimate.
Category selectivity
To assess how selective each functional complex of interest was for a given visual category, we measured the ratio of category activation amplitude divided by non-category activation amplitude, for both activity-flow-mapped data and actual data, per participant. For example, for the functional complex of the extrastriate body area (EBA) and fusiform body area (FBA) (see above, Identification of functional complexes), we quantified body activations divided by non-body activations. Per participant and functional complex, we used Formula 2:
We will henceforth refer to the ratio in Formula 2 as category selectivity, which can be interpreted as how many times larger category responses were than non-category responses. For example, if a given participant’s body-category selectivity score (for the EBA and FBA complex) was 1.5, then their activations to body images were 1.5 times larger than their activations to non-body images. A category selectivity score of 1.0 reflected the hypothesized null mean given that it quantifies category activations as equal to non-category activations. The main advantage of utilizing a ratio versus a difference score is that it is less sensitive to beta value scale differences across brain regions.
Prior to calculating the category selectivity score (Formula 2) it was necessary to normalize the range of activation values. This was necessary for two reasons: (1) avoiding negative numbers (which would limit interpretability of the selectivity values) and (2) rescaling activation values such that every participant’s data was in the same range (which allows for group level comparisons). We used a standard normalization procedure termed min-max normalization (also termed feature scaling). We used Formula 3:
Where a and b are the lower and upper bounds of the rescaled data, respectively. We used a=0 and b=1.0, which were selected to increase interpretability of the category selectivity ratio since all ratios were positive and included a possible minimum of 0. The minimum and maximum values in Formula 3 were calculated based on the actual activations of a given region set (e.g., EBA and FBA complex for body selectivity) across all 24 task conditions (x) for each participant separately. Note that results were highly similar if minimum and maximum values were calculated separately for mapped and actual activations. We additionally identified and removed outlier participants (in category selectivity) following the methods described by Leys et al. (2013) (median absolute deviation), using a highly conservative threshold of -/+ 5 deviations relative to the median. Given this high threshold, outlier removal excluded a small number of participants with category selectivity scores that were not representative of the group average, which is how category selectivity was reported (which had a large sample size of n=176 per discovery and replication datasets). Note that statistical significance (see Experimental design and statistical analysis) was not impacted by removal of outlier participants.
In assessments of the fully-distributed network interaction scheme (Fig. 1A), we estimated the relative contribution of distributed network interactions to a given category-selective response by computing the percent of actual category selectivity (distributed plus local processes) captured by activity-flow-mapped category selectivity (i.e., a model based upon distributed processes only, Fig. 2B). We propose that the percent leftover (i.e., not captured via activity flow mapping) is due to either local processes (not isolated in the model) or other sources of model error (e.g., noise in the data). We used Formula 4:
If the percentage given by Formula 4 was statistically greater than 50%, we inferred that category selectivity in that functional complex was generated primarily via distributed activity flow processes. Note that these estimates reflect a lower bound on the contribution of distributed processes, since they can be driven lower not only by local processes but by other sources of model error.
Network analyses
In assessments of the fully-distributed network interaction scheme of Fig. 1A, we tested the contributions of each large-scale CAB-NP functional network (Ji et al. 2019, see main text Fig. 3C) to activity-flow-mapped responses. We did this at both the levels of: (1) mapped category-specific responses, and (2) mapped response profile.
Category-specific network contributions
The activity flow map (Fig. 2C step 3: “activity flow products”, or just “flows”) quantified how much activity of a given source region was weighted by functional connectivity with the held out target region (i.e., before summing in Formula 1). We used the network-based averages of these flow values for each functional complex and its respective visual processing category to probe category-specific network contributions to the activity flow map. Source regions were averaged together based upon their CAB-NP assignments. These contributions were each statistically compared to one another (see Experimental design and statistical analysis) to discover the networks that contributed the most to a given activity flow map.
Response profile network contributions
To measure network contributions to activity flow mapping at the response profile level (i.e., across all 24 HCP conditions), we conducted a dominance analysis (Azen and Budescu 2003). We first re-computed activity flow mapping (across all conditions for a given functional complex) with source regions iteratively restricted by CAB-NP network assignment. Note that these values exactly summed to the whole-cortex, activity flow mapping, thus this approach simply parsed a given mapping into its network components. Then, we conducted multiple linear regression (per participant, across conditions), where network-level mapped activations were the predictor variables (12 in total) and actual activity was the response variable. Following dominance analysis, this yielded the “full model” explained variance (R2), which in the present study refers to the proportion of variance in the actual response profile explained by the network-based, mapped response profile. An iterative subset of multiple regression models were then performed, with all combinations of predictors tested. These can be thought of as subset models because they were iterated on choosing one through eleven predictors, and performing multiple regression on all combinations therein (procedural details are openly available, and can be found here: github.com/dominance-analysis/dominance-analysis). This followed a standard combination calculation, nCr, where n was the full number of predictors, or 12, and r was iterated between one and eleven (note that r=12 describes the full model). In the present study, there were a total of 4095 subsets of models for each functional complex assessed. The incremental contribution of each predictor to R2 was quantified for each of these subsets of models. Then, the cross-model average of these incremental contributions was taken as the partial R2 of each network, which added up to equal the full model R2. Given this property of dominance analysis (that the partial R2 values add up to the full R2), the percentage of relative importance of each network was calculated as the partial R2 divided by the full R2, times 100. The dominance analysis approach has the benefit of being robust to multicollinear predictor variables (Kraha et al. 2012) (i.e., estimating unmixed partial R2).
Control analyses
To assess how critical each functional complex’s unique, fully-distributed (i.e., whole-cortex, as in Fig. 1A) connectivity fingerprint (as in Passingham et al. 2002; Mars et al. 2018) was to respective visual category selectivity results, we performed an FC substitution experiment. For each target functional complex (and per participant), we substituted each complex’s connectivity fingerprint with each other functional complex’s fingerprint, then recomputed category selectivity (see above, Category selectivity). This meant that for each of the four original, or true, mappings (bodies, faces, places, and tools), there were three null model comparisons. For example: original body complex rsFC substituted with (1) face complex rsFC, (2) place complex rsFC, and (3) tool complex rsFC (repeated likewise for each original model). To preserve an essential property of the connectivity graphs used in the present study, we set self-connections to zero (e.g., right EBA/FBA target to right EBA/FBA source) (for more on constraint considerations for network null models, see: Wiedermann et al. 2016; Markello and Misic 2020). Additionally, to account for the zero FC weights that were in the null functional complex’s self-connection locations, we set connectivity estimates between the original target complex and the null model complex with the transposed estimate, per hemisphere (e.g., right EBA/FBA target to right FFA/pSTS source set with right FFA/pSTS target to right EBA/FBA source). We performed paired samples t-testing to test the alternative hypothesis that the true mapped category selectivity was greater than the null mapped category selectivity (for each of the four visual categories and in each of the three respective FC substitution null models)
Stimulus-driven visual category selectivity
We sought to test the possibility that stimulus-driven activity flow processes conferred by V1 are capable of generating visual category selectivity (Fig. 1B), and compare this to selectivity generated by the fully distributed, whole-cortex mappings (Fig. 1A, see Activity flow mapping and Category selectivity above). All analyses assessing stimulus-driven network interactions were initialized with vertex-level data. Firstly, we estimated rsFC between all visual system (i.e., VIS1 and VIS2 network vertices from Ji et al. 2019, see Network analyses above) vertices, and refer to this visual sub-system as “VIS” throughout corresponding Results and Figures. Note that we included all functional complexes in this VIS sub-system. FC was estimated with combinedFC (Sanchez-Romero and Cole 2021a) as in prior analyses (see Resting-state functional connectivity estimation), except regularized regression was used instead of multiple regression to scale edge weights, given that there were more vertices than time points in vertex-level resting-state data. For each participant and target vertex (i.e., dependent variable, or, vertex timeseries that connections were to be estimated with), we used L1 regularized, lasso regression with cross-validation across resting-state runs. The penalty term was chosen from a wide sweep of 100 log-spaced terms. We then conducted a multi-step activity flow mapping procedure. We report step 1 as the V1, stimulus-driven model, and steps 2+ as the extended VIS model.
In step 1, only activation patterns of source vertices from V1 (of both hemispheres), shaped by their connectivity fingerprints with each VIS vertex (see Activity flow mapping and Formula 1), were used to map the task-evoked activation patterns of each VIS vertex (Fig. 1B) (resulting maps for functional complex vertices were the focus of the step 1 analysis, but the entire VIS sub-system was mapped because it was later used in steps 2+). Note that cortical region V1 was identified based on the information provided by the MMP atlas of Glasser et al. (2016). Cortical V1 vertices were all those inside of the V1 border, with 10 mm dilated masking applied to activity flow mapping (with VIS) as described previously (see Resting state functional connectivity estimation and Activity flow mapping, Fig. 2) to control for the effects of spatial autocorrelation. When mapping held-out, target vertices of each functional complex, the across-vertex (i.e., across V1) mean activation value to the corresponding category-specific condition was subtracted from each V1 source vertex to account for the potential confound that there may still be category-specific information fed back to V1 from the complex of interest (at the temporal resolution of fMRI). These mappings were then used to generate visual category selectivity (see Category selectivity and Formula 2) in each functional complex. Note that here, mapped and actual category selectivity was estimated for each vertex in each functional complex, then averaged (i.e., average selectivity of vertices in each complex). Thus, actual category selectivity in corresponding Results marginally varied from the region-level estimates based on “fully distributed” mappings (which was estimated for each region, then averaged). Additionally, we removed outliers (using a conservative threshold of +/-5 median deviations) as before (see Category selectivity). The degrees of freedom are indicated in corresponding Results that report hypothesis testing statistics.
We report how well stimulus-driven visual category selectivity compared to selectivity generated by fully distributed (i.e., whole-cortex source regions) activity flow processes. We also performed a control analysis testing whether each functional complex’s unique connectivity fingerprint with V1 determined its visual category selectivity better than a null rsFC network model. Here, we randomly permuted the connectivity architectures (per participant) 100 times, but maintained edge strength and degree (Newman and Girvan 2004; Fosdick et al. 2018; Markello and Misic 2021). Note that this was used instead of rsFC substitution (see Control analyses above) because there was a variable number of vertices in each functional complex, and thus they could not be used to substitute each other’s connectivity fingerprints. Nonparametric estimates of statistical significance given by Max-T (see Experimental design and statistical analyses) were used to compare real mapped category selectivity to the null network mapped category selectivity.
Then, in steps 2+ (“extended VIS”) we used the mapped activation patterns from step 1 as the source activation patterns in activity flow mapping (Formula 1), and extended this to include all other VIS sub-system source vertices (i.e., testing Fig. 1C). We repeated this until a “settling threshold” was reached. This was to simulate potential feedback and/or recurrent processes (initialized with stimulus-driven processes) that may occur over VIS network interactions. We defined a settling threshold as the step when the mapped activation patterns no longer changed within four decimal point precision, which can be thought of as the beginning of a horizontal asymptote in mapped activation values. We then performed all analyses as we did in step 1, to assess whether stimulus-driven activity flow processes further shaped by VIS network interactions improve the generation of visual category selectivity.
Category selectivity generated by stimulus-driven network interactions further shaped by fully distributed network interactions
Expanding upon the analyses in the previous section (Stimulus driven category selectivity), we sought to test whether V1-initialized activity flow mappings (step 1) that are then processed over all cortical network interactions (step 2) (i.e., testing Fig. 1D) – as opposed to a step 2 that incorporates only visual subsystems – would accurately generate visual category selectivity. Here, modulations conferred by all other functional networks (e.g., possible top-down effects from source regions in the attention network) were allowed to contribute to activity flow mapping in a second step. As before, we firstly initialized activity flow mapping with stimulus-driven interactions with V1 (as in Stimulus-driven visual category selectivity above). Then, instead of extending this to include multiple steps of source network interactions from a VIS sub-system, we conducted one additional step that included all cortical source regions. Given the computational infeasibility of vertex-level regularized regression (with cross-validation) across the whole cortex, this second step was performed at the region level. Following prior analyses, to control for multiple comparisons where a normal distribution was not expected a priori, we used max-T nonparametric permutation testing (see Experimental design and statistical analysis) to assess whether visual category selectivity was significantly above a hypothesized null mean of 1.0 (e.g., where responses in a given functional complex are equivalent to the category and non-categories of interest, see Formula 2). We used 100,000 permutations (i.e., p of 0.00001) and reported the max-T threshold at the 95% confidence interval.
Experimental design and statistical analysis
Each functional complex of brain regions that was proposed to be highly responsive to the visual categories of bodies, faces, places, and tools (see above, Identification of functional complexes) had the following estimates: actual activity and activity-flow-mapped activity. For each of these estimates and each visual category, we contrasted category responses to non-category responses (e.g., body vs non-body), hypothesizing that a functional complex would exhibit higher activity (in actual and mapped data) to the category vs the non-category. To test these hypotheses we employed one-tailed, paired samples t-tests on cross-participant data. For each category of interest, non-category activity was the average of the three remaining categories. For example, if body was the category of interest, responses to faces, places, and tools were averaged for non-body activations. This provided a consistent contrast across all four sets of analyses (i.e., always category vs non-category). In each of these sets of analyses, we corrected for false discovery rate (FDR) following Nichols and Hayasaka (2003) in estimates of statistical significance. The p-values reported in the Results section contrasting category vs non-category responses reflect FDR adjusted p-values, following the resampling procedure described by Yekutieli and Benjamini (1999). To assess accuracy of the activity-flow-mapped (Formula 1) results, we compared the actual and activity-flow-mapped activations in each of the four visual category analyses with Pearson’s r, MAE, and R2 (Fig. 2). Lastly, we implemented a Max-T nonparametric permutation approach (shuffling conditions over 100,000 permutations) to correct for multiple comparisons when testing whether category selectivity scores (Formula 2) revealed for each functional complex were greater than 1.0 (the null mean), as well as testing whether estimates of distributed processing contributions (Formula 4) were greater than 50% (see Category selectivity for more on these measures). We likewise used a Max-T nonparametric permutation approach (here with 10,000 permutations due to computational limits) to address multiple comparisons in the network-based analyses (i.e., comparing each network-level result to every other network-level result). These Max-T analyses derived a 95% confidence interval from the maximum T distribution (Nichols and Holmes 2002). The max-T procedure was used in any analysis where we could not assume a normal distribution a priori. Throughout pertinent Results sections we refer to the statistical threshold derived from the permuted models as the max-T threshold (d.f.), wherein a statistically significant p value indicates that the real t-statistic was larger than this threshold. Note that for category selectivity and percent distributed processing analyses, degrees of freedom were based on sample size with outlier participants removed (see Category selectivity). All findings are reported for cortical regions in the right hemisphere for the discovery dataset. Left hemisphere and replication dataset results are reported in the Supplement.
Code, software, and data accessibility
All MATLAB, python, demonstration code, and software are publicly available on GitHub. The master repository for the present study can be found here: https://github.com/ColeLab/ActFlowCategories. The Activity Flow Toolbox and related software (Cole et al. 2016) can be found here: https://github.com/ColeLab/ActflowToolbox. Further, the HCP maintains open access imaging data at various levels of processing here: https://www.humanconnectome.org/. Data at other levels of processing or analysis pertinent to the present study are available upon request.
Results
Response profiles are captured by activity flow processes
Our preliminary hypothesis was that activity flow processes could accurately map responses in classically studied visual cortex regions. Firstly, we extended prior work (Cole et al. 2016; Ito et al. 2020c; Cocuzza et al. 2022) by implementing a resting-state functional connectivity (rsFC) method (Sanchez-Romero and Cole 2021a) that estimates a more plausible network (Fig. 3A-B), and by analyzing a large cohort (N=352) split into discovery and replication datasets. These updates serve to improve the inferential validity of activity flow mapping results (Reid et al. 2019). Secondly, we quantified response profiles (also termed population receptive fields) of functional complexes identified as integral to four visual categories (Tables 1-2). A response profile (rows of Fig. 3D) was defined as the task-evoked activity levels across all 24 HCP conditions. Establishing response profiles allowed us to verify that each complex was responsive to their respective visual categories (outlined in black: Fig. 3D right), and verify that activity flow mapping (Fig. 3D left) was reliable across a variety of cognitive domains.
Activity-flow-mapped response profiles maintained high accuracy in both hemispheres (right: r = 0.92, MAE = 3.93, R2 = 0.80; left: r = 0.92, MAE = 3.96, R2 = 0.79; replication dataset: Table S1), which corroborates prior findings that activity flow processes (specified by connectivity patterns) can accurately map task-evoked activity in held out brain regions across diverse cognitive domains (Cole et al. 2016). Moreover, this suggests that intrinsic (i.e., resting-state) functional network organization (Fig. 3C) plays a prominent role in generating local response profiles. Subsequent analyses examined the processes involved in category-selective responses (i.e., focusing on black-outlined conditions in Fig. 3D), as generated by activity flow processes.
Local visual category selective responses are sufficiently generated via fully distributed network interactions
We next tested the hypothesis that the classically observed selectivity for visual categories in select visual cortex regions is shaped by their unique connectivity fingerprints fully distributed across the whole cortex (Fig. 1A). Each visual category studied enjoys an extensive literature reporting brain regions thought to represent (or compute) said categories and/or their component features (Fig. 4, Table 1). An emerging literature suggests that network interactions and/or distributed activation patterns are integral to visual category processing (Haxby et al. 2001; Norman-Haignere et al. 2012; Li et al. 2019). Here, we sought to test the generative capacity of fully distributed network interactions by modeling activity flow processes across the whole cortex (Cole et al. 2016). The four visual category models are reported in separate sections below with similar formatting to aid cross-comparison. The functional complexes identified based on prior literature (Fig. 4, Tables 1-2) were anatomically positioned along occipital (anterior and lateral to V1) and temporal (tending posterior and ventral to A1) cortices, which reflects the associative processing demands of visual category computations (Wierenga et al. 2009; Op de Beeck et al. 2019). Functional complex regions belonged primarily to the secondary visual (VIS2) network, with additional locations in dorsal attention (DAN), default mode (DMN), and language networks (LAN) (Fig. 4) (Ji et al. 2019).
(A) A schematic of the cortical surface in three gross orientations, with MMP regions (Glasser et al. 2016) outlined in silver and functional complexes relevant to processing body features (see Tables 1-2, Methods) colored in based on CAB-NP network affiliations (Ji et al. 2019; Fig. 3C) and identified with text labels and arrows. EBA = extrastriate body area; FBA = fusiform body area. VIS2 = secondary visual network; DAN = dorsal attention network. “r” denotes right hemisphere; “l” denotes left hemisphere. (B) Functional complexes related to face processing. FFA = fusiform body area; pSTS = posterior superior temporal sulcus; LAN = language network; DMN = default mode network. (C) Functional complexes related to places and scenes. RSC = retrosplenial complex; PPA = parahippocampal place area. (D) Functional complexes related to processing images of tools and objects. LOC = lateral occipital complex.
We tested the hypothesis that fully distributed activity flow processes (Fig. 1A) – shaped by the connectivity fingerprint of each functional complex (Fig. 4) – sufficiently generates category-selective responses. Before this, we began with two confirmations essential for subsequent tests of this hypothesis. We first constrained prior activity flow mappings (Cole et al. 2016) to body, face, place, and tool conditions and compared actual versus activity-flow-mapped responses across the whole cortex (Fig. S1). We then focused on activations in each functional complex (Fig. 4) and in each hemisphere, and quantified “benchmark” contrasts to corroborate the literature by comparing the category of interest versus non-category responses (e.g., body versus non-body) in the actual and activity-flow-mapped data (note that category selectivity is reported in subsequent sections). Whole-cortex, activity-flow-mapped responses to visual categories were highly accurate (bodies: r = 0.89, MAE = 5.27, R2 = 0.78; faces: r = 0.86, MAE = 5.83, R2 = 0.72; places: r = 0.88, MAE = 5.85, R2 = 0.77; tools: r = 0.89, MAE = 5.62, R2 = 0.78; Fig. S1). Significant benchmark contrasts in each functional complex were also observed in both the actual (bodies: t(175) = 26.65, p = 4.42 x 10-63, Cohen’s d = 1.42, faces: t(175) = 20.37, p = 5.26 x 10-48, Cohen’s d = 1.23; places: t(175) = 39.53, p = 2.95 x 10-88, Cohen’s d = 2.78; tools: t(175) = 8.49, p = 4.71 x 10-15, Cohen’s d = 0.31) and activity-flow-mapped data (bodies: t(175) = 17.91, p = 1.79 x 10-41, Cohen’s d = 0.88; faces: t(175) = 10.06, p = 2.72 x 10-19, Cohen’s d = 0.56; places: t(175) = 26.06 p = 7.62 x 10-62, Cohen’s d = 1.78; tools: t(175) = 9.94, p = 5.55 x 10-19, Cohen’s d = 0.32) (Fig. S2) (left hemisphere and replication: Tables S1-S2).
Distributed processes contribute to body category selectivity
Processing of human body images (Barch et al., 2013) is associated with the fusiform body area (FBA) and the extrastriate body area (EBA) (Downing et al. 2001; Taylor et al. 2007; Orgs et al. 2016), which are located in the lateral occipito-temporal cortex (Tables 1-2, Fig. 4A). EBA computations are thought to discriminate between bodily attributes and parts (Carey et al. 2019) and provide postural information to frontoparietal regions (Zimmermann et al. 2018), indicating a pivotal role in action planning. The FBA is thought to process images of whole bodies (Taylor et al. 2007) and pairs of body parts (Bratch et al. 2018). Extending the fully distributed hypothesis to the processing of body images, we hypothesized that activity flowing via the connectivity fingerprint of the EBA and FBA (henceforth: EBA/FBA) (Fig. 5A-B) — its unique pattern of distributed cortical connections — sufficiently determines its body-selective responses.
(A) The activity flow mapping procedure (steps match Methods Fig. 2C) generating body responses in EBA/FBA (black), projected onto cortical schematics (right hemisphere). Green: source vertices excluded from analyses. Step 2 was not blacked out for visual comparison with step 4, however it was held out in-analysis. Step 4 color scale shows maximum of all regions’ mapped activations to body images for visual comparison with step 2. (B) The connectivity fingerprint (Passingham et al. 2002; Mars et al. 2018) of the right EBA/FBA via rsFC (black lines). Radial lines: source regions connected to EBA/FBA, clustered by functional network assignments (Ji et al. 2019) (colored per legend; Fig. 3C). 95% confidence interval: across participants. (C) Right EBA/FBA body selectivity: activity-flow-mapped (purple) and actual (coral). Gray dots: individual participants’ scores. (D) Estimated contribution of distributed network interactions to body selective responses in EBA/FBA. (E) The activity flows (as in A3) of each source region contributing to mapped EBA/FBA responses to body images (statistical significance asterisks at network-mean level). (F) Variance explained by each network-restricted activity flow model (unmixed partial R2 in gray) of EBA/FBA’s response profile. Black lines: 95% confidence interval across participants. Asterisks: statistical significance versus each other network. E-F suggests that activity flowing over interactions with VIS2 and DAN represents a general network coding mechanism for EBA/FBA. See Methods for full details of each analysis.
We assessed body selectivity in the EBA/FBA — quantified as the ratio of body to non-body category activations — in both activity-flow-mapped (Fig. 5C, purple) and actual (Fig. 5C, coral) data. As expected, mapped and actual body selectivity were statistically significant relative to a selectivity ratio of 1.0 (i.e., greater than the null hypothesis of equivalent responses to bodies and non-bodies): activity-flow-mapped mean body selectivity = 1.35, Cohen’s d = 1.29; actual mean body selectivity = 1.67, Cohen’s d = 1.55; max-T threshold(166) = 5.88, p < 0.00001; Fig. 5C). We next estimated the contribution of distributed activity flow processes to body selectivity via the percentage of actual body selectivity captured by mapped body selectivity (Fig. 5D). Given that the fully distributed activity flow approach maps task-evoked activity based on distributed features (Cole et al. 2016, Fig. 2B), this ratio approximates the extent that distributed network interactions contribute to the generation of body selectivity in the EBA/FBA (relative to local processes and/or error). As expected, the estimated contribution of distributed activity flow processes was 81%, which was statistically greater than 50% (see Methods) (Cohen’s d = 2.52, max-T threshold(166) = 6.09, p < 0.00001; Fig. 5D) (left hemisphere and replication: Table S3).
Next, we examined how each large-scale functional network (Fig. 3C, Ji et al. 2019) contributed to the observation that activity flowing over a fully distributed intrinsic network architecture (connectivity fingerprint: Fig. 5B) shaped body selectivity in the EBA/FBA. The activity flows map (Fig. 5A, step 3) — where source activity was weighted by rsFC with the (held out) target region — was an entry point to assess such network contributions. Firstly, we averaged the estimated activity flows (i.e., source region contributions to mapped body activations of the EBA/FBA) based on network assignment (Fig. 5E). Secondly, we used dominance analysis to identify each functional network’s unmixed contribution (partial R2, see Methods) to the activity-flow-mapped response profile (i.e., activations across all 24 conditions, Fig. 3D) of the EBA/FBA (Fig. 5F). Secondary visual network (VIS2) activity flows exhibited the largest contribution (versus each other network) to EBA/FBA’s body-evoked activations (Fig. 5E, max-T threshold (175) = 3.39, p < 0.0001; left hemisphere and replication: Table S4). Dorsal attention network (DAN) activity flows were also significantly higher (same max-T thresholds, see Methods) than most other networks, except VIS2 and posterior multimodal network (PMM). VIS2 also accounted for most of the explained variance in EBA/FBA’s response profile (partial R2 = 0.51, 61% of the full model R2, Fig. 5F). DAN accounted for the next highest amount of explained variance (partial R2 = 0.09, 10% of the full model R2, Fig. 5F; left hemisphere and replication: Tables S5-S7). Thus, for both body-specific conditions and across all conditions (i.e., response profile), activity flowing over VIS2 and DAN exhibited substantial contributions to EBA/FBA activation patterns.
As hypothesized by the fully distributed network interaction model (Fig. 1A), we found evidence that activity flowing over whole-cortex, resting-state functional connections shaped body selectivity in the EBA/FBA. Results also suggested that distributed processes chiefly specified local body-selective responses in the EBA/FBA. The total explained variance in the EBA/FBA’s 24-condition response profile was significantly greater than 50% (total R2 = 0.83, versus 0.5: t(175) = 66.07, p = 1.14 x 10-125; left hemisphere and replication: Tables S5-S7), suggesting that distributed activity flow processes predominantly influence EBA/FBA responses across a variety of cognitive domains.
Distributed processes contribute to face category selectivity
The fusiform face area (FFA) is a hallmark region exhibiting localized category selectivity given its well-replicated relationship with face processing (Kanwisher et al. 1997; Kanwisher and Yovel 2006). The posterior superior temporal sulcus (pSTS) is thought to be a multisensory processing region responsive to the changeable aspects of faces (Puce et al. 1998; Osher et al. 2016). The FFA and pSTS are loci of specialization in a two pathway model (Baseler et al. 2014; Li et al. 2019), representing facial identity (relatively invariant) and facial expression (dynamic), respectively (Haxby et al. 2001; Pitcher et al. 2020). While both the FFA and pSTS are part of the VTC, the pSTS is more dorsal and lies adjacent to the temporoparietal junction; the FFA is more posterior and typically identified on the ventral aspect of the cortex (i.e., the fusiform gyrus) (Tables 1-2, Fig. 4B). Extending the fully distributed hypothesis (Fig. 1A) to the processing of face images, we hypothesized that activity flow processes via the whole-cortex, resting-state connectivity fingerprint of the FFA and pSTS (Fig. 6A-B) (henceforth: FFA/pSTS) sufficiently determine its face selective responses.
All figure specifications follow Fig. 5. (A) The activity flow procedure mapping activations to face categories, projected onto cortical schematics (right hemisphere). Right FFA/pSTS was the held out target complex. (B) The connectivity fingerprint (Passingham et al. 2002; Mars et al. 2018) of the right FFA/pSTS via whole-cortex rsFC (black lines). Radial lines: source regions connected to the FFA/pSTS, clustered by functional network assignments (Ji et al. 2019) (colored per legend and Fig. 3C). (C) Face category selectivity exhibited by the right FFA/pSTS. (D) Estimated contribution of distributed activity flow processes to face selectivity exhibited by the right FFA/pSTS. (E) Activity flows (as in A step 3) of each source region contributing to the mapping of FFA/pSTS responses to face images. VIS2 regions contributed most to the FFA/pSTS mapped activation magnitude to faces. (F) Variance explained by each network-restricted activity flow model (unmixed partial R2 via dominance analysis; Methods) of the right FFA/pSTS’ response profile. VIS2 accounted for the most variance, altogether suggesting that activity flowing over VIS2 regions represents a general network coding mechanism for FFA/pSTS processing. DAN and DMN regions also accounted for a nontrivial amount of variance at the response-profile level suggesting that, across diverse cognitive domains, FFA/pSTS processing is impacted by activity flowing over DAN and DMN regions, in addition to VIS2 (in the face-specific case).
We followed the foregoing pipeline for body selectivity exactly, except analyzing face images and the FFA/pSTS (Fig. 6A, black regions). Overall, as expected, the pattern of results observed for body processing (EBA/FBA) extended to face processing (FFA/pSTS). Significant face selectivity was observed in both the actual and activity-flow-mapped data (mapped mean face selectivity = 1.33, Cohen’s d = 0.83; actual mean face selectivity = 1.37, Cohen’s d = 0.89; max-T threshold (170) = 7.35, p < 0.00001; Fig. 6C), and the estimated contribution of distributed activity flow processes was 96% (Cohen’s d = 2.27, max-T threshold (170) = 4.83, p < 0.00001; Fig. 6D) (left hemisphere and replication: Table S3). We found that face activations in the FFA/pSTS were most influenced by activity flows over VIS2 connections (max-T(175) = 3.39, p<0.0001; Fig. 6E; left hemisphere and replication: Table S4). VIS2 also accounted for the most variance in FFA/pSTS processing at the response profile level (40%; left hemisphere and replication: Tables S5-S7).
These results suggest that face selectivity in the FFA/pSTS was significantly shaped by activity flow processes over a fully distributed intrinsic network architecture. These processes were most prominently influenced by activity flowing over VIS2, similar to the EBA/FBA. Across all conditions, activity flowing over dorsal attention (DAN) and default mode (DMN) network connections were predictive as well (DAN: 11%, DMN: 9%). The total explained variance in this response profile model (Fig. 6F) was R2 = 0.89 (versus 0.5: t(175) = 92.61, p = 1.32 x 10-150; left hemisphere and replication: Tables S5-S7) suggesting that distributed processes chiefly influenced FFA/pSTS responses across many cognitive domains.
Distributed processes contribute to place category selectivity
The next visual category contained place images, sometimes termed scenes, environment, or topography. Place-specific regions include the parahippocampal place area (PPA) and the retrosplenial cortex (RSC) (Tables 1-2, Fig. 4C), which are thought to act cooperatively toward a cohesive percept (Epstein and Higgins 2007; Park and Chun 2009; Sulpizio et al. 2014). The PPA is thought to compute viewpoint-specific discrimination (Epstein et al. 2003), as well as mediating (or binding) contextual associations pertinent for episodic memory (Aminoff et al. 2013). The RSC is thought to provide the medial temporal lobe with visuospatial information (Kravitz et al. 2011), and to integrate viewpoint-invariant information for navigation and learning (Park and Chun 2009). The PPA consisted of previously identified regions (Weiner et al. 2018), and the dorsal RSC corresponded to Brodmann areas 29 and 30 (Vann et al. 2009) (Fig. 4C). Extending the fully distributed hypothesis (Fig. 1A) to the processing of place images, we hypothesized that activity flow processes that were parameterized by the whole-cortex, resting-state connectivity fingerprint of the PPA and RSC (Fig. 7A-B) (henceforth: PPA/RSC) sufficiently determines its place-selective responses.
All formatting and figure specifications follow Fig. 5. (A) Activity flow mapping of activations to place categories, projected onto cortical schematics (right hemisphere only). Right PPA/RSC was the held out target complex. (B) The connectivity fingerprint of the right PPA/RSC, as in Passingham et al. (2002), except with whole-cortex rsFC (black lines). Radial lines: source regions connected to the PPA/RSC, clustered by CAB-NP functional network assignments (Ji et al. 2019) (colored per legend and Fig. 3C). (C) Place category selectivity exhibited by the PPA and RSC in the right hemisphere. (D) Estimated contribution of distributed activity flow processes to the emergence of place selective responses in the right PPA/RSC. (E) Activity flows (as in A step 3) of each source region contributing to the mapping of PPA/RSC responses to place images. VIS1, VIS2 and DAN contributed most to the right PPA/RSC mapped activation magnitude to place categories. Note that VIS1, VIS2, and DAN were all statistically greater than each other network, except for each other. (F) Variance explained by each network-restricted activity flow model (partial R2 via dominance analysis; Methods) of the right PPA/RSC’s response profile. VIS2, DAN, and DMN accounted for the most variance, suggesting that activity flowing over regions in these networks represents a general network coding mechanism for PPA and RSC processing, while VIS1 contributes to place-specific responses.
Significant place selectivity was observed in the PPA/RSC (mapped mean place selectivity = 1.8, Cohen’s d = 1.49; actual mean place selectivity = 2.59, Cohen’s d = 1.74; max-T threshold (166) = 3.08, p < 0.00001; Fig. 7C), and the estimated contribution of distributed processes was 69% (versus 50%: Cohen’s d = 1.15, max-T threshold (166) = 6.88, p < 0.00001; Fig. 7D) (left hemisphere and replication: Table S3). Activity flowing over VIS1, VIS2, and DAN provided the largest contributions to PPA/RSC’s place activations (all significant except when compared to each other: max-T threshold (175) = 3.4, p<0.0001; Fig. 7E) (left hemisphere and replication: Table S4). A similar set of networks accounted for the most variance at the response profile level (Fig. 7F), including VIS2 (44%), DAN (13%) and DMN contributions (21%) (left hemisphere and replication: Tables S5-S7).
As hypothesized by the fully distributed model of Fig. 1A, the whole-cortex connectivity fingerprint of the PPA/RSC significantly shaped its place selectivity. PPA/RSC responses were influenced by activity flowing over VIS2 and DAN. Additionally, VIS1 was particularly important for place-specific activity; and DMN for cross-domain activity (Fig. 7E-F), suggesting that PPA/RSC’s activity flow processes were more heterogeneous than prior models. The total variance in the PPA/RSC response profile explained by activity flow processes (Fig. 7F) was greater than 50% (total R2 = 0.68; t(175) = 23.93, p = 4.51 x 10-57; left hemisphere and replication: Tables S5-S7), which provides evidence that distributed processes predominantly influenced PPA/RSC activations to a variety of cognitive domains.
Distributed processes contribute to tool category selectivity
The final visual category included tool images (Barch et al., 2013), sometimes termed inanimate objects. Following an extensive literature on object recognition (Grill-Spector et al. 2001; Bar 2004), we hypothesized that tool selectivity is exhibited by the lateral occipital complex (LOC) (Beauchamp et al. 2002; Osher et al. 2016), which is posteriorly located and wraps around the cortex in the ventromedial direction (Fig. 4D). The LOC is thought to represent higher-level information of objects, as opposed to low level visual features (Kourtzi and Kanwisher 2001), suggesting a role at the category level of visual processing. However, reports vary on the degree of semantic content processed by the LOC (Victoria et al. 2019). Thus, the link between the LOC and tool selectivity was the least clear (of all four models) a priori. Extending the fully distributed hypothesis (Fig. 1A) to the processing of tool images, we hypothesized that activity flow processes that were parameterized by the whole-cortex, resting-state connectivity fingerprint of the LOC (Fig. 8A-B) sufficiently determines its tool selective responses.
All formatting and figure specifications as in Fig. 5. (A) Activity flow mapping of activations to tool categories in the held out target -the right LOC - projected onto cortical schematics (right hemisphere). (B) The connectivity fingerprint of the right LOC, as in Passingham et al. (2002), except with whole-cortex rsFC (black lines). Radial lines: source regions connected to the LOC, clustered by CAB-NP functional network assignments (Ji et al. 2019) (colored per legend and Fig. 3C). (C) Tool category selectivity exhibited by the LOC in the right hemisphere. (D) Estimated contribution of distributed activity flow processes to the emergence of tool selective responses in the right LOC. (E) Activity flows (as in A step 3) of each source region contributing to the mapping of LOC responses to tool images. VIS2 contributed most to the right LOC mapped activation magnitude to tool categories (F) Variance explained by each network-restricted activity flow model (partial R2 via dominance analysis; Methods) of the right LOC’s response profile. VIS2 accounted for the most variance, suggesting that activity flowing over regions in these networks represents a network coding mechanism for LOC processing.
Tool selectivity in the LOC was statistically significant (mapped tool selectivity = 1.12, Cohen’s d = 0.88; actual tool selectivity = 1.15, Cohen’s d = 0.95; max-T threshold (174) = 9.61, p < 0.00001; Fig. 8C), and the estimated contribution of distributed activity flow processes to tool selectivity in the LOC was particularly high at 98% (Cohen’s d = 8.45, max-T threshold (174) = 5.15; Fig. 8D) (left hemisphere and replication: Table S3). VIS2 activity flows demonstrated a strikingly high contribution to LOC’s tool-specific responses (max-T(175) = 3.4, p<0.0001; Fig. 8E) (left hemisphere and replication: Table S4), and accounted for the majority of the variance in LOC’s response profile (79%; Fig. 8F) (left hemisphere and replication: Tables S5-S7).
As hypothesized, tool selectivity was strongly influenced by distributed activity flow processes (97-98%). As in all other models (bodies, faces, and places) the total explained variance that activity flow mapping captured for the LOC response profile was greater than 50% (total R2 = 0.90 vs 0.5: t(175) = 115.55, p = 3.71 x 10-167). Given that the standard application of activity flow mapping (Cole et al. 2016) (i.e., following the fully distributed network interaction scheme of Fig. 1A) is a distributed processing model (Fig. 2B), this suggests that distributed processes represent a general mechanism in LOC activations.
Control analyses: null connectivity fingerprints reduce visual category selectivity
To assess the extent that each functional complex’s unique, whole-cortex, resting-state connectivity fingerprint (Figs. 5B-8B) — its placement in the brain’s intrinsic network architecture — determined its visual category selectivity (Figs. 5C-8C), we built null models based on FC substitution. We hypothesized that a null connectivity fingerprint (i.e., a fingerprint based on the wrong functional complex) would confer significantly lower activity-flow-mapped category selectivity.
As hypothesized, visual category selectivity was significantly greater when based upon the true whole-cortex connectivity fingerprints (Table 3). This assessment is more stringent than a null model that randomly scrambles FC because there is likely some similarity in visual processing (and therefore activity flow patterns) across the four functional complex’s fully distributed network interactions. However, no substituted connectivity fingerprint was sufficiently similar to the true fingerprint to generate the original mapped category selectivity. Each of these results were corroborated in the left hemisphere and in the replication dataset (Tables S8-S9). These results corroborate that the fully distributed, resting-state connectivity fingerprint unique to each functional complex significantly shaped its visual category selective response. Given that connectivity fingerprints are the bases of activity flow processes (Fig. 1A, Formula 1) which were used to generate mapped category selectivity, these results further support the proposition that fully distributed network interactions can sufficiently generate local visual category selectivity in four functional complexes.
Null model connectivity fingerprints do not generate visual category selective responses in functional complexes.
Stimulus-driven activity flow processes are a key step in generating category selectivity
We sought to test a refined hypothesis that localized visual category selectivity is sufficiently generated by stimulus-driven activity flowing over a functional complex’s connectivity fingerprint with V1 only (Fig. 1B, Fig. 9A). This does not negate the “fully distributed” hypotheses (Fig. 1A), but instead specifies that activity flow processes instantiated in V1 are key to our prior observations that fully distributed network interactions can sufficiently generate local visual category selectivity. Alternatively, it is possible that network interactions with V1 account for only a small proportion of variance in visual category selectivity generated by the fully distributed network interaction models (Figs. 5-8). The latter possibility is supported by evidence that visual information processing (which may not be strictly hierarchical; see St-Yves et al. 2022; Sexton and Love 2022) is modulated by attention (Wojciulik et al. 1998; Saalmann et al. 2007; Kay and Yeatman 2017), prior expectations (Summerfield and de Lange 2014; de Lange et al. 2018), task goals/context (Vaziri-Pashkam and Xu 2017; Bracci et al. 2017; De Cesarei et al. 2019), and synthesis-related processes (i.e., continuously maintaining a percept from a noisy or ambiguous visual scene) (von Helmholtz 1867; Yuille and Kersten 2006; Fang et al. 2019). However, it is unclear if these top-down modulations (and other non-visual contributions, such as emotional modulation; as in Vuilleumier and Driver 2007) are indeed just modulations, or requirements for visual category selectivity.
(A) Theoretical schematic of stimulus-driven activity flow processes generating visual category selectivity (as in Fig. 1B). Given prior literature, mapped activity flow processes (gray arrow) have a refined inference: from V1 to later visual regions, we inferred that network interactions were stimulus driven. (B) Activity flow mapping procedure for the stimulus-driven model. All steps were conducted at the vertex level. Step 1: V1 sources were used to map targets across VIS. Note that the usage of “step 1” serves as a prelude to later steps tested in an extended visual system model (Fig. 10). (C) The null connectivity fingerprint (rsFC) model used for control analyses. The top depicts the true VIS network and the bottom depicts semi-random (edge degree and strength preserved) network architectures over 100 permutations. For visualization purposes, networks are shown at the region level, but analyses were conducted at the vertex level. (D) Actual (coral) and mapped (purple) visual category selectivity exhibited by the right EBA/FBA, FFA/pSTS, PPA/RSC, and LOC (left to right). Category selectivity exhibited by V1 (for each respective image category) is shown to demonstrate that activation patterns in V1 alone do not sufficiently account for mapped visual category selectivity.
We additionally sought to address a theoretical limitation borne out of whole-cortex, undirected network estimation from fMRI data. Namely, a limited ability to isolate whether activity flow processes contributing to a given complex’s category-selective response also contain some information from that complex itself. For example, do fully distributed activity flows generating FFA/pSTS face selectivity (Fig. 6) contain some information (potentially via feedback) from the FFA/pSTS? This type of causal circularity is a theoretical concern for the ongoing goal of delineating the extent that network interactions contribute to the generation of local processes with refined detail. However, given the accuracy of whole-cortex activity flow mapping (Figs. 5-8), it is likely that fully distributed network interactions sufficiently capture these processes, just at a broad (i.e., aggregated) scale (but with inferential limits).
To these ends, we developed an intrinsic-connectivity-based, generative model of category-selective responses that reduces the impact of causal circularity by constraining activity flows to V1 (sources) and functional complexes (targets) (Fig. 9A-B). Given robust evidence that activity in V1 represents retinotopic mapping and feature detection (Carandini et al. 2005; Wandell and Winawer 2011), this model parameterized mapped activation patterns as being stimulus-driven. We first estimated rsFC between all vertices in the visual system (including all functional complexes) (Fig. 9C, top). We will henceforth refer to this sub-system as “VIS”. In the following steps of this analysis, we focused on the interactions between V1 sources and functional complex targets, but it was important to include the rest of the visual system in the initial rsFC estimation steps. This is based on the assumption that, for any given visual system target (i.e., response variable), connectivity estimates (i.e., regression coefficients) should also account for the influence of other sources (i.e., predictors) that are thought to work in a functionally-linked system (as in the large-scale visual networks of Ji et al. 2019). We found that for each functional complex, stimulus-driven, activity-flow-mapped category selectivity was significantly above 1.0 (see Methods): right hemisphere EBA/FBA mean body selectivity = 1.12, max-T threshold (174) = 5.64, Cohen’s D = 0.45); right FFA/pSTS mean face selectivity = 1.14, max-T threshold (172) = 5.07, Cohen’s D = 0.41); right PPA/RSC mean place selectivity = 1.13, max-T threshold (173) = 5.27, Cohen’s D = 0.42); right LOC mean tool selectivity = 1.07, max-T threshold (172) = 5.41, Cohen’s D = 0.3) (Fig. 9D, see Table S10 for left hemisphere and replication dataset). In each of these four models, mapped category selectivity was statistically significant, but lower than when mapped via network interactions of the whole-cortex (Figs. 5C-8C). This suggests that while stimulus-driven activity flow processes alone are likely key for generating visual category selectivity (i.e., statistically significant), selectivity is underspecified when compared to distributed activity flow processes over the whole cortex (Figs. 5C-8C).
As in prior control analyses (Table 3), we tested whether each functional complex’s unique resting-state connectivity fingerprint (here, with V1) determined its visual category selectivity by using a null rsFC model. Note that V1 itself did not exhibit significant selectivity for any of the four visual categories (V1 body selectivity = 0.63, t(175) = -17.3, p = 1, Cohen’s D = -1.32; face selectivity = 0.87, t(175) = -12.9, p = 1, Cohen’s D = -0.99; place selectivity = 0.64, t(175) = -13.6, p = 1, Cohen’s D = -1.04; tool selectivity = 0.66, t(175) = -12.8, p = 1, Cohen’s D = -0.97) (Fig. 9D; see Table S11 for replication dataset), thus downstream selectivity would not be driven by activation patterns in V1 alone (also supported by: Grill-Spector et al. 1998a; Bryan et al. 2016; Coggan et al. 2017; Poltoratski et al. 2021). Given the varied number of vertices in each complex, we used randomly permuted connectivity architectures (Fig. 9C) – maintaining edge strength and degree (Newman and Girvan 2004; Fosdick et al. 2018; Markello and Misic 2021) – instead of rsFC substitution. Body, face, and place selectivity were significantly greater than the aggregate of 100 null rsFC models (body: t(175) = 6.4, p = 6.6 x 10-10, Cohen’s D = 0.49; face: t(175) = 11.6, p = 8.2 x 10-24, Cohen’s D = 0.89; place: t(175) = 3.5, p = 2.5 x 10-4, Cohen’s D = 0.27), but not tool selectivity (t(175) = -1.15, p = 0.8, Cohen’s D = -0.09) (see Table S12 for left hemisphere and replication dataset). This suggests that stimulus-driven category selectivity exhibited by EBA/FBA, FFA/pSTS, and PPA/RSC were significantly shaped by their unique connectivity patterns with V1. Given that cortex-wide network interactions were excluded from the model, it is possible that the mapped tool selectivity in Figure 9D received relatively more influence from V1 activation patterns than from its VIS connectivity fingerprint.
These results suggest that stimulus-driven network interactions based in V1 are a key step within whole-cortex, distributed activity flow processes (Fig. 1B). For some functional complexes, such as the LOC, early information patterns across V1 can select category-selectivity-generating activity flow processes in a manner less dependent on its VIS connectivity fingerprint (but note that LOC’s connectivity fingerprint appears more critical in the whole-cortex model; Table 3). For other complexes such as the FFA/pSTS, the stimulus-driven account of category selectivity appears to be influenced to a large extent by its VIS connectivity fingerprint, given its strong statistical significance over the null rsFC model.
Stepwise activity flowing across the visual system improves response profile accuracies, but not category selectivity
Given evidence that higher-level visual representations (such as category selectivity) involve information processing across the visual system (Fig. 10A) (Nagy et al. 2012; Grill-Spector et al. 2017; Conway 2018; Kietzmann et al. 2019), we sought to extend our stimulus-driven model and test the hypothesis that category selectivity mappings (which were significant but underspecified in the V1 model) improve when further accounting for all VIS network interactions (Fig. 1C). We conducted this analysis by using the mapped activation patterns from the stimulus-driven, V1 model (Step 1 in Figs. 9B-10B) – weighted by rsFC across VIS – to model later visual cortex activity flow processes (that were initialized by early visual computations in V1) (Fig. 10B, Step 2). This step was repeated to model potential bidirectional (i.e., reciprocal) and/or recurrent processes within the visual system (Kietzmann et al. 2019; Sexton and Love 2022) (Fig. 10B, Steps 3+), until a settling threshold was reached (i.e., where mapped activation patterns no longer changed, Fig. 10C, see Methods).
(A) Stimulus driven activity flow processes, further shaped by later visual interactions, generate local visual category selectivity (Fig. 1C). From V1 to later visual regions, we inferred that activity flow processes are stimulus driven (Fig. 1B, Fig. 9). Within the visual system, we inferred that activity flow processes are bidirectional and/or recurrent. (B) Activity flow mapping procedure for the extended visual system model. All steps were conducted at the vertex level. Step 1: V1 sources were used to map targets across the visual cortex (VIS) (Fig. 9). Step 2: mapped VIS activation patterns from Step 1 (weighted by connectivity estimates as in all activity flow models; see Methods) were used as sources to map held-out targets across visual cortex. Steps 3+: step 2 was repeated until a settling threshold was reached – or the point at which mapped values stopped changing (see Methods). (C) The settling threshold was reached at step 3. All further analyses only included steps 1-3 from panel B. (D) Accuracy of mapped activation patterns across all conditions (left: an average of all visual regions; right: functional complexes studied herein). Across the visual cortex, explained variance tended to increase with each step, indicating that the extended visual system model was improving mapped response profiles across the cortex. This pattern was also observed across functional complexes, with some exceptions, such as the PPA/RSC (which appears most accurate at step 2). (E-H) Actual (coral) and visual-system-mapped (purple) category selectivity (see Methods) exhibited by the right EBA/FBA (E), FFA/pSTS (F), PPA/RSC (G), and LOC (H). Mapped category selectivity is shown for each step in panel B.
Across-condition (i.e., response profile) mapping accuracy (of all VIS vertices) increased with each step (Fig. 10D), suggesting that overall, this extended model improved the specification of visual system responses. However, category selectivity was not improved consistently across functional complexes of interest. Mapped body selectivity in the right EBA/FBA was improved in step two (body selectivity = 1.18, max-T threshold (170) = 7.34, p<0.00001, Cohen’s D = 0.75) but became nonsignificant in step three (body selectivity = 1.01, max-T threshold (174) = 7.37, not significant, Cohen’s D = 0.03) (Fig. 10E). Face selectivity diminished in step 2 (face selectivity = 1.03, max-T threshold (174) = 7.11, not significant, Cohen’s D = 0.14) and step 3 (face selectivity = 0.9, max-T threshold (167) = 7.12, not significant, Cohen’s D = -0.71) (Fig. 10F), suggesting that within the confines of the visual system, stimulus-driven activity flows from V1 (step 1) were the best predictors of right FFA/pSTS face selectivity. Place selectivity in the right PPA/RSC improved in step 2 (place selectivity = 1.28, max-T threshold (172) = 5.3, p<0.00001, Cohen’s D = 1.43), but decreased to the original level in step 3 (place selectivity = 1.15, max-T threshold (171) = 0.97, p<0.00001, Cohen’s D = 0.88) (Fig. 10G). Tool selectivity in the right LOC improved in step 2 (tool selectivity = 1.28, max-T threshold (170) = 8.31, p<0.00001, Cohen’s D = 1.5), but decreased in step 3, albeit to a level higher than the initial V1 step (tool selectivity = 1.16, max-T threshold (171) = 8.74, p<0.00001, Cohen’s D = 1.23) (Fig. 10H, see Tables S13-14 for left hemisphere and replication dataset results). Interestingly, tool selectivity generated by step 2 was greater than the actual tool selectivity exhibited by the LOC (Fig. 10H, coral), suggesting that extended visual system modulations to activity flow processes initially over-specified tool selectivity, and that step 3 modeled the best-performing network interactions for mapping LOC tool selectivity. Thus, for two complexes (EBA/FBA and PPA/RSC), one additional step of extended visual system network interactions best-specified category selectivity. For one complex (FFA/pSTS), the V1 stimulus-driven (step 1) model was best, and for another complex (LOC) three steps were best.
Therefore, even though mapped response profiles tended to improve when modeling bidirectional and/or recurrent processes across the visual system, visual-system-mapped category selectivies did not consistently (i.e., in a generalizable manner) improve beyond what was specified by stimulus-driven activity flows from V1. Additionally, given that all observations were most robust in the fully distributed model (Fig. 1A; Figs. 5-8) (including response profile accuracies, which explained a maximum of 45% variance in the extended visual model, Fig. 10D, and up to 92% in the whole-cortex model, Fig. 3D), the visual system is likely further modulated by other systems (e.g., DAN interactions) to provide the full groundwork for distributed activity flow processes that stably generate local category selective responses (as in Figs. 5-8).
Visual category selectivity can be sufficiently generated by stimulus-driven activity flow processes further shaped by fully distributed network interactions
Given evidence that stimulus-driven network interactions with V1 significantly generated visual category selectivity (Fig. 9), but to a lesser extent than the fully distributed mappings (Figs. 5-8), and that this inconsistently improved when extended to later visual system interactions (Fig. 10), we sought to extend the stimulus-driven model to fully distributed network interactions (Fig. 1D, Fig. 11A). As in prior analyses, fully distributed network interactions refer to activity flow processes over all cortical source regions (Formula 1). First, using the extended mapping approach as in Fig. 10B, we initialized activity flow mapping with V1 network interactions, then incorporated all other network interactions in a second step (i.e., here, with network regions beyond just VIS, see Methods for full details). Across all functional complexes, we found that activity-flow-mapped category selectivity was not only statistically significant (via max-T permutation testing, see Methods), but was also remarkably close to selectivity generated by fully distributed network interactions (Figs. 5-8), and in some cases exhibited an improvement.
(A) A schematic of V1-initialized activity flow processes that are further propagated across all cortical network interactions (as in Fig. 1D). Here, fully distributed network interactions are initially established by activity flowing over the connectivity fingerprints of each functional complex with V1. We used the multistep mapping procedure of Fig. 10B (but with only two steps), except for at the region level (Methods). (B) Mapped (purple) and actual (coral) body selectivity exhibited by the right hemisphere EBA/FBA (gray dots: individual participants; boxplot line: median). Statistical significance is reported in the main text. (C-E) Same as B, but: (C) face selectivity in FFA/pSTS, (D) place selectivity in PPA/RSC, and (E) tool selectivity in LOC. Across all functional complexes, stimulus-driven and fully distributed mappings generated visual category selectivity that was greater than the stimulus-driven alone model of Fig. 9 and closely matched (and in some cases improved upon) the fully distributed alone model of Figs. 5-8. This suggests that activity flow processes initialized in V1, that are further processed over all cortical network interactions, are capable of generating highly accurate visual category selectivity, with reduced causal confounds (see text for full rationale).
Body selectivity in the EBA/FBA was statistically significant (mapped body selectivity = 1.41, Cohen’s d = 0.83; actual body selectivity = 1.67, Cohen’s d = 1.55; max-T threshold (166) = 6.21, p < 0.00001; Fig. 11B). Here, activity-flow-mapped body selectivity generated by stimulus-driven then fully distributed network interactions was an improvement upon selectivity generated from fully distributed interactions (mean = 1.35, Fig. 5). Face selectivity in the FFA/pSTS (Fig. 11C) was close to the fully distributed model (Fig. 6, face selectivity from the fully distributed mapping = 1.33) and statistically significant (mapped mean face selectivity = 1.26, Cohen’s d = 0.57; actual mean face selectivity = 1.38, Cohen’s d = 0.90; max-T threshold (165) = 7.30, p < 0.00001). Place selectivity in the PPA/RSC was improved (Fig. 7: place selectivity from fully distributed mapping = 1.8) in the stimulus-driven and fully distributed model (mapped place selectivity = 1.94, Cohen’s d = 1.0; actual place selectivity = 2.63, Cohen’s d = 1.58; max-T threshold (166) = 4.62, p < 0.00001; Fig. 11D). Lastly, tool selectivity in the LOC was statistically significant (mapped tool selectivity = 1.09, Cohen’s d = 0.58; actual tool selectivity = 1.15, Cohen’s d = 0.97; max-T threshold (168) = 7.4, p < 0.00001; Fig. 11E) (left hemisphere and replication: Table S15). Tool selectivity generated by the stimulus-driven and fully distributed mapping was lower than with the fully distributed mapping (mean = 1.12, Fig. 8) but higher than the stimulus-driven alone mapping (mean = 1.06, Fig. 10).
It is noteworthy that mapped visual category selectivity initialized only by stimulus-driven activity flow processes in V1 (then further shaped by whole-cortex interactions, Fig. 1D) closely matched selectivity generated by the fully distributed mappings (Fig. 1A). This was surprising given that the only information inputted into activity flow mapping (Formula 1) was V1 activation patterns and connectivity patterns of the resting-state network architecture. Indeed, when stimulus-driven activity flow processes were only extended to a visual subsystem (Fig. 10), mapped category selectivity inconsistently improved, and only to a marginal extent. However, when extended to network interactions of the entire cortex, mapped category selectivity markedly improved with consistency across functional complexes (Fig. 11). Firstly, this suggests that stimulus-driven activity flow processes conferred by V1 are key to local visual category selectivity (also supported by Fig. 9). Secondly, this suggests that distributed network interactions outside of just the visual system(s) (such as attention network regions, see Discussion) critically shaped V1-initialized activity flow processes to generate local visual category selectivity. Inferentially, there is less potential for circularity in the “stimulus-driven and fully distributed” mappings, given that activity flow processes were separately initialized via V1. Altogether, this account provides key details to our observations that distributed network interactions, shaped by unique connectivity fingerprints, can sufficiently generate visual category selectivity in four functional complexes.
Discussion
The foregoing results suggest that local category selectivity in classically studied regions of visual cortex is sufficiently generated by distributed activity flowing over functional connections (Figs. 5A-8A), which was best-specified when initialized with stimulus-driven network interactions (Fig. 11). We observed significant visual category selectivity in four functional complexes for both actual responses and responses generated via fully distributed activity flow processes (Fig. 1A, Figs. 5C-8C). Null network architectures revealed that each region’s unique, intrinsic connectivity fingerprint was a key parameter for activity flow mapping to be able to generate category selectivity. Lastly, comparing evidence across tests of stimulus-driven refinements to activity flow processes (Fig. 1B-D) supported the proposition that V1-initialized network interactions that were further shaped by fully distributed network interactions (Fig. 1D, Fig. 11) sufficiently generated category selectivity with improved inferential validity. Altogether, these findings support the hypothesis that distributed activity flow processes, specified by resting-state connectivity fingerprints of visual cortex regions, chiefly shape their category selective responses.
These findings build upon the notion of distributed and overlapping representations in the visual cortex (Haxby et al. 2001), as well as the general hypothesis that connectivity fingerprints determine a region’s functional repertoire (Passingham et al. 2002; Mars et al. 2018). Results here also extend observations that structural connectivity fingerprints (of the same regions studied here) can be used to predict visual category responses (Osher et al. 2016) by parameterizing activity flow processes (Cole et al. 2016) with empirically-estimated, intrinsic connectivity fingerprints. This goes beyond purely predictive models by explicitly formulating activity flow processes, in order to probe the allocation of distributed network interactions contributing to category selectivity. Activity flow mapping (Cole et al. 2016) treats cognitive computations as the local convergence of information propagating over a distributed, functionally-organized system (Ito et al. 2020c). In the present study, we leveraged the substantial evidence for localized category selectivity in visual cortex regions to test various network interaction schemes (and associated null comparison models), revealing how localized activations can be sufficiently generated by activity flow processes that are shaped by intrinsic functional connectivity fingerprints.
Importantly, fully distributed activity flow mappings (Figs. 5-8) did not account for all task-evoked activity variance, leaving room for local mechanisms to carry out critical computations. Here we adopted a definition of local computation consistent in fMRI and electroencephalography research: local activity reflects local field potentials, which are thought to reflect inputs and local/within-region processing (Logothetis 2003). Therefore it was not a guarantee that distributed processes would exhibit a predominant influence in this fMRI dataset, but two major lines of evidence support this claim. Firstly, fully distributed activity flow processes mapped visual category responses with high accuracy in all functional complexes. If local mechanisms strictly accounted for these responses we would expect lower accuracy, given prior simulation-based findings demonstrating that successful activity flow mapping requires high distributed processing (global coupling) and low local processing (self coupling/recurrence) (Fig. 2B, Cole et al. 2016). Mapped category selectivity within the fully distributed scheme (Fig. 1A) was thus a proxy for the degree of distributed processes underlying category selectivity. Secondly, the estimated contribution of distributed processes to category selectivity was significantly greater than 50% for all functional complexes assessed. We additionally observed that activity flow processes explained the majority of variance in response profiles (across 24 diverse task conditions) exhibited by each complex (Figs. 5F-8F), suggesting that activity flowing over global, intrinsic connectivity patterns is a general mechanism in the emergence of localized fMRI responses. However, large-scale functional networks had differential influence depending on the cognitive domain and target complex. For example, LOC responses exhibited a general, strong influence from VIS2 interactions (Fig. 8E-F), however, FFA/pSTS exhibited strong VIS2 influence in the face-specific model (Fig. 6E) with additional influence from DAN and DMN in the response profile model (Fig. 6F).
Given this, and the well-known observation that visual features are cortically established in V1 (Carandini et al. 2005; Wandell and Winawer 2011), we adapted activity flow mapping to test refined hypotheses about reasonable schemes of stimulus-driven network interactions that can generate visual category selectivity (Fig. 1B-D). We found evidence that when solely initialized by V1 source regions, activity flow processes generated significant but underspecified (compared to the fully distributed scheme) category selectivity (Fig. 1B, Fig. 9). This inconsistently improved when activity flow processes were further shaped by visual subsystem interactions (Fig. 1C, Fig. 10), but markedly (and consistently) improved when further shaped by fully distributed network interactions (Fig. 1D, Fig. 11). These results suggest that fully distributed network interactions that were initially established in V1 can generate local category selectivity. This approach allowed for improved inferential validity with respect to the theoretical causal confound that fully distributed (but not initialized with any specificity, as in Fig. 1A) interactions might contain some information from the functional complex itself. It is possible that an extended step of fully distributed interactions outperformed extended steps of visual subsystem interactions because of the influence of top-down modulations thought to be critical for visual processing (Wojciulik et al. 1998; Saalmann et al. 2007; Kay and Yeatman 2017; Vaziri-Pashkam and Xu 2017; Bracci et al. 2017). We encourage future work that systematically tests which non-visual-network interactions are most critical to further shaping stimulus-driven activity flow processes.
The present results shift the plausibility of neural processing hypotheses away from those assuming primarily localized processing and toward those establishing the mechanistic influence of distributed network interactions. This follows from our demonstrations that category selectivity in localized brain regions can be generated via an empirically-based distributed processing model: task-evoked activity flowing over connectivity fingerprints. However, while our resting-state connectivity estimates address causal confounds better than standard FC measures (Fig. 3A), they lack directional information. For instance, if ongoing within-region process variance strongly influenced downstream regions, then this distributed output activity would contribute (in the incorrect causal direction) to the mapping of the local processes. This possibility is unlikely, however, since such strongly category selective output would likely drive downstream regions to also exhibit robust category selectivity. However, all (or most) of the strongly category selective regions were included in each functional complex in our analyses, removing them from prediction sources. In addition, we partially addressed this possibility in the stimulus-driven network interaction models (Fig. 1B-D, see Methods), but there are still directionality constraints in any assessment of fMRI data. Nonetheless, our results specifically rule out hypotheses that assume a substantial role for within-region (e.g., recurrent) processes in driving localized category selectivity. For instance, our results rule out the hypothesis that elevated task-evoked face activity in FFA primarily reflects within-region computations (e.g., within-region interactions between neural populations) to detect or identify faces, with the output to other regions indicating whether a face is present. Instead, the present results suggest elevated task-evoked face activity in FFA primarily reflects connectivity-selected activity flows summing together weakly category-selective distributed responses to generate highly category-selective localized responses. It will be important for future research to disentangle input and output processes to and from visual category selective brain regions in order to further adjudicate between alternate hypotheses regarding the specific allocation of distributed network interactions.
In summary, we observed that distributed network interactions, specified by intrinsic connectivity fingerprints, are a primary contributor to the emergence of category selectivity in visual cortex regions. This built upon a history of theories regarding distributed and overlapping activation patterns (Haxby et al. 2001), connectivity fingerprints specifying regional functioning (Passingham et al. 2002; Mars et al. 2018), and a wealth of literature on locally specialized functioning. We leveraged activity flow mapping’s inherent sensitivity to global processing features (Cole et al. 2016) to estimate the contribution of distributed processes upon visual category selectivity. Looking forward, the present findings can facilitate examinations on the generative capacity of local and distributed processing mechanisms by constraining and contextualizing hypotheses.
Supplement
This section contains supplementary figures and tables. Supplementary figures and tables are explained and referenced throughout the main text and are referred to using the “S” prefix, for example: Fig. S1 and Table S1. The supplementary figures include two extended visualizations for results that corroborate prior work. The supplementary tables include all statistics and hypothesis testing results for the left hemisphere and for the replication dataset (right hemisphere, discovery dataset results reported in the main text).
Supplementary tables
Across select portions of cortex (whole cortex = all 360 MMP cortical regions; Glasser et al. 2016) and all n=176 participants, the accuracy of activity flow mapping was estimated by comparing of mapped and actual responses to select HCP conditions (response profile = across all 24 HCP conditions) via Pearson’s r, mean absolute error (MAE), and the coefficient of determination (R2).
p-values were FDR adjusted (as described in Methods and corresponding Results sections reporting right hemisphere discovery data).
Number of permutations in max-T nonparametric permutation testing: 100,000. d.f. = degrees of freedom (see Methods for details on outlier removal procedure).
Number of permutations in max-T nonparametric permutation testing: 10,000. sig. = significant. VIS1 = primary visual network; VIS2 = secondary visual network; DAN = dorsal attention network. These results corroborate results presented in Figures 4E-7E (right hemisphere discovery data; statistics reported in main text).
Source network = network based source of explained variance in activity-flow-mapped activations across 24 conditions (i.e., the response profile). VIS1 = primary visual network; VIS2 = secondary visual network; SMN = somatomotor network; CON = cingulo-opercular network; DAN = dorsal attention network; LAN = language network; FPN = frontoparietal network; AUD = auditory network; DMN = default mode network; PMM = posterior multimodal network; VMM = ventral multimodal network; OAN = orbito-affective network. rel. % = percent of relative importance to the full model. Asterisks = statistically significant network contributions (p < 0.0001, number of permutations = 10,000). EBA/FBA max-T(175) = 3.34; FFA/pSTS max-T(175) = 3.4; PPA/RSC max-T(175) = 3.35; LOC max-T(175) = 3.38. Statistical results listed in the bottom two rows refer to 1 sample t-testing of the total R2 value for each model versus 0.5, which assesses whether the mapped response profile for a given functional complex explains more than 50% of the variance in the actual response profile. This provides evidence that distributed processes (as captured by activity flow mapping) are the dominant influence in generating a given functional complexes activations to a diverse set of cognitive domains. n/a = not applicable. These results corroborate results presented in Figures 4F-7F (right hemisphere discovery data; statistics reported in main text).
Abbreviations as in Table S5. EBA/FBA max-T(175) = 3.34; FFA/pSTS max-T(175) = 3.4; PPA/RSC max-T(175) = 3.35; LOC max-T(175) = 3.38. Statistical results listed in the bottom two rows refer to 1 sample t-testing of the total R2 value for each model versus 0.5, which assesses whether the mapped response profile for a given functional complex explains more than 50% of the variance in the actual response profile. This provides evidence that distributed processes (as captured by activity flow mapping) are the dominant influence in generating a given functional complexes activations to a diverse set of cognitive domains. n/a = not applicable. These results corroborate results presented in Figures 4F-7F (right hemisphere discovery data; statistics reported in main text).
Abbreviations as in Table S5. EBA/FBA max-T(175) = 3.41; FFA/pSTS max-T(175) = 3.39; PPA/RSC max-T(175) = 3.42; LOC max-T(175) = 3.38. Statistical results listed in the bottom two rows refer to 1 sample t-testing of the total R2 value for each model versus 0.5, which assesses whether the predicted response profile for a given functional complex explains more than 50% of the variance in the actual response profile. This provides evidence that distributed processes (as captured by activity flow mapping) are the dominant influence in generating a given functional complexes activations to a diverse set of cognitive domains. n/a = not applicable. These results corroborate results presented in Figures 4F-7F (right hemisphere discovery data; statistics reported in main text).
In each analysis, the true model category selectivity scores (i.e., activity-flow-mapped with true connectivity fingerprint) were compared to the null model body selectivity scores (i.e., activity-flow-mapped with substituted connectivity fingerprints) across participants (paired samples t-test).
Same analyses as Table S8, but for left hemisphere functional complexes.
Number of permutations in max-T nonparametric permutation testing: 100,000. d.f. = degrees of freedom (see Methods for details on outlier removal procedure). This analysis corresponds to category selectivity generated from V1-initialized, stimulus-driven activity flow processes (Fig. 1A, Step 1 of Fig. 9).
As in the main text (reporting the discovery dataset results), bilateral V1 (vertex-level data, see Methods) itself did not exhibit significant category selectivity for any of the four visual categories. Given this observation, it is likely that intrinsic connectivity patterns (between V1 and functional complexes of interest), and not source activation patterns in V1, were key to stimulus-driven activity flow processes being able to generate visual category selectivity.
In each analysis, the true model category selectivity scores (i.e., via stimulus-driven activity flow mapping with true connectivity fingerprint with V1) were compared to the null model (Fig. 9C) category selectivity scores (i.e., activity flow mapping with randomly shuffled connectivity fingerprints, see Methods) across participants (paired samples t-test).
Number of permutations in max-T nonparametric permutation testing: 100,000. d.f. = degrees of freedom (see Methods for details on outlier removal procedure). This analysis corresponds to category selectivity generated from V1-initialized, stimulus-driven activity flow processes that are further shaped by one step of VIS subsystem network interactions (Fig. 1C, Fig. 10).
Number of permutations in max-T nonparametric permutation testing: 100,000. d.f. = degrees of freedom (see Methods for details on outlier removal procedure). This analysis corresponds to category selectivity generated from V1-initialized, stimulus-driven activity flow processes that are further shaped by two steps of VIS subsystem network interactions (Fig. 1C, Fig. 10).
Number of permutations in max-T nonparametric permutation testing: 100,000. d.f. = degrees of freedom (see Methods for details on outlier removal procedure). This analysis corresponds to category selectivity generated from V1-initialized, stimulus-driven activity flow processes that are further shaped by fully distributed network interactions (Fig. 1D, Fig. 11).
Supplementary figures
This assessment demonstrates that activity flow mapping of cross-cortex activations to each visual category exhibits high accuracy. (A) Left: Cross-participant average actual (empirical) task activations to body categories projected onto the MMP cortical atlas (Glasser et al. 2016). Right: Cross-participant average activity-flow-mapped task activations to body categories projected onto the MMP cortical atlas. The mapped and actual activations exhibited a high degree of overlap: r=0.89. (B-D) The same as in A, but for face (B), place (C), and tool categories (D) respectively. In all cases, accuracy was high, demonstrating that activity flow processes mapped predicted cross-cortex responses to each visual category of interest well.
To corroborate findings in the literature that each of the four functional complexes assessed in the present study exhibit significantly higher activations to images in their respective visual categories, we conducted standard t-test contrasts (discovery dataset, right hemisphere results depicted here; left hemisphere statistics reported in Table S1). (A) Box and swarm plots depicting activations to images of bodies and body parts exhibited by the EBA/FBA (dots = individual participant’s data). The actual activity is shown in coral and the activity-flow-mapped activity is shown in purple. Body versus non-body activations were contrasted with a one-tailed, paired samples t-test (hypothesizing that body activity was larger than non-body activity in the EBA/FBA), with an asterisk indicating a statistically significant difference (FDR adjusted p-values reported in the main text Results). The EBA/FBA exhibited statistically greater activations to body versus non-body images, benchmarking the prior work in the literature. (B) Same as in A, but for face versus non-face images and the FFA/pSTS functional complex. (C) Same as in A, but for place versus non-place images and the PPA/RSC functional complex. (D) Same as in A, but for tool versus non-tool images and the LOC functional complex. All results were corroborated by the replication dataset (N=176 in each; statistics given in main Results text).
Footnotes
References
- 1.
- 2.↵
- 3.↵
- 4.
- 5.
- 6.↵
- 7.
- 8.↵
- 9.
- 10.↵
- 11.↵
- 12.
- 13.↵
- 14.↵
- 15.
- 16.↵
- 17.
- 18.↵
- 19.↵
- 20.↵
- 21.↵
- 22.
- 23.
- 24.↵
- 25.↵
- 26.
- 27.
- 28.
- 29.↵
- 30.↵
- 31.↵
- 32.
- 33.↵
- 34.↵
- 35.↵
- 36.↵
- 37.↵
- 38.↵
- 39.↵
- 40.↵
- 41.
- 42.
- 43.↵
- 44.↵
- 45.
- 46.
- 47.↵
- 48.↵
- 49.
- 50.
- 51.↵
- 52.↵
- 53.↵
- 54.↵
- 55.
- 56.↵
- 57.↵
- 58.
- 59.↵
- 60.↵
- 61.
- 62.
- 63.
- 64.
- 65.
- 66.↵
- 67.
- 68.↵
- 69.
- 70.↵
- 71.↵
- 72.↵
- 73.↵
- 74.↵
- 75.↵
- 76.↵
- 77.
- 78.↵
- 79.↵
- 80.
- 81.
- 82.↵
- 83.↵
- 84.↵
- 85.↵
- 86.↵
- 87.
- 88.↵
- 89.↵
- 90.↵
- 91.
- 92.
- 93.↵
- 94.↵
- 95.↵
- 96.
- 97.↵
- 98.↵
- 99.↵
- 100.↵
- 101.↵
- 102.↵
- 103.
- 104.↵
- 105.
- 106.↵
- 107.↵
- 108.↵
- 109.↵
- 110.↵
- 111.↵
- 112.↵
- 113.↵
- 114.↵
- 115.↵
- 116.↵
- 117.↵
- 118.
- 119.↵
- 120.↵
- 121.↵
- 122.↵
- 123.↵
- 124.
- 125.
- 126.
- 127.↵
- 128.↵
- 129.↵
- 130.
- 131.↵
- 132.
- 133.
- 134.
- 135.↵
- 136.↵
- 137.↵
- 138.
- 139.↵
- 140.↵
- 141.↵
- 142.
- 143.
- 144.
- 145.↵
- 146.
- 147.↵
- 148.↵
- 149.↵
- 150.↵
- 151.
- 152.
- 153.
- 154.↵
- 155.↵
- 156.↵
- 157.
- 158.
- 159.
- 160.↵
- 161.
- 162.↵
- 163.↵
- 164.↵
- 165.
- 166.↵
- 167.↵
- 168.↵
