Contradictory Results

Investigating functional brain network integrity using a traditional and novel diagnostic system for neurodevelopmental disorders

Dina R. Dajani, Catherine A. Burrows, Paola Odriozola, Adriana Baez, Mary Beth Nebel, Stewart H. Mostofsky, Lucina Q. Uddin
doi: https://doi.org/10.1101/396317

Abstract

Background Current diagnostic systems for neurodevelopmental disorders do not have clear links to underlying neurobiology, limiting their utility in identifying targeted treatments for individuals. Several factors contribute to this issue, including the use of small samples in neuroimaging research and heterogeneity within diagnostic categories. Here, we aimed to investigate differences in functional brain network integrity between traditional diagnostic categories (autism spectrum disorder [ASD], attention-deficit/hyperactivity disorder [ADHD], typically developing [TD]) and carefully consider the impact of comorbid ASD and ADHD on functional brain network integrity in a large sample. We also assess the neurobiological validity of a novel, potential alternative nosology based on behavioral measures of executive function.

Method Five-minute resting-state fMRI data were obtained from 168 children (128 boys, 40 girls) with ASD, ADHD, comorbid ASD and ADHD, and TD children. Independent component analysis and dual regression were used to compute within- and between-network functional connectivity metrics at the individual level.

Results No significant group differences in within- nor between-network functional connectivity were observed between traditional diagnostic categories (ASD, ADHD, TD) even when stratified by comorbidity (ASD+ADHD, ASD, ADHD, TD). Similarly, subgroups classified by executive functioning levels showed no group differences.

Conclusions Using clinical diagnosis and behavioral measures of executive function, no group differences were observed among the categories examined. Therefore, we suggest that brain imaging metrics may more effectively define clinical subgroups than behavioral metrics, and may contribute to the establishment of a neurobiologically valid nosology for neurodevelopmental disorders.

Background

Autism spectrum disorder (ASD) and attention-deficit/hyperactivity disorder (ADHD) are behaviorally heterogeneous, prevalent neurodevelopmental disorders with few clear links between diagnostic criteria and specific neurobiological alterations1,2. These disorders exhibit shared deficits in executive function3 and associated functional brain alterations4,5, which may be exacerbated by the high rates of comorbidity between ASD and ADHD6. These challenges limit the utility of the diagnostic and statistical manual of mental disorders’ (DSM-57) criteria as predictors of etiology or treatment response8. The NIH has proposed the Research Domain Criteria (RDoC) framework, which instead suggests the use of neurocognitive constructs that are linked to underlying neurobiology to investigate mental health disorders9. Following these guidelines, a potential alternative nosology may be developed that is specifically tied to targeted treatments. In this study, we first assess the neurobiological validity of traditional diagnostic categories by evaluating differences in functional brain network integrity between children with ASD, ADHD and typically developing (TD) children. Comorbidity between ASD and ADHD is rarely considered in neuroimaging studies. Here, we explicitly examine brain network connectivity in children with comorbid ASD and ADHD separately from children with ASD (without ADHD), ADHD (without ASD), and TD children. Finally, we assess the neurobiological validity of a novel classification system based on behavioral measures of executive function as an alternative to the DSM-5 classification system.

The DSM-5 defines ASD according to behavioral symptoms of social/communication deficits and restricted and repetitive behaviors; ADHD is defined by either primarily inattentive or hyperactive symptoms, or a combination of both7. Although there exists a large body of work implicating functional brain network alterations in ASD and ADHD1, to date, no biomarkers have been identified to supplement diagnosis of these disorders. Here, we focus on functional connectivity as an important marker for dysfunction in these neurodevelopmental disorders.

Case-control studies comparing functional connectivity in ASD and ADHD to TD children have produced largely inconsistent results11,12, although studies of children with ADHD appear to be more consistent than studies of ASD14. Both hyperconnectivity15,16 and hypoconnectivity17,18 have been reported in children with ADHD and ASD. The neuroscience field is becoming increasingly aware of the lack of reproducible findings across studies, which may be the result of low sample sizes, inflated false-positive rates due to analytic choices, and heterogeneity inherent to groups of interest19. In response, the field is calling for a focus on replication studies using adequately powered samples20. Here, we aim to resolve the discrepancy in prior studies characterizing brain networks of children with ASD and ADHD by comparing functional network integrity between groups in a large sample of children.

Few neuroimaging studies have directly compared network functional connectivity of children with ASD and ADHD, resulting in inconclusive findings of both common and distinct network alterations across case-control studies4,21. Further, only two prior functional neuroimaging studies considered the impact of comorbidity on diagnostic group differences by examining an ASD with comorbid ADHD group distinct from non-comorbid ASD and ADHD groups4,5. These studies demonstrated that children with ASD and comorbid ADHD exhibit functional brain network abnormalities that resemble alterations in children with ASD (without ADHD) and ADHD (without ASD) plus unique abnormalities specific to comorbidity between ASD and ADHD4,5. These prior results suggest that some brain network findings may be nonspecific across ASD and ADHD, and some may only hold for subgroups within a disorder (e.g., a comorbid group). The current study addresses both of these concerns in the context of two highly prevalent neurodevelopmental disorders. The first aim of this study is to test differences in functional connectivity between children with ASD, ADHD, comorbid ASD and ADHD, and TD children.

Inconsistent findings and non-specificity of functional network alterations in ASD and ADHD may also be attributed to the heterogeneity characteristic of these disorders. Individual differences among children within a DSM diagnostic category are increasingly recognized across the biological psychiatry field, with recent calls to account for this variability in research studies1,22. One possible approach for parsing heterogeneity in these disorders is to define more homogeneous subgroups of children with ASD and ADHD. Based on DSM-5 criteria, there are no currently defined subgroups for ASD7. Although ADHD has three DSM-defined subgroups (Inattentive, Hyperactive/Impulsive, and Combined), these are currently inadequate to capture the full range of symptoms23 or to predict treatment response2. In addition to high levels of within- group variability, there is considerable overlap in both phenotypic and biological alterations in ASD and ADHD, one of the most striking similarities being common difficulties in executive function22. Shared alterations in structural10 and functional4 neural underpinnings of executive function across ASD and ADHD have likewise been reported. Importantly, Chantiluke et al. (2014) showed that children with “pure” ASD and ADHD exhibited little to no disorder-specific functional alterations during a temporal discounting task, whereas the comorbid group exhibited pronounced functional differences compared with all other groups5. This finding highlights the inadequacy of defining subgroups solely within a single disorder, and instead calls for the definition of subgroups that may cut across disorders.

To this end, we recently leveraged both theoretical (focusing on executive functions) and data-driven (using a subgrouping method called latent profile analysis) computational psychiatry approaches to develop a possible alternative categorization of neurodevelopmental disorders. We demonstrated that behavioral measures of executive function can be used to define subgroups of children across various diagnostic groups: ASD, ADHD, comorbid ASD and ADHD, and TD children24. Three subgroups emerged— “above average,” “average,” and “impaired” executive functions— which crossed traditional diagnostic boundaries. Here, we follow up these findings by assessing the neurobiological validity of the current DSM categories of ASD and ADHD, in addition to considering comorbidity. Moreover, we assess the neurobiological validity of an alternative nosology based on executive function subgroups (i.e., “above average,” “average,” and “impaired” subgroups). We operationalize neurobiological validity as the separability of groups based on within- and between-network functional connectivity of three major intrinsic connectivity networks (ICNs) important for cognition27: the frontoparietal network (FPN), salience network (SN), and DMN28.

We predicted that children with ADHD would exhibit reduced DMN connectivity and stronger DMN-FPN and DMN-SN coupling14 compared with TD children. We expected that children with ASD would exhibit hyperconnectivity within FPN, DMN, and SN15,29 compared with TD children. Inconsistent findings comparing ASD, ADHD, and ASD with comorbid ADHD groups precludes establishing well-founded a priori hypotheses specific to the comorbid ASD and ADHD group. Further, we anticipated that there would be a parametric increase in functional connectivity of the FPN and SN across executive function subgroups, such that the “impaired” subgroup would exhibit the lowest functional connectivity and the “above average” group would exhibit the highest functional connectivity30.

Methods and Materials

Participants

Participants aged 8 to 13 years (N=168) included a subset of children used in our previous study24. Written informed consent was obtained from all legal guardians and written assent was obtained from all children. All procedures were approved by the Institutional Review Board at the Johns Hopkins School of Medicine and all methods were carried out in accordance with the approved guidelines.

Diagnostic and IQ measures

Community diagnoses of ASD were confirmed with the Autism Diagnostic Observation Schedule (ADOS-G32 or ADOS-233, based on study enrollment date) and Autism Diagnostic Interview-Revised (ADI-R34). The Diagnostic Interview for Children and Adolescents IV35 was used to confirm community ADHD diagnoses, determine whether children with ASD had comorbid ADHD, and for exclusionary purposes. Community diagnoses of ADHD were also confirmed with the Conners’ Parent Rating Scales (CPRS-R:L36 or CPRS-337, based on study enrollment date) and the ADHD Rating Scale IV, Home version38 (Table 1). Executive functions used for the latent profile analysis were measured primarily with a parent-report (eight subscales of the BRIEF40), in addition to two laboratory measures, the Statue subscale of the NEPSY-II41 and the backward digit span of the WISC-IV. See supplementary information for more details.

View this table:
Table 1.

Diagnostic information.

Subsamples

To compare the integrity of ICNs between groups delineated by primary diagnosis (ASD, ADHD, TD), three equally sized diagnostic groups (N=129, three groups of n=43) were randomly selected from the larger sample of 168 participants in order to produce unbiased group ICA results (hereafter, ‘Diagnostic group sample’). These diagnostic groups were delineated by primary diagnosis; individuals may have had additional secondary diagnoses. For example, the “ASD” group included individuals with comorbid psychiatric diagnoses, including ADHD (n=26, 61%). Some children in the “ADHD” group had comorbid psychiatric diagnoses, but this did not include comorbid ASD, given that ASD is commonly considered a primary and not secondary diagnosis (see Table 1 for more details). Diagnostic groups did not differ in age, sex, handedness, FSIQ, or head motion (mean framewise displacement [FD42], translational and rotational motion, Table 2).

View this table:
Table 2.

‘Diagnostic group sample’ demographics

To address the issue of comorbidity in assessing diagnostic group differences, an additional group analysis was performed. Diagnostic groups were categorized as TD, ADHD, ASD (without ADHD), and ASD+ADHD (ASD with comorbid ADHD). Due to sample size limitations, group size for this analysis was limited to 22 (N=88, ‘Diagnostic comorbid sample’). These groups did not differ in age, sex, handedness, or head motion (Table S1). Due to sample size limitations, we were unable to match the diagnostic comorbid samples on FSIQ (p<.001).

A subset of the 168 participants eligible for this study was generated to ensure equal group sizes for each executive function (EF) subgroup (‘EF subgroup sample’, N=129, three groups of n=43). The ‘EF subgroup sample’ was representative of the Dajani et al. (2016) sample24 in EF scores, age, FSIQ and the distribution of diagnostic categories within each EF subgroup. EF subgroups did not differ in age, sex, handedness or mean head motion, but did differ on FSIQ (p<.001) and reached a near-significant difference in mean FD for the raw data (p=.06, Table S2). Following preprocessing, there were no group differences in mean FD (p=.36, Figure 2). The difference in FSIQ was expected given the subgroups were delineated based on EF, which tends to be highly correlated with IQ metrics43.

Figure 1.
Figure 1.

Fifteen-component ICA results for the ‘EF subgroup’ sample.

Components E (posterior DMN), F (L FPN), H (anterior DMN), I (R FPN), and K (SN) were used to assess group differences in network connectivity. One artifactual component emerged (component J).

Figure 2.
Figure 2.

Effect of rigorous motion correction.

Mean FD is in mm, displayed on raw data and data after preprocessing with ICA-AROMA

Data acquisition

Resting state fMRI (rs-fMRI) data were acquired for participants on a Phillips 3T scanner using an 8-channel head coil (TR=2.5s, flip angle=70°, sensitive encoding acceleration factor=2, 3mm slices, voxel size= 2.7×2.7×3 mm). Most participants had a 156-volume dataset, but a subset had a shorter acquisition time of 128 volumes (156 volumes: n=114, 128 volumes: n=15 in the ‘EF subgroup sample’). High-resolution T1-weighted scans were also acquired to facilitate registration of the functional image to standard space (TR=8.0ms, TE=3.7ms, 1mm isotropic voxels). Participants were asked to withhold stimulant medication (e.g., Adderall) the day before and on the day of MRI scanning, similar to prior neuroimaging studies comparing children with ASD and ADHD4,44. Non-stimulant medications were continued as prescribed (e.g., antidepressants, allergy medication, see Table S3 for detailed medication status information).

Experimental Design and Statistical Analysis

Preprocessing

Dual regression analyses employed in this study require all participants to have equal rs-fMRI scan lengths, therefore all participants’ rs-fMRI data were truncated to the shortest participant’s scan length by removing volumes at the end of the scan, resulting in all scans including 121 volumes (5 minutes of data). Standard preprocessing was conducted in FSL 5.0.9. In addition, a state-of-the-art ICA-based denoising procedure (ICA-AROMA45) was used to remove motion-related artifacts in native space (see supplemental information).

Group ICA

We ran a group ICA using the FSL MELODIC v3.14 toolbox (https://fsl.fmrib.ox.ac.uk/fsl/fslwiki/MELODIC) with temporal concatenation to identify common spatial patterns across participants. Five components of interest were manually identified from the group ICA by two of the authors (DRD and LQU): right FPN50, left FPN50, SN51, and anterior and posterior DMN50. To ensure unbiased networks were produced for each subsample and subsequent analyses, a separate group ICA model was run for each of the three subsamples. Networks were qualitatively highly similar across subsamples (Figure S1).

Dual Regression

Dual regression is a reliable technique which allows for the identification of group differences in the spatial and temporal features of ICNs common to the entire sample25,26. Using FSL’s dual regression command (https://fsl.fmrib.ox.ac.uk/fsl/fslwiki/DualRegression/UserGuide), individual-level spatial and temporal maps were constructed for each of the five components of interest generated from the group ICA. To test group differences in within- network connectivity, the normalized individual-level spatial maps were subjected to permutation testing using FSL’s randomise tool for each of the five components of interest (https://fsl.fmrib.ox.ac.uk/fsl/fslwiki/Randomise/UserGuide, 5000 permutations). To test for group differences in between-network connectivity, the FSLNets package was implemented in MATLAB (https://fsl.fmrib.ox.ac.uk/fsl/fslwiki/FSLNets). For each subsample, F-tests were conducted to ascertain the presence of group differences in within- network and between-network connectivity strength. Significance was determined using a threshold-free cluster enhancement of p<.05 (FWE-corrected) for both within- and between-network analyses, as in previous studies15,29.

Results

Effect of rigorous motion correction

A 2×1 repeated-measures ANOVA was used to assess whether motion (indexed by mean FD42) decreased as a function of preprocessing with ICA-AROMA by comparing mean FD for raw and preprocessed data. The ANOVA demonstrated a significant decrease in mean FD following preprocessing, F(1, 332)=25.36, p<.001 (raw: M=.22 SD=.14, preprocessed: M=.04 SD=.02, Figure 2).

Comparisons between diagnostic groups

No within- or between-network connectivity differences were found when comparing diagnostic groups (TD, ADHD, ASD; within- network: FWE-corrected p’s>.26, between-network: FWE-corrected p’s>.37). When comparing diagnostic groups and statistically controlling for the effects of EF subgroup, F-tests remained non-significant (within- network: FWE-corrected p’s>.20, between-network: FWE-corrected p’s>.11). Comparisons between comorbid diagnostic groups

When diagnostic groups were broken down according to comorbidity (TD, ADHD [without ASD], ASD [without ADHD], ASD+ADHD), F-tests revealed no group differences in within- network (FWE-corrected p’s>.30) or between-network connectivity (FWE-corrected p’s>.14).

Comparisons between EF subgroups

Group differences in network connectivity strength were assessed across five ICNs between children with “above average,” “average,” or “impaired” EF. F-tests revealed no significant differences between EF subgroups in within- network nor between-network connectivity (within- network: FWE-corrected p’s>.24, between-network: FWE-corrected p’s>.29). Likewise, after controlling for diagnostic status, no differences in connectivity emerged (within- network: FWE-corrected p’s>.25, between-network: FWE-corrected p’s>.47).

Discussion

With the recent exponential increase in computational power and growing awareness of the limitations of current psychiatric diagnostic systems, there have been numerous recent calls to leverage the strengths of computational psychiatry approaches to develop a more parsimonious and neurobiologically valid nosology of mental health disorders52,53. This study is the first to use ICA dual regression, a reliable data-driven approach to investigate differences in both within- and between-network functional connectivity between clinical groups, to assess the neurobiological separability of a traditional diagnostic classification system and a novel subgrouping system based on behavioral measures of executive function, while rigorously correcting for motion-related artifacts that are pervasive in pediatric psychiatric populations. We also carefully consider the impact of comorbidity between ASD and ADHD on brain network functional connectivity. Results indicate that the current DSM categories classify children into groups exhibiting negligible functional connectivity differences of major cognitive networks, suggesting limited neurobiological validity of current DSM categories. Contrary to our hypotheses, executive function subgroups displayed limited differences in functional connectivity of major cognitive networks, suggesting that a nosology based solely on a behavioral subgrouping system may not map onto differences in underlying neurobiology.

Comparisons between traditional diagnostic categories (i.e., ASD, ADHD, and TD), which are based on observable symptoms according to DSM criteria, showed no differences in functional connectivity across major cognitive networks. Several reasons may explain these null findings. First, it may be that these diagnostic categories do exhibit differences in functional connectivity, but findings may be method-specific and do not extend to dual regression ICA-based analyses. Second, it is possible that a moderating factor, such as sample-specific IQ, gender distribution or symptom severity, may affect whether differences in functional connectivity are observed. Third, previously published findings may simply represent false positives. Finally, it may be that this study represents a false negative.

Prior studies comparing TD children and children with ASD and ADHD using ICA-derived networks report network connectivity differences between diagnostic groups. Specifically, studies of individuals with ADHD report reduced segregation of DMN-FPN networks in children, adolescents, and adults54,55, but results from within- network connectivity studies are less coherent. For example, both hyperconnectivity16 and hypoconnectivity18 have been reported in ADHD for the FPN and the DMN. The ASD literature is similarly inconsistent, even when limited to studies using dual regression ICA. For example, there have been reports of both within- network hyperconnectivity15,29 and hypoconnectivity of the FPN17 and DMN17,56. Although results are not consistent across studies, they clearly demonstrate that positive results are possible when using ICA dual-regression to investigate network integrity in children with ASD and ADHD, ruling out the possibility that previously reported network connectivity differences in ASD and ADHD are method-specific. Discrepancies in results across studies may be due to individual differences in network connectivity within these traditional diagnostic categories, thus diminishing their neurobiological separability, and ultimately, their validity.

Heterogeneity within ASD and ADHD categories, characterized by a wide range of IQ, symptom severity, and the presence or absence of comorbid psychiatric disorders, may lead to inconsistent findings across studies, ultimately culminating in non-generalizable results. Our null findings when comparing diagnostic groups are in line with a recent review of classification studies of ASD and ADHD using neuroimaging techniques, which suggested that heterogeneity within and between disorders may be limiting researchers’ (and machines’) ability to distinguish disorders based on neurobiology57. Two factors may be contributing to the limited neurobiological separability of ASD and ADHD categories: high within- category heterogeneity and high between-category similarity. There is considerable evidence for the existence of subgroups within ASD58 and ADHD diagnostic categories59, which are often delineated with neuropsychological measures. Recent evidence has also emerged for neurobiological subgroups of ADHD based on network connectivity of the reward system60 and other large-scale brain networks61. In addition, there is substantial overlap in ASD and ADHD categories in symptomatology (e.g., social skills deficits), behavioral domains beyond symptoms (e.g., executive dysfunction), and genetic factors62 in addition to high rates of comorbidity between ASD and ADHD. In sum, these diagnostic categories are not distinct from one another based on numerous criteria. Evidence for both high within- group heterogeneity and between-group overlap reduces the validity of current DSM categories for ASD and ADHD, and thus may explain our lack of dissociability between disorders based on functional connectivity metrics.

Nonetheless, the results presented here contradict past findings of significant differences in functional connectivity between children with ASD, ADHD and TD children, which may be explained by the improved methodology used in this study. Past studies using ICA dual regression to compare children with ASD or ADHD to TD children used small sample sizes (n=20-25 per group)15,18,29,56,63,64. It is well established that low sample sizes contribute to reduced power to detect true results. However, it is less appreciated that positive results reported from underpowered studies also have a greater likelihood of being false19,65. Further, the exorbitant number of researcher degrees of freedom available in fMRI analyses, including choice of preprocessing pipeline, analysis method, group matching procedures, thresholding procedures, and multiple comparison correction lead to inflated false-positive rates19. Here, we nearly double the sample size of past studies and arrive at null results, which in combination with the fact that past results are inconsistent across studies, suggests that some previous results could be attributed to false positives65. In addition, we employ a rigorous motion correction procedure not used in any previous study of functional connectivity differences in ASD and ADHD. ICA-AROMA is a denoising approach that reduces the likelihood of group differences emerging solely due to differences in in-scanner motion. The strength of long-range functional connectivity is underestimated in cases of increased head motion66, which is rife in studies of youth with neurodevelopmental disorders46. Previous studies of within- network connectivity of the DMN in both children with ASD56,64 and ADHD14 that did not use ICA-AROMA may have been influenced by residual effects of motion. Taken together, these results suggest that findings of reduced long-range connectivity in ASD and ADHD may be a byproduct of increased motion artifacts in these clinical groups.

Few neuroimaging studies consider the impact of comorbid ASD and ADHD on case-control findings, and instead rely on the primary diagnosis of children to classify patients. This leads to inherent heterogeneity in clinical groups studied, likely leading to inconsistencies in findings across the neuroimaging literature. The current study is one of the first functional neuroimaging studies to directly compare children with ASD, ADHD, comorbid ASD and ADHD, and TD children, and the first to do so using ICA dual regression combined with ICA-AROMA. Here, we do not find statistically significant differences between ASD and ADHD groups, even when accounting for comorbidity.

Compared to the group difference tests for the traditional diagnostic groups (ASD, ADHD, TD) and executive function subgroups, the analysis accounting for comorbidity may have been underpowered, with only 22 participants per group. Therefore, it is possible that the lack of observed differences in functional connectivity in children with comorbid ASD and ADHD represent a false negative. Ideally, this analysis should be replicated in future studies with a larger sample size to confirm whether group differences in functional connectivity exist between children with comorbid ASD and ADHD and other clinical and typically developing populations.

As an alternative nosology to DSM-5 classifications, we tested whether there existed differences in functional connectivity between “above average,” “average” and “impaired” EF subgroups24. Contrary to our hypotheses, we observed no differences in functional connectivity between these subgroups, suggesting that EF-defined subgroups are not neurobiologically distinct based on networks important for cognition. There are multiple possible explanations for this null result. One possibility is that EF in children may be best assessed dimensionally rather than categorically67. The primary goal of this study was to assess the validity of the traditional categorical nosology (i.e., the DSM-5) and an alternative categorical nosology (i.e., EF subgroups). In characterizing neurodevelopmental and psychiatric disorders, recent studies have revealed the importance of testing whether categorical, dimensional, and hybrid categorical-dimensional models best describe psychiatric disorders67. Future studies would benefit from testing both symptom-based and executive function-based dimensional models to determine whether these may better characterize neurobiological differences between children. Additionally, our results may have been influenced by our choice of measures for EF subgrouping, which were primarily based on a parent-report of EF symptoms (i.e., the BRIEF40). We chose to focus on a parent-report of EF symptoms because of the strong psychometric properties of the measure and ease of obtaining the information in a clinical setting. Although using a parent-report of observable EF clearly has easily translatable implications for clinical practice, which is a current challenge for the field68, the choice to focus on a parent-report of behavior to characterize psychiatric disorders may be many steps removed from the underlying neurobiology. This leads to a rather simple explanation for these results – that behavior does not map one-to-one to underlying neurobiology.

There are numerous lines of evidence to suggest that brain-behavior relationships are not simply one-to-one69. Varying types of functional network miswirings across development may manifest as a singular phenotype70, suggesting that distinct brain abnormalities may appear behaviorally as the same neurodevelopmental disorder. Likewise, disparate genetic etiologies may lead to similar behavioral profiles10,71. Diagnostic categories should necessarily define neurobiologically homogeneous groups to allow for the development of targeted treatments specific to a neurobiological signature of the disorder. This suggests that advances in mental health research necessarily rely on characterizing underlying neurobiological mechanisms of pathophysiology, and that psychiatry may be fundamentally limited as long as assessments are limited to observable phenomena72.

Applying principles of computational psychiatry to clinical research has the potential to transform the mental health field from the current trial-and-error choice of treatments towards precision medicine. Current psychiatric diagnostic systems rely on observable behaviors to classify disease, with unknown links to underlying neurobiology52. Biomarkers, on the other hand, may provide information that may be able to stratify current diagnostic categories or replace symptom-based classification systems completely. Here, we aimed to leverage the strengths of computational psychiatry methodology to propose an alternative nosology based on behavioral measures of executive function, which we predicted would lead to neurobiologically distinct subgroups. Contrary to predictions, we found that EF subgroups could not be distinguished based on within- or between-network connectivity metrics of major cognitive networks, emphasizing the notion that nosologies reliant on behavioral data alone may not lead to discovery of neurobiologically distinct categories, limiting their utility in predicting prognosis and efficacious treatments.

Limitations

Although the present study had numerous strengths including a larger sample than previous similar studies, the results reported here should be considered in light of several limitations. It is difficult to draw strong conclusions from null results, as it is possible that they may be due to under-powered samples. Children in the clinical samples had various psychiatric comorbidities aside from ASD and ADHD (most commonly, oppositional defiant disorder [ODD]). While this is expected for children with ADHD given the high rates of comorbidity with ODD73, it may have introduced additional confounds that were not taken into account in this study. Future studies may consider the impact of different types and number of comorbid disorders on brain network integrity. One alternative explanation for our finding of no group differences in functional network integrity between EF subgroups is that another RDoC domain, such as social communication, may be better suited for developing an alternative nosology for children with neurodevelopmental disorders. We look forward to future studies exploring the efficacy of various domains and units of analysis outlined by the RDoC matrix.

Conclusions

We present findings that traditional diagnostic categories of ASD and ADHD could not be distinguished from one another or from TD children based on within- and between-network functional connectivity of three major cognitive networks: the frontoparietal, salience, and default-mode networks. Likewise, EF subgroups did not reflect neurobiologically distinct subgroups. Our study provides evidence that behavioral variables may be less informative than initially thought to create neurobiologically informed subgroups. The field of computational psychiatry is nascent53, and will therefore require the pursuit of new avenues and the continuous refinement and validation of new hypotheses. Based on the current findings, we suggest future work should employ data-driven approaches applied to neurobiological variables, such as functional connectivity metrics, to create parsimonious, biologically validated categories of neuropsychiatric diseases.

Disclosures

All authors reported no biomedical financial interests potential conflicts of interest.

Acknowledgements

We thank Jeanette Mumford for her contribution to the statistical modeling of the group difference tests. This work was supported by the National Institute of Mental Health (R01MH107549) to LQU. This work was also funded by Autism Speaks and NIH: R01NS048527, R01 MH085328, R01 MH078160, the Johns Hopkins University School of Medicine Institute for Clinical and Translational Research, and NIH/NCRR CTSA Program, UL1-RR025005, to S.H.M.

References

  1. 1.↵
    Ameis S. Heterogeneity Within and Between Autism Spectrum Disorder and Attention-Deficit/Hyperactivity Disorder: Challenge or Opportunity? AMA psychiatry. 2017.
  2. 2.↵
    Mueller A, Hong DS, Shepard S, Moore T. Linking ADHD to the Neural Circuitry of Attention. Trends in Cognitive Sciences. 2017.
  3. 3.↵
    Leno VC, Chandler S, White P, et al. Testing the specificity of executive functioning impairments in adolescents with ADHD, ODD/CD and ASD. European child & adolescent psychiatry. 2017:1–10.
  4. 4.↵
    Di Martino A, Zuo XN, Kelly C, et al. Shared and distinct intrinsic functional network centrality in autism and attention-deficit/hyperactivity disorder. Biol Psychiatry. 2013;74(8):623–632.
    OpenUrlCrossRefPubMedWeb of Science
  5. 5.↵
    Chantiluke K, Christakou A, Murphy CM, et al. Disorder-specific functional abnormalities during temporal discounting in youth with Attention Deficit Hyperactivity Disorder (ADHD), Autism and comorbid ADHD and Autism. Psychiatry research. 2014;223(2):113–120.
    OpenUrlCrossRefPubMed
  6. 6.↵
    Leitner Y. The co-occurrence of autism and attention deficit hyperactivity disorder in children - what do we know? Fronteirs in Human Neuroscience. 2014;8:268.
    OpenUrl
  7. 7.↵
    American Psychiatric Association. DSM 5. American Psychiatric Association; 2013.
  8. 8.↵
    Clark L, Cuthbert B, Lewis-Fernández R, Narrow W, Reed G. ICD-11, DSM-5, and RDoC: Three approaches to understanding and classifying mental disorder. Psychological Science in the Public Interest. 2017;18(2):xxx-xxx.
    OpenUrl
  9. 9.↵
    Insel T, Cuthbert B, Garvey M, et al. Research domain criteria (RDoC): toward a new classification framework for research on mental disorders. American Journal of Psychiatry. 2010;167(7):748–751.
    OpenUrlCrossRefPubMedWeb of Science
  10. 10.↵
    Dougherty CC, Evans DW, Myers SM, Moore GJ, Michael AM. A Comparison of Structural Brain Imaging Findings in Autism Spectrum Disorder and Attention-Deficit Hyperactivity Disorder. Neuropsychology review. 2016;26(1):25–43.
    OpenUrlPubMed
  11. 11.↵
    Mash LE, Reiter MA, Linke AC, Townsend J, Muller RA. Multimodal approaches to functional connectivity in autism spectrum disorders: An integrative perspective. Dev Neurobiol. 2017.
  12. 12.↵
    Hull JV, Jacokes ZJ, Torgerson CM, Irimia A, Van Horn JD. Resting-State Functional Connectivity in Autism Spectrum Disorders: A Review. Frontiers in psychiatry. 2016;7:205.
    OpenUrl
  13. 13.
    Mahone EM, Denckla MB. Attention-Deficit/Hyperactivity Disorder: A Historical Neuropsychological Perspective. Journal of the International Neuropsychological Society: JINS. 2017;23(9-10):916–929.
    OpenUrl
  14. 14.↵
    Castellanos FX, Aoki Y. Intrinsic Functional Connectivity in Attention-Deficit/Hyperactivity Disorder: A Science in Development. Biol Psychiatry Cogn Neurosci Neuroimaging. 2016;1(3):253–261.
    OpenUrl
  15. 15.↵
    Uddin LQ, Supekar K, Lynch CJ, et al. Salience network-based classification and prediction of symptom severity in children with autism. JAMA psychiatry. 2013;70(8):869–879.
    OpenUrl
  16. 16.↵
    de Celis Alonso B, Hidalgo Tobon S, Dies Suarez P, Garcia Flores J, de Celis Carrillo B, Barragan Perez E. A multi-methodological MR resting state network analysis to assess the changes in brain physiology of children with ADHD. PLoS One. 2014;9(6):e99119.
    OpenUrlCrossRefPubMed
  17. 17.↵
    Bos DJ, van Raalten TR, Oranje B, et al. Developmental differences in higher-order resting-state networks in Autism Spectrum Disorder. Neuroimage Clin. 2014;4:820–827.
    OpenUrl
  18. 18.↵
    Yoo JH, Kim D, Choi J, Jeong B. Treatment effect of methylphenidate on intrinsic functional brain network in medication-naïve ADHD children: A multivariate analysis. Brain Imaging and Behavior. 2017:1–14.
  19. 19.↵
    Poldrack RA, Baker CI, Durnez J, et al. Scanning the horizon: towards transparent and reproducible neuroimaging research. Nature reviews Neuroscience. 2017;18(2):115–126.
    OpenUrlCrossRefPubMed
  20. 20.↵
    Fletcher PC, Grafton ST. Repeat after me: replication in clinical neuroimaging is critical. NeuroImage: Clinical. 2013;2:247.
    OpenUrl
  21. 21.↵
    Christakou A, Murphy CM, Chantiluke K, et al. Disorder-specific functional abnormalities during sustained attention in youth with Attention Deficit Hyperactivity Disorder (ADHD) and with autism. Molecular psychiatry. 2013;18(2):236–244.
    OpenUrlCrossRefPubMedWeb of Science
  22. 22.↵
    Corbett BA, Constantine LJ, Hendren R, Rocke D, Ozonoff S. Examining executive functioning in children with autism spectrum disorder, attention deficit hyperactivity disorder and typical development. Psychiatry research. 2009;166(2- 3):210–222.
    OpenUrlCrossRefPubMedWeb of Science
  23. 23.↵
    Lee S, Burns GL, Snell J, McBurnett K. Validity of the Sluggish Cognitive Tempo Symptom Dimension in Children: Sluggish Cognitive Tempo and ADHD- Inattention as Distinct Symptom Dimensions. Journal of Abnormal Child Psychology. 2014;42(1):7–19.
    OpenUrlCrossRefPubMed
  24. 24.↵
    Dajani DR, Llabre MM, Nebel MB, Mostofsky SH, Uddin LQ. Heterogeneity of executive functions among comorbid neurodevelopmental disorders. Scientific Reports. 2016;6:36566.
    OpenUrl
  25. 25.↵
    Filippini N, MacIntosh BJ, Hough MG, et al. Distinct patterns of brain activity in young carriers of the APOE-ε4 allele. Proceedings of the National Academy of Sciences. 2009;106(17):7209–7214.
    OpenUrlAbstract/FREE Full Text
  26. 26.↵
    Zuo XN, Kelly C, Adelstein JS, Klein DF, Castellanos FX, Milham MP. Reliable intrinsic connectivity networks: test-retest evaluation using ICA and dual regression approach. Neuroimage. 2010;49(3):2163–2177.
    OpenUrlCrossRefPubMedWeb of Science
  27. 27.↵
    Menon V, Uddin LQ. Saliency, switching, attention and control: a network model of insula function. Brain Struct Funct. 2010;214(5-6):655–667.
    OpenUrlCrossRefPubMedWeb of Science
  28. 28.↵
    Uddin LQ. Salience processing and insular cortical function and dysfunction. Nature Reviews Neuroscience. 2015;16(1):55–61.
    OpenUrlCrossRefPubMed
  29. 29.↵
    Nomi JS, Uddin LQ. Developmental changes in large-scale network connectivity in autism. NeuroImage: Clinical. 2015.
  30. 30.↵
    Reineberg AE, Andrews-Hanna JR, Depue BE, Friedman NP, Banich MT. Resting-state networks predict individual differences in common and specific aspects of executive function. NeuroImage. 2015;104(Supplement C):69–78.
    OpenUrlCrossRefPubMed
  31. 31.
    Satterthwaite TD, Wolf DH, Erus G, et al. Functional maturation of the executive system during adolescence. Journal of Neuroscience. 2013;33(41):16249–16261.
    OpenUrlAbstract/FREE Full Text
  32. 32.↵
    Lord C, Risi S, Lambrecht L, et al. The Autism Diagnostic Observation Schedule—Generic: A standard measure of social and communication deficits associated with the spectrum of autism. Journal of autism and developmental disorders. 2000;30(3):205–223.
    OpenUrlCrossRefPubMedWeb of Science
  33. 33.↵
    Lord C, Rutter M, DiLavore PC, Risi S, Gotham K, Bishop SL. Autism diagnostic observation schedule: ADOS-2. Western Psychological Services Los Angeles, CA; 2012.
  34. 34.↵
    Rutter M, Le Couteur A, Lord C, Faggioli R. ADI-R: Autism diagnostic interview-- revised: Manual. OS, Organizzazioni speciali; 2005.
  35. 35.↵
    Reich W, Welner Z, Herjanic B. Diagnostic interview for children and adolescents-IV (DICA-IV) Multi-Health Systems. Toronto, Canada. 1997.
  36. 36.↵
    Conners CK. Conners’ Rating Scales--revised: User’s Manual. Multi-Health Systems, Incorporated; 1997.
  37. 37.↵
    Conners CK. Conners 3rd edition. Multi-Health Systems Toronto, Ontario, Canada; 2008.
  38. 38.↵
    DuPaul GJ, Power TJ, Anastopoulos AD, Reid R. ADHD Rating Scale-IV: Checklists, norms, and clinical interpretation. Vol 25: Guilford Press New York; 1998.
  39. 39.↵
    Wechsler D. Wechsler Intelligence Scale for Children-WISC-IV. Psychological Corporation; 2003.
  40. 40.↵
    Gioia GA, Isquith PK, Guy SC, Kenworthy L. Behavior Rating Inventory of Executive Function: BRIEF. Psychological Assessment Resources Odessa, FL; 2000.
  41. 41.↵
    Korkman M, Kirk U, Kemp S. NEPSY-II: administration manual. San Antonio, TX: The Psychological Corporation. 2007.
  42. 42.↵
    Power JD, Barnes KA, Snyder AZ, Schlaggar BL, Petersen SE. Spurious but systematic correlations in functional connectivity MRI networks arise from subject motion. Neuroimage. 2012;59(3):2142–2154.
    OpenUrlCrossRefPubMedWeb of Science
  43. 43.↵
    Arffa S. The relationship of intelligence to executive function and non-executive function measures in a sample of average, above average, and gifted youth. Archives of Clinical Neuropsychology. 2007;22(8):969–978.
    OpenUrlCrossRefPubMedWeb of Science
  44. 44.↵
    Dennis EL, Jahanshad N, McMahon KL, et al. Development of insula connectivity between ages 12 and 30 revealed by high angular resolution diffusion imaging. Hum Brain Mapp. 2014;35(4):1790–1800.
    OpenUrlCrossRefPubMed
  45. 45.↵
    Pruim RH, Mennes M, Buitelaar JK, Beckmann CF. Evaluation of ICA-AROMA and alternative strategies for motion artifact removal in resting state fMRI. Neuroimage. 2015;112:278–287.
    OpenUrlCrossRefPubMed
  46. 46.↵
    Yerys BE, Jankowski KF, Shook D, et al. The fMRI success rate of children and adolescents: typical development, epilepsy, attention deficit/hyperactivity disorder, and autism spectrum disorders. Human brain mapping. 2009;30(10):3426–3435.
    OpenUrlCrossRefPubMedWeb of Science
  47. 47.↵
    Pruim RH, Mennes M, van Rooij D, Llera A, Buitelaar JK, Beckmann CF. ICA- AROMA: A robust ICA-based strategy for removing motion artifacts from fMRI data. Neuroimage. 2015;112:267–277.
    OpenUrlCrossRefPubMed
  48. 48.↵
    Satterthwaite TD, Wolf DH, Loughead J, et al. Impact of in-scanner head motion on multiple measures of functional connectivity: Relevance for studies of neurodevelopment in youth. NeuroImage. 2012;60(1):623–632.
    OpenUrlCrossRefPubMedWeb of Science
  49. 49.↵
    Salimi-Khorshidi G, Douaud G, Beckmann CF, Glasser MF, Griffanti L, Smith SM. Automatic denoising of functional MRI data: combining independent component analysis and hierarchical fusion of classifiers. Neuroimage. 2014;90:449–468.
    OpenUrlCrossRefPubMedWeb of Science
  50. 50.↵
    Smith SM, Fox PT, Miller KL, et al. Correspondence of the brain’s functional architecture during activation and rest. Proceedings of the National Academy of Sciences of the United States of America. 2009;106(31):13040–13045.
    OpenUrlAbstract/FREE Full Text
  51. 51.↵
    Seeley WW, Menon V, Schatzberg AF, et al. Dissociable intrinsic connectivity networks for salience processing and executive control. The Journal of neuroscience: the official journal of the Society for Neuroscience. 2007;27(9):2349–2356.
    OpenUrlAbstract/FREE Full Text
  52. 52.↵
    Huys QJ, Maia TV, Frank MJ. Computational psychiatry as a bridge from neuroscience to clinical applications. Nat Neurosci. 2016;19(3):404–413.
    OpenUrlCrossRefPubMed
  53. 53.↵
    Chekroud AM, Lane CE, Ross DA. Computational Psychiatry: Embracing Uncertainty and Focusing on Individuals, Not Averages. Biol Psychiatry. 2017;82(6):e45–e47.
    OpenUrl
  54. 54.↵
    Kessler D, Angstadt M, Welsh RC, Sripada C. Modality-spanning deficits in attention-deficit/hyperactivity disorder in functional networks, gray matter, and white matter. The Journal of neuroscience: the official journal of the Society for Neuroscience. 2014;34(50):16555–16566.
    OpenUrlAbstract/FREE Full Text
  55. 55.↵
    Sudre G, Szekely E, Sharp W, Kasparek S, Shaw P. Multimodal mapping of the brain’s functional connectivity and the adult outcome of attention deficit hyperactivity disorder. Proceedings of the National Academy of Sciences of the United States of America. 2017;114(44):11787–11792.
    OpenUrlAbstract/FREE Full Text
  56. 56.↵
    Washington SD, Gordon EM, Brar J, et al. Dysmaturation of the default mode network in autism. Hum Brain Mapp. 2014;35(4):1284–1296.
    OpenUrlCrossRefPubMedWeb of Science
  57. 57.↵
    Uddin L, Dajani D, Voorhies W, Bednarz H, Kana R. Progress and roadblocks in the search for brain-based biomarkers of autism and attention-deficit/hyperactivity disorder. Translational psychiatry. 2017;7(8):e1218.
    OpenUrl
  58. 58.↵
    Feczko E, Balba N, Miranda-Dominguez O, et al. Subtyping cognitive profiles in Autism Spectrum Disorder using a random forest algorithm. Neuroimage. 2017.
  59. 59.↵
    Fair DA, Bathula D, Nikolas MA, Nigg JT. Distinct neuropsychological subgroups in typically developing youth inform heterogeneity in children with ADHD. Proceedings of the National Academy of Sciences. 2012;109(17):6769–6774.
    OpenUrlAbstract/FREE Full Text
  60. 60.↵
    Costa Dias TG, Iyer SP, Carpenter SD, et al. Characterizing heterogeneity in children with and without ADHD based on reward system connectivity. Dev Cogn Neurosci. 2015;11:155–174.
    OpenUrl
  61. 61.↵
    Gates KM, Molenaar PC, Iyer SP, Nigg JT, Fair DA. Organizing heterogeneous samples using community detection of GIMME-derived resting state functional networks. PloS one. 2014;9(3):e91322.
    OpenUrlCrossRefPubMed
  62. 62.↵
    Rommelse NN, Franke B, Geurts HM, Hartman CA, Buitelaar JK. Shared heritability of attention-deficit/hyperactivity disorder and autism spectrum disorder. Eur Child Adolesc Psychiatry. 2010;19(3):281–295.
    OpenUrlCrossRefPubMedWeb of Science
  63. 63.↵
    Choi J, Jeong B, Lee SW, Go HJ. Aberrant development of functional connectivity among resting state-related functional networks in medication-naive ADHD children. PLoS One. 2013;8(12):e83516.
    OpenUrl
  64. 64.↵
    Starck T, Nikkinen J, Rahko J, et al. Resting state fMRI reveals a default mode dissociation between retrosplenial and medial prefrontal subnetworks in ASD despite motion scrubbing. Frontiers in human neuroscience. 2013;7:802.
    OpenUrl
  65. 65.↵
    Ioannidis JP. Why most published research findings are false. PLoS Med. 2005;2(8):e124.
    OpenUrlCrossRefPubMed
  66. 66.↵
    Van Dijk KRA, Sabuncu MR, Buckner RL. The influence of head motion on intrinsic functional connectivity MRI. NeuroImage. 2012;59(1):431–438.
    OpenUrlCrossRefPubMedWeb of Science
  67. 67.↵
    Barch DM. Biotypes: Promise and Pitfalls. Biol Psychiatry. 2017;82(1):2–3.
    OpenUrl
  68. 68.↵
    Paulus MP, Huys QJ, Maia TV. A Roadmap for the Development of Applied Computational Psychiatry. Biol Psychiatry Cogn Neurosci Neuroimaging. 2016;1(5):386–392.
    OpenUrl
  69. 69.↵
    Pessoa L. Understanding brain networks and brain organization. Physics of life reviews. 2014;11(3):400–435.
    OpenUrl
  70. 70.↵
    Di Martino A, Fair DA, Kelly C, et al. Unraveling the miswired connectome: a developmental perspective. Neuron. 2014;83(6):1335–1353.
    OpenUrl
  71. 71.↵
    Pelphrey K. Charting a Course for Autism Biomarkers. Biol Psychiatry. 2017;82(3):155–156.
    OpenUrlCrossRefPubMed
  72. 72.↵
    Pine DS. Clinical Advances From a Computational Approach to Anxiety. Biol Psychiatry. 2017;82(6):385–387.
    OpenUrl
  73. 73.↵
    Angold A, Costello EJ, Erkanli A. Comorbidity. Journal of child psychology and psychiatry. 1999;40(1):57–87.
    OpenUrlCrossRefPubMedWeb of Science

References

  1. 1.
    Dajani DR, Llabre MM, Nebel MB, Mostofsky SH, Uddin LQ. Heterogeneity of executive functions among comorbid neurodevelopmental disorders. Scientific Reports. 2016;6:36566.
    OpenUrl
  2. 2.
    Lord C, Risi S, Lambrecht L, et al. The Autism Diagnostic Observation Schedule—Generic: A standard measure of social and communication deficits associated with the spectrum of autism. Journal of autism and developmental disorders. 2000;30(3):205–223.
    OpenUrlCrossRefPubMedWeb of Science
  3. 3.
    Lord C, Rutter M, DiLavore PC, Risi S, Gotham K, Bishop SL. Autism diagnostic observation schedule: ADOS-2. Western Psychological Services Los Angeles, CA; 2012.
  4. 4.
    Rutter M, Le Couteur A, Lord C, Faggioli R. ADI-R: Autism diagnostic interview-- revised: Manual. OS, Organizzazioni speciali; 2005.
  5. 5.
    Conners CK. Conners’ Rating Scales--revised: User’s Manual. Multi-Health Systems, Incorporated; 1997.
  6. 6.
    Conners CK. Conners 3rd edition. Multi-Health Systems Toronto, Ontario, Canada; 2008.
  7. 7.
    DuPaul GJ, Power TJ, Anastopoulos AD, Reid R. ADHD Rating Scale-IV: Checklists, norms, and clinical interpretation. Vol 25: Guilford Press New York; 1998.
  8. 8.
    Reich W, Welner Z, Herjanic B. Diagnostic interview for children and adolescents-IV (DICA-IV) Multi-Health Systems. Toronto, Canada. 1997.
  9. 9.
    Pruim RH, Mennes M, van Rooij D, Llera A, Buitelaar JK, Beckmann CF. ICA- AROMA: A robust ICA-based strategy for removing motion artifacts from fMRI data. Neuroimage. 2015;112:267–277.
    OpenUrlCrossRefPubMed
Back to top
Posted August 22, 2018.
Download PDF
Email
Investigating functional brain network integrity using a traditional and novel diagnostic system for neurodevelopmental disorders
Dina R. Dajani, Catherine A. Burrows, Paola Odriozola, Adriana Baez, Mary Beth Nebel, Stewart H. Mostofsky, Lucina Q. Uddin
bioRxiv 396317; doi: https://doi.org/10.1101/396317
Twitter logo Facebook logo LinkedIn logo Mendeley logo
Citation Tools

Subject Area