Towards Reproducible Brain-Wide Association Studies

Abstract
Magnetic resonance imaging (MRI) continues to drive many important neuroscientific advances. However, progress in uncovering reproducible associations between individual differences in brain structure/function and behavioral phenotypes (e.g., cognition, mental health) may have been undermined by typical neuroimaging sample sizes (median N=25)1,2. Leveraging the Adolescent Brain Cognitive Development (ABCD) Study3 (N=11,878), we estimated the effect sizes and reproducibility of these brain-wide associations studies (BWAS) as a function of sample size. The very largest, replicable brain-wide associations for univariate and multivariate methods were r=0.14 and r=0.34, respectively. In smaller samples, typical for brain-wide association studies (BWAS), irreproducible, inflated effect sizes were ubiquitous, no matter the method (univariate, multivariate). Until sample sizes started to approach consortium-levels, BWAS were underpowered and statistical errors assured. Multiple factors contribute to replication failures4–6; here, we show that the pairing of small brain-behavioral phenotype effect sizes with sampling variability is a key element in wide-spread BWAS replication failure. Brain-behavioral phenotype associations stabilize and become more reproducible with sample sizes of N⪆2,000. While investigator-initiated brain-behavior research continues to generate hypotheses and propel innovation, large consortia are needed to usher in a new era of reproducible human brain-wide association studies.
Competing Interest Statement
Nico Dosenbach and Damien Fair are co-founders of Nous Imaging
Subject Area
- Biochemistry (7766)
- Bioengineering (5666)
- Bioinformatics (21234)
- Biophysics (10552)
- Cancer Biology (8157)
- Cell Biology (11902)
- Clinical Trials (138)
- Developmental Biology (6736)
- Ecology (10387)
- Epidemiology (2065)
- Evolutionary Biology (13838)
- Genetics (9693)
- Genomics (13054)
- Immunology (8120)
- Microbiology (19932)
- Molecular Biology (7824)
- Neuroscience (42955)
- Paleontology (318)
- Pathology (1276)
- Pharmacology and Toxicology (2256)
- Physiology (3350)
- Plant Biology (7207)
- Synthetic Biology (1998)
- Systems Biology (5528)
- Zoology (1126)