TY - JOUR T1 - Numerical Instabilities in Analytical Pipelines Lead to Large and Meaningful Variability in Brain Networks JF - bioRxiv DO - 10.1101/2020.10.15.341495 SP - 2020.10.15.341495 AU - Gregory Kiar AU - Yohan Chatelain AU - Pablo de Oliveira Castro AU - Eric Petit AU - Ariel Rokem AU - Gaël Varoquaux AU - Bratislav Misic AU - Alan C. Evans AU - Tristan Glatard Y1 - 2020/01/01 UR - http://biorxiv.org/content/early/2020/10/29/2020.10.15.341495.abstract N2 - The analysis of brain-imaging data requires complex and often non-linear transformations to support findings on brain function or pathologies. And yet, recent work has shown that variability in the choices that one makes when analyzing data can lead to quantitatively and qualitatively different results, endangering the trust in conclusions1–3. Even within a given method or analytical technique, numerical instabilities could compromise findings4–7. We instrumented a structural-connectome estimation pipeline with Monte Carlo Arithmetic8,9, a technique to introduce random noise in floating-point computations, and evaluated the stability of the derived connectomes, their features10,11, and the impact on a downstream analysis12,13. The stability of results was found to be highly dependent upon which features of the connectomes were evaluated, and ranged from perfectly stable (i.e. no observed variability across executions) to highly unstable (i.e. the results contained no trustworthy significant information). While the extreme range and variability in results presented here could severely hamper our understanding of brain organization in brain-imaging studies, it also leads to an increase in the reliability of datasets. This paper highlights the potential of leveraging the induced variance in estimates of brain connectivity to reduce the bias in networks alongside increasing the robustness of their applications in the detection or classification of individual differences. This paper demonstrates that stability evaluations are necessary for understanding error and bias inherent to scientific computing, including but not limited to neuroimaging, and that they should be a component of typical analytical workflows.Competing Interest StatementThe authors have declared no competing interest. ER -