Abstract
To acquire larger samples for answering complex questions in neuroscience, researchers have increasingly turned to multi-site neuroimaging studies. However, these studies are hindered by differences in images acquired across multiple scanners. These effects have been shown to bias comparison between scanners, mask biologically meaningful associations, and even introduce spurious associations. To address this, the field has focused on harmonizing data by removing scanner-related effects in the mean and variance of measurements. Contemporaneously with the increase in popularity of multi-center imaging, the use of multivariate pattern analysis (MVPA) has also become commonplace. These approaches have been shown to provide improved sensitivity, specificity, and power due to their modeling the joint relationship across measurements in the brain. In this work, we demonstrate that methods for removing scanner effects in mean and variance may not be sufficient for MVPA. This stems from the fact that such methods fail to address how correlations between measurements can vary across scanners. Data from the Alzheimer’s Disease Neuroimaging Initiative is used to show that considerable differences in covariance exist across scanners and that popular harmonization techniques do not address this issue. We also propose a novel methodology that harmonizes covariance of multivariate image measurements across scanners and demonstrate its improved performance in data harmonization.
Competing Interest Statement
The authors have declared no competing interest.
Footnotes
↵e Data used in preparation of this article were obtained from the Alzheimer’s Disease Neuroimaging Initiative (ADNI) database (adni.loni.usc.edu). As such, the investigators within the ADNI contributed to the design and implementation of ADNI and/or provided data but did not participate in analysis or writing of this report. A complete listing of ADNI investigators can be found at: http://adni.loni.usc.edu/wp-content/uploads/how_to_apply/ADNI_Acknowledgement_List.pdf
New work undertaken -To evaluate a more classical analysis technique, we conducted multivariate analysis of variance (MANOVA) as an alternative approach of multivariate pattern analysis (MVPA) and show that mitigation of scanner effects and preservation of biological variability are achieved. -We conduct signed-rank tests to evaluate differences in AUC in the MVPA experiment. -We designed a new simulation to evaluate how our proposed method performs in a scenario with simple covariance site effects for which the CovBat model is well specified. -In this new Simple Covariance Effects simulation, we compare sample site-specific covariance matrices to true covariance without site effects. These analyses show the degree of error introduced by covariance estimation and site confounding before and after harmonization. -We have conducted additional simulations to evaluate robustness of the MVPA results to sample size and number of features. Major changes to the manuscript -The introduction includes additional discussions and details on why harmonization using neither generative adversarial networks (GANs) nor a distance-based approach are both not directly comparable to our proposed method. -Metrics based on correlation have been modified to instead focus on covariance. -Section 2.1 has been clarified to better explain the two samples that are used throughout. -Section 2.1 includes a brief statement to explain why the harmonization method proposed in Zhou et al. (2018) cannot apply to the ADNI dataset used throughout the paper. -Section 2.3 now includes more detail as to the rationale and implementation of CovBat. -Section 2.5 has been reworked to describe two different simulation designs and multiple scenarios among those designs. -Section 3.2 and 3.3 include MANOVA results for scanner manufacturer, sex, and diagnosis. -Section 3.4 includes a figure to show that harmonization better recovers the true covariance structure in the Simple Covariance Effects simulation. -The Discussion section has been expanded to comment on all major results on the paper and acknowledge limitations of the methodology and study design. Minor edits and fixes -The description of the sensitivity analysis in Section 3.1 now matches the caption of Table 2. -Captions have been reduced in length and we no longer report mean and standard deviation, instead opting to report median and interquartile range to be more consistent with the markers shown on the plots.
