Comparison study of sixteen differential abundance testing methods using two large Parkinson disease gut microbiome datasets

Background When studying the relationship between the microbiome and a disease, a common question asked is what individual microbes are differentially abundant between a disease and healthy state. Numerous differential abundance (DA) testing methods exist and range from standard statistical tests to methods specifically designed for microbiome data. Comparison studies of DA testing methods have been performed, but none were performed on microbiome datasets collected for the study of real, complex disease. Due to this, we performed DA testing of microbial genera using 16 DA methods in two large, uniformly collected gut microbiome datasets on Parkinson disease (PD), and compared their results.

Results Pairwise concordances between methods ranged from 46%-99% similarity. Average pairwise concordance per dataset was 76%, and dropped to 62% when taking replication of signals across datasets into account. Certain methods consistently resulted in above average concordances (e.g. Kruskal-Wallis, ALDEx2, GLM with centered-log-ratio transform), while others consistently resulted in lower than average concordances (e.g. edgeR, fitZIG). Overall, ∼80% of genera tested were detected as differentially abundant by at least one method in each dataset. Requiring associations to replicate across datasets reduced significant signals by almost half. Further requirement of signals to be replicated by the majority of methods (≥8) yielded 19 associations. Only one genus (Agathobacter) was replicated by all methods. Use of hierarchical clustering revealed three groups of DA signatures that were (1) replicated by the majority of methods and included genera previously associated with PD, (2) replicated by few or no methods, and (3) replicated by a subset of methods and included rarer genera, all enriched in PD.

Conclusions Differential abundance tests yielded varied results. Using one method on one dataset may find true associations, but may also detect non-reproducible signals, adding to inconsistency in the literature. To help lower false positives, one might analyze data with two or more DA methods to gauge concordance, and use a built-in replication dataset to show reproducibility. This study corroborated previously reported microorganism associations in PD, and revealed a potential new group of microorganisms whose abundance is significantly elevated in PD, and might be worth pursuing in future investigations.