PT  - JOURNAL ARTICLE
AU  - Alexander Sczyrba
AU  - Peter Hofmann
AU  - Peter Belmann
AU  - David Koslicki
AU  - Stefan Janssen
AU  - Johannes Dröge
AU  - Ivan Gregor
AU  - Stephan Majda
AU  - Jessika Fiedler
AU  - Eik Dahms
AU  - Andreas Bremges
AU  - Adrian Fritz
AU  - Ruben Garrido-Oter
AU  - Tue Sparholt Jørgensen
AU  - Nicole Shapiro
AU  - Philip D. Blood
AU  - Alexey Gurevich
AU  - Yang Bai
AU  - Dmitrij Turaev
AU  - Matthew Z. DeMaere
AU  - Rayan Chikhi
AU  - Niranjan Nagarajan
AU  - Christopher Quince
AU  - Fernando Meyer
AU  - Monika Balvoit
AU  - Lars Hestbjerg Hansen
AU  - Søren J. Sørensen
AU  - Burton K. H. Chia
AU  - Bertrand Denis
AU  - Jeff L. Froula
AU  - Zhong Wang
AU  - Robert Egan
AU  - Dongwan Don Kang
AU  - Jeffrey J. Cook
AU  - Charles Deltel
AU  - Michael Beckstette
AU  - Claire Lemaitre
AU  - Pierre Peterlongo
AU  - Guillaume Rizk
AU  - Dominique Lavenier
AU  - Yu-Wei Wu
AU  - Steven W. Singer
AU  - Chirag Jain
AU  - Marc Strous
AU  - Heiner Klingenberg
AU  - Peter Meinicke
AU  - Michael Barton
AU  - Thomas Lingner
AU  - Hsin-Hung Lin
AU  - Yu-Chieh Liao
AU  - Genivaldo Gueiros Z. Silva
AU  - Daniel A. Cuevas
AU  - Robert A. Edwards
AU  - Surya Saha
AU  - Vitor C. Piro
AU  - Bernhard Y. Renard
AU  - Mihai Pop
AU  - Hans-Peter Klenk
AU  - Markus Göker
AU  - Nikos C. Kyrpides
AU  - Tanja Woyke
AU  - Julia A. Vorholt
AU  - Paul Schulze-Lefert
AU  - Edward M. Rubin
AU  - Aaron E. Darling
AU  - Thomas Rattei
AU  - Alice C. McHardy
TI  - Critical Assessment of Metagenome Interpretation – a benchmark of computational metagenomics software
AID  - 10.1101/099127
DP  - 2017 Jan 01
TA  - bioRxiv
PG  - 099127
4099  - http://biorxiv.org/content/early/2017/06/12/099127.short
4100  - http://biorxiv.org/content/early/2017/06/12/099127.full
AB  - In metagenome analysis, computational methods for assembly, taxonomic profiling and binning are key components facilitating downstream biological data interpretation. However, a lack of consensus about benchmarking datasets and evaluation metrics complicates proper performance assessment. The Critical Assessment of Metagenome Interpretation (CAMI) challenge has engaged the global developer community to benchmark their programs on datasets of unprecedented complexity and realism. Benchmark metagenomes were generated from ~700 newly sequenced microorganisms and ~600 novel viruses and plasmids, including genomes with varying degrees of relatedness to each other and to publicly available ones and representing common experimental setups. Across all datasets, assembly and genome binning programs performed well for species represented by individual genomes, while performance was substantially affected by the presence of related strains. Taxonomic profiling and binning programs were proficient at high taxonomic ranks, with a notable performance decrease below the family level. Parameter settings substantially impacted performances, underscoring the importance of program reproducibility. While highlighting current challenges in computational metagenomics, the CAMI results provide a roadmap for software selection to answer specific research questions.