PT  - JOURNAL ARTICLE
AU  - Mathieu Seppey
AU  - Mosè Manni
AU  - Evgeny M. Zdobnov
TI  - LEMMI: A Live Evaluation of Computational Methods for Metagenome Investigation
AID  - 10.1101/507731
DP  - 2018 Jan 01
TA  - bioRxiv
PG  - 507731
4099  - http://biorxiv.org/content/early/2018/12/28/507731.short
4100  - http://biorxiv.org/content/early/2018/12/28/507731.full
AB  - Metagenomics promotes our understanding of microbial communities. The methodology profoundly relies on computational interpretation of the sequencing reads in the light of our evolving understanding of microbial taxonomy. The fast growing volumes of such readouts and the reference databases prompted development of a multitude of computational procedures over recent years. Only a subset of these has appeared in independent benchmarking assessments, which are also quickly becoming obsolete. This is obscuring an informed choice of a method for biologists as well as the impact of certain innovations for method developers.Here we present the LEMMI benchmarking platform (https://lemmi.ezlab.org) enabling the continuous integration of taxonomic profilers and binners, their assessment on a variety of data, as well as dynamic presentation of their ranking according to alternative objectives of required analyses or technical limitations. The platform is container-based and exploits the ability of many methods to construct a reference database on demand to compare them under highly controlled conditions and with identical inputs. Curated references can also be provided by method developers, enabling the advantages of particular databases to be highlighted. LEMMI creates a link between method developers and their users. It provides automated and unbiased benchmarking that are valuable to both. Moreover, the standardised and easy-to-use containers created by us or developers can be downloaded by metagenomic tool users, who will get exactly the same software as it appears in evaluation.