Skip to main content
bioRxiv
  • Home
  • About
  • Submit
  • ALERTS / RSS
Advanced Search
New Results

Best Practices for Benchmarking Germline Small Variant Calls in Human Genomes

Peter Krusche, Len Trigg, Paul C. Boutros, Christopher E. Mason, Francisco M. De La Vega, Benjamin L. Moore, Mar Gonzalez-Porta, Michael A. Eberle, Zivana Tezak, Samir Labadibi, Rebecca Truty, George Asimenos, Birgit Funke, Mark Fleharty, Brad A. Chapman, Marc Salit, Justin M Zook, and the Global Alliance for Genomics and Health Benchmarking Team
doi: https://doi.org/10.1101/270157
Peter Krusche
1Illumina Cambridge Ltd, Chesterford Research Park, Little Chesterford, Saffron Walden, Essex, CB10 1XL, UK
  • Find this author on Google Scholar
  • Find this author on PubMed
  • Search for this author on this site
Len Trigg
2Real Time Genomics, Level 1, South Bloc, 19 Knox St. Hamilton. New Zealand
  • Find this author on Google Scholar
  • Find this author on PubMed
  • Search for this author on this site
Paul C. Boutros
3Ontario Institute for Cancer Research, Toronto, Ontario, Canada
  • Find this author on Google Scholar
  • Find this author on PubMed
  • Search for this author on this site
Christopher E. Mason
4Weill Cornell Medicine, New York, NY
  • Find this author on Google Scholar
  • Find this author on PubMed
  • Search for this author on this site
Francisco M. De La Vega
5Department of Biomedical Data Science, Stanford University School of Medicine, Stanford, CA
  • Find this author on Google Scholar
  • Find this author on PubMed
  • Search for this author on this site
Benjamin L. Moore
1Illumina Cambridge Ltd, Chesterford Research Park, Little Chesterford, Saffron Walden, Essex, CB10 1XL, UK
  • Find this author on Google Scholar
  • Find this author on PubMed
  • Search for this author on this site
Mar Gonzalez-Porta
1Illumina Cambridge Ltd, Chesterford Research Park, Little Chesterford, Saffron Walden, Essex, CB10 1XL, UK
  • Find this author on Google Scholar
  • Find this author on PubMed
  • Search for this author on this site
Michael A. Eberle
6Illumina Inc., 5200 Illumina Way, San Diego, CA 92122
  • Find this author on Google Scholar
  • Find this author on PubMed
  • Search for this author on this site
Zivana Tezak
7Center for Devices and Radiological Health, FDA, Silver Spring, MD
  • Find this author on Google Scholar
  • Find this author on PubMed
  • Search for this author on this site
Samir Labadibi
8Office of Health Informatics, Office of the Commissioner, FDA, Silver Spring, MD
  • Find this author on Google Scholar
  • Find this author on PubMed
  • Search for this author on this site
Rebecca Truty
9Invitae, 1400 16th St, San Francisco, CA 94103
  • Find this author on Google Scholar
  • Find this author on PubMed
  • Search for this author on this site
George Asimenos
10DNAnexus, 730 Market St Suite 2100, San Francisco, CA 94103
  • Find this author on Google Scholar
  • Find this author on PubMed
  • Search for this author on this site
Birgit Funke
11Veritas Genetics, 99 Conifer Hill Dr, Danvers, MA 01923
  • Find this author on Google Scholar
  • Find this author on PubMed
  • Search for this author on this site
Mark Fleharty
12Broad Institute, 415 Main Street, Cambridge, MA 02142
  • Find this author on Google Scholar
  • Find this author on PubMed
  • Search for this author on this site
Brad A. Chapman
13Harvard T.H. Chan School of Public Health Bioinformatics Core, 655 Huntington Ave, Boston, MA 02115
  • Find this author on Google Scholar
  • Find this author on PubMed
  • Search for this author on this site
Marc Salit
14Material Measurement Laboratory, Joint Initiative for Metrology in Biology, National Institute of Standards and Technology, Stanford, CA
  • Find this author on Google Scholar
  • Find this author on PubMed
  • Search for this author on this site
Justin M Zook
15Material Measurement Laboratory, National Institute of Standards and Technology, 100 Bureau Dr, MS8301, Gaithersburg, MD 20899
  • Find this author on Google Scholar
  • Find this author on PubMed
  • Search for this author on this site
  • Abstract
  • Full Text
  • Info/History
  • Metrics
  • Supplementary material
  • Preview PDF
Loading

Abstract

Standardized benchmarking methods and tools are essential to robust accuracy assessment of NGS variant calling. Benchmarking variant calls requires careful attention to definitions of performance metrics, sophisticated comparison approaches, and stratification by variant type and genome context. To address these needs, the Global Alliance for Genomics and Health (GA4GH) Benchmarking Team convened representatives from sequencing technology developers, government agencies, academic bioinformatics researchers, clinical laboratories, and commercial technology and bioinformatics developers for whom benchmarking variant calls is essential to their work. This team addressed challenges in (1) matching variant calls with different representations, (2) defining standard performance metrics, (3) enabling stratification of performance by variant type and genome context, and (4) developing and describing limitations of high-confidence calls and regions that can be used as “truth”. Our methods are publicly available on GitHub (https://github.com/ga4gh/benchmarking-tools) and in a web-based app on precisionFDA, which allow users to compare their variant calls against truth sets and to obtain a standardized report on their variant calling performance. Our methods have been piloted in the precisionFDA variant calling challenges to identify the best-in-class variant calling methods within high-confidence regions. Finally, we recommend a set of best practices for using our tools and critically evaluating the results.

Copyright 
The copyright holder for this preprint is the author/funder, who has granted bioRxiv a license to display the preprint in perpetuity. It is made available under a CC-BY 4.0 International license.
Back to top
PreviousNext
Posted May 24, 2018.
Download PDF

Supplementary Material

Email

Thank you for your interest in spreading the word about bioRxiv.

NOTE: Your email address is requested solely to identify you as the sender of this article.

Enter multiple addresses on separate lines or separate them with commas.
Best Practices for Benchmarking Germline Small Variant Calls in Human Genomes
(Your Name) has forwarded a page to you from bioRxiv
(Your Name) thought you would like to see this page from the bioRxiv website.
CAPTCHA
This question is for testing whether or not you are a human visitor and to prevent automated spam submissions.
Share
Best Practices for Benchmarking Germline Small Variant Calls in Human Genomes
Peter Krusche, Len Trigg, Paul C. Boutros, Christopher E. Mason, Francisco M. De La Vega, Benjamin L. Moore, Mar Gonzalez-Porta, Michael A. Eberle, Zivana Tezak, Samir Labadibi, Rebecca Truty, George Asimenos, Birgit Funke, Mark Fleharty, Brad A. Chapman, Marc Salit, Justin M Zook, and the Global Alliance for Genomics and Health Benchmarking Team
bioRxiv 270157; doi: https://doi.org/10.1101/270157
Digg logo Reddit logo Twitter logo Facebook logo Google logo LinkedIn logo Mendeley logo
Citation Tools
Best Practices for Benchmarking Germline Small Variant Calls in Human Genomes
Peter Krusche, Len Trigg, Paul C. Boutros, Christopher E. Mason, Francisco M. De La Vega, Benjamin L. Moore, Mar Gonzalez-Porta, Michael A. Eberle, Zivana Tezak, Samir Labadibi, Rebecca Truty, George Asimenos, Birgit Funke, Mark Fleharty, Brad A. Chapman, Marc Salit, Justin M Zook, and the Global Alliance for Genomics and Health Benchmarking Team
bioRxiv 270157; doi: https://doi.org/10.1101/270157

Citation Manager Formats

  • BibTeX
  • Bookends
  • EasyBib
  • EndNote (tagged)
  • EndNote 8 (xml)
  • Medlars
  • Mendeley
  • Papers
  • RefWorks Tagged
  • Ref Manager
  • RIS
  • Zotero
  • Tweet Widget
  • Facebook Like
  • Google Plus One

Subject Area

  • Genomics
Subject Areas
All Articles
  • Animal Behavior and Cognition (4092)
  • Biochemistry (8784)
  • Bioengineering (6490)
  • Bioinformatics (23376)
  • Biophysics (11761)
  • Cancer Biology (9163)
  • Cell Biology (13267)
  • Clinical Trials (138)
  • Developmental Biology (7420)
  • Ecology (11379)
  • Epidemiology (2066)
  • Evolutionary Biology (15106)
  • Genetics (10407)
  • Genomics (14017)
  • Immunology (9133)
  • Microbiology (22084)
  • Molecular Biology (8792)
  • Neuroscience (47417)
  • Paleontology (350)
  • Pathology (1421)
  • Pharmacology and Toxicology (2483)
  • Physiology (3709)
  • Plant Biology (8057)
  • Scientific Communication and Education (1433)
  • Synthetic Biology (2213)
  • Systems Biology (6019)
  • Zoology (1251)