Skip to main content
bioRxiv
  • Home
  • About
  • Submit
  • ALERTS / RSS
Advanced Search
New Results

Scalable, ultra-fast, and low-memory construction of compacted de Bruijn graphs with Cuttlefish 2

View ORCID ProfileJamshed Khan, View ORCID ProfileMarek Kokot, View ORCID ProfileSebastian Deorowicz, View ORCID ProfileRob Patro
doi: https://doi.org/10.1101/2021.12.14.472718
Jamshed Khan
1Department of Computer Science, University of Maryland, College Park, Maryland, USA
2Center for Bioinformatics and Computational Biology, University of Maryland, College Park, Maryland, USA
  • Find this author on Google Scholar
  • Find this author on PubMed
  • Search for this author on this site
  • ORCID record for Jamshed Khan
Marek Kokot
3Faculty of Automatic Control, Electronics and Computer Science, Silesian University of Technology, Gliwice, Poland
  • Find this author on Google Scholar
  • Find this author on PubMed
  • Search for this author on this site
  • ORCID record for Marek Kokot
Sebastian Deorowicz
3Faculty of Automatic Control, Electronics and Computer Science, Silesian University of Technology, Gliwice, Poland
  • Find this author on Google Scholar
  • Find this author on PubMed
  • Search for this author on this site
  • ORCID record for Sebastian Deorowicz
Rob Patro
1Department of Computer Science, University of Maryland, College Park, Maryland, USA
2Center for Bioinformatics and Computational Biology, University of Maryland, College Park, Maryland, USA
  • Find this author on Google Scholar
  • Find this author on PubMed
  • Search for this author on this site
  • ORCID record for Rob Patro
  • For correspondence: rob@cs.umd.edu
  • Abstract
  • Full Text
  • Info/History
  • Metrics
  • Supplementary material
  • Data/Code
  • Preview PDF
Loading

Abstract

The de Bruijn graph has become a key data structure in modern computational genomics, and of keen interest is its compacted variant. The compacted de Bruijn graph provides a lossless representation of the graph, and it is often considerably more efficient to store and process than its non-compacted counterpart. Construction of the compacted de Bruijn graph resides upstream of many genomic analyses. As the quantity of sequencing data and the number of reference genomes on which to perform these analyses grow rapidly, efficient construction of the compacted graph becomes a computational bottleneck for these tasks.

We present Cuttlefish 2, significantly advancing the existing state-of-the-art methods for construction of this graph. On a typical shared-memory machine, it reduces the construction of the compacted de Bruijn graph for 661K bacterial genomes (2.58 Tbp of input reference genomes) from about 4.5 days to 17–23 hours. Similarly on sequencing data, it constructs the graph for a 1.52 Tbp white spruce read set in about 10 hours, while the closest competitor, which also uses considerably more memory, requires 54–58 hours.

Competing Interest Statement

RP is a co-founder of Ocean Genomics, inc.

Footnotes

  • https://github.com/COMBINE-lab/cuttlefish

  • 15 This bound is not tight, as KMC 3 actually distributes sequences longer than (ν + 1)-mers—reducing computation. See Section 4.3.4.

Copyright 
The copyright holder for this preprint is the author/funder, who has granted bioRxiv a license to display the preprint in perpetuity. It is made available under a CC-BY-NC-ND 4.0 International license.
Back to top
PreviousNext
Posted December 16, 2021.
Download PDF

Supplementary Material

Data/Code
Email

Thank you for your interest in spreading the word about bioRxiv.

NOTE: Your email address is requested solely to identify you as the sender of this article.

Enter multiple addresses on separate lines or separate them with commas.
Scalable, ultra-fast, and low-memory construction of compacted de Bruijn graphs with Cuttlefish 2
(Your Name) has forwarded a page to you from bioRxiv
(Your Name) thought you would like to see this page from the bioRxiv website.
CAPTCHA
This question is for testing whether or not you are a human visitor and to prevent automated spam submissions.
Share
Scalable, ultra-fast, and low-memory construction of compacted de Bruijn graphs with Cuttlefish 2
Jamshed Khan, Marek Kokot, Sebastian Deorowicz, Rob Patro
bioRxiv 2021.12.14.472718; doi: https://doi.org/10.1101/2021.12.14.472718
Digg logo Reddit logo Twitter logo Facebook logo Google logo LinkedIn logo Mendeley logo
Citation Tools
Scalable, ultra-fast, and low-memory construction of compacted de Bruijn graphs with Cuttlefish 2
Jamshed Khan, Marek Kokot, Sebastian Deorowicz, Rob Patro
bioRxiv 2021.12.14.472718; doi: https://doi.org/10.1101/2021.12.14.472718

Citation Manager Formats

  • BibTeX
  • Bookends
  • EasyBib
  • EndNote (tagged)
  • EndNote 8 (xml)
  • Medlars
  • Mendeley
  • Papers
  • RefWorks Tagged
  • Ref Manager
  • RIS
  • Zotero
  • Tweet Widget
  • Facebook Like
  • Google Plus One

Subject Area

  • Bioinformatics
Subject Areas
All Articles
  • Animal Behavior and Cognition (3477)
  • Biochemistry (7316)
  • Bioengineering (5294)
  • Bioinformatics (20189)
  • Biophysics (9972)
  • Cancer Biology (7698)
  • Cell Biology (11243)
  • Clinical Trials (138)
  • Developmental Biology (6416)
  • Ecology (9912)
  • Epidemiology (2065)
  • Evolutionary Biology (13271)
  • Genetics (9347)
  • Genomics (12544)
  • Immunology (7667)
  • Microbiology (18928)
  • Molecular Biology (7415)
  • Neuroscience (40870)
  • Paleontology (298)
  • Pathology (1226)
  • Pharmacology and Toxicology (2125)
  • Physiology (3138)
  • Plant Biology (6836)
  • Scientific Communication and Education (1268)
  • Synthetic Biology (1891)
  • Systems Biology (5295)
  • Zoology (1083)