Skip to main content
bioRxiv
  • Home
  • About
  • Submit
  • ALERTS / RSS
Advanced Search
New Results

Spectral Jaccard Similarity: A new approach to estimating pairwise sequence alignments

Tavor Z. Baharav, Govinda M. Kamath, View ORCID ProfileDavid N. Tse, Ilan Shomorony
doi: https://doi.org/10.1101/800581
Tavor Z. Baharav
1Department of Electrical Engineering, Stanford University
  • Find this author on Google Scholar
  • Find this author on PubMed
  • Search for this author on this site
Govinda M. Kamath
2Microsoft Research New England, Cambridge, Massachusetts
  • Find this author on Google Scholar
  • Find this author on PubMed
  • Search for this author on this site
David N. Tse
1Department of Electrical Engineering, Stanford University
  • Find this author on Google Scholar
  • Find this author on PubMed
  • Search for this author on this site
  • ORCID record for David N. Tse
Ilan Shomorony
3Department of Electrical and Computer Engineering, University of Illinois, Urbana-Champaign
  • Find this author on Google Scholar
  • Find this author on PubMed
  • Search for this author on this site
  • For correspondence: ilans@illinois.edu
  • Abstract
  • Full Text
  • Info/History
  • Metrics
  • Data/Code
  • Preview PDF
Loading

Abstract

A key step in many genomic analysis pipelines is the identification of regions of similarity between pairs of DNA sequencing reads. This task, known as pairwise sequence alignment, is a heavy computational burden, particularly in the context of third-generation long-read sequencing technologies, which produce noisy reads. This issue is commonly addressed via a two-step approach: first, we filter pairs of reads which are likely to have a large alignment, and then we perform computationally intensive alignment algorithms only on the selected pairs. The Jaccard similarity between the set of k-mers of each read can be shown to be a proxy for the alignment size, and is usually used as the filter. This strategy has the added benefit that the Jaccard similarities don’t need to be computed exactly, and can instead be efficiently estimated through the use of min-hashes. This is done by hashing all k-mers of a read and computing the minimum hash value (the min-hash) for each read. For a randomly chosen hash function, the probability that the min-hashes are the same for two distinct reads is precisely their k-mer Jaccard similarity. Hence, one can estimate the Jaccard similarity by computing the fraction of min-hash collisions out of the set of hash functions considered.

However, when the k-mer distribution of the reads being considered is significantly non-uniform, Jaccard similarity is no longer a good proxy for the alignment size. In particular, genome-wide GC biases and the presence of common k-mers increase the probability of a min-hash collision, thus biasing the estimate of alignment size provided by the Jaccard similarity. In this work, we introduce a min-hash-based approach for estimating alignment sizes called Spectral Jaccard Similarity which naturally accounts for an uneven k-mer distribution in the reads being compared. The Spectral Jaccard Similarity is computed by considering a min-hash collision matrix (where rows correspond to pairs of reads and columns correspond to different hash functions), removing an offset, and performing a singular value decomposition. The leading left singular vector provides the Spectral Jaccard Similarity for each pair of reads. In addition, we develop an approximation to the Spectral Jaccard Similarity that can be computed with a single matrix-vector product, instead of a full singular value decomposition.

We demonstrate improvements in AUC of the Spectral Jaccard Similarity based filters over Jaccard Similarity based filters on 40 datasets of PacBio reads from the NCTC collection. The code is available at https://github.com/TavorB/spectral_jaccard_similarity.

Footnotes

  • tavorb{at}stanford.edu, gokamath{at}microsoft.com, dntse{at}stanford.edu

  • https://github.com/TavorB/spectral_jaccard_similarity

Copyright 
The copyright holder for this preprint is the author/funder, who has granted bioRxiv a license to display the preprint in perpetuity. It is made available under a CC-BY 4.0 International license.
Back to top
PreviousNext
Posted March 26, 2020.
Download PDF
Data/Code
Email

Thank you for your interest in spreading the word about bioRxiv.

NOTE: Your email address is requested solely to identify you as the sender of this article.

Enter multiple addresses on separate lines or separate them with commas.
Spectral Jaccard Similarity: A new approach to estimating pairwise sequence alignments
(Your Name) has forwarded a page to you from bioRxiv
(Your Name) thought you would like to see this page from the bioRxiv website.
CAPTCHA
This question is for testing whether or not you are a human visitor and to prevent automated spam submissions.
Share
Spectral Jaccard Similarity: A new approach to estimating pairwise sequence alignments
Tavor Z. Baharav, Govinda M. Kamath, David N. Tse, Ilan Shomorony
bioRxiv 800581; doi: https://doi.org/10.1101/800581
Reddit logo Twitter logo Facebook logo LinkedIn logo Mendeley logo
Citation Tools
Spectral Jaccard Similarity: A new approach to estimating pairwise sequence alignments
Tavor Z. Baharav, Govinda M. Kamath, David N. Tse, Ilan Shomorony
bioRxiv 800581; doi: https://doi.org/10.1101/800581

Citation Manager Formats

  • BibTeX
  • Bookends
  • EasyBib
  • EndNote (tagged)
  • EndNote 8 (xml)
  • Medlars
  • Mendeley
  • Papers
  • RefWorks Tagged
  • Ref Manager
  • RIS
  • Zotero
  • Tweet Widget
  • Facebook Like
  • Google Plus One

Subject Area

  • Bioinformatics
Subject Areas
All Articles
  • Animal Behavior and Cognition (4397)
  • Biochemistry (9630)
  • Bioengineering (7123)
  • Bioinformatics (24939)
  • Biophysics (12670)
  • Cancer Biology (9995)
  • Cell Biology (14404)
  • Clinical Trials (138)
  • Developmental Biology (7989)
  • Ecology (12147)
  • Epidemiology (2067)
  • Evolutionary Biology (16025)
  • Genetics (10949)
  • Genomics (14778)
  • Immunology (9906)
  • Microbiology (23739)
  • Molecular Biology (9506)
  • Neuroscience (51050)
  • Paleontology (370)
  • Pathology (1545)
  • Pharmacology and Toxicology (2692)
  • Physiology (4038)
  • Plant Biology (8693)
  • Scientific Communication and Education (1512)
  • Synthetic Biology (2404)
  • Systems Biology (6458)
  • Zoology (1350)