Skip to main content
bioRxiv
  • Home
  • About
  • Submit
  • ALERTS / RSS
Advanced Search
New Results

Cooler: scalable storage for Hi-C data and other genomically-labeled arrays

View ORCID ProfileNezar Abdennur, View ORCID ProfileLeonid Mirny
doi: https://doi.org/10.1101/557660
Nezar Abdennur
1Institute for Medical Engineering and Science, Massachusetts Institute of Technology, Cambridge, MA, 02139, United States
  • Find this author on Google Scholar
  • Find this author on PubMed
  • Search for this author on this site
  • ORCID record for Nezar Abdennur
  • For correspondence: nezar@mit.edu
Leonid Mirny
1Institute for Medical Engineering and Science, Massachusetts Institute of Technology, Cambridge, MA, 02139, United States
2Department of Physics, Massachusetts Institute of Technology, Cambridge, MA, 02139, United States
  • Find this author on Google Scholar
  • Find this author on PubMed
  • Search for this author on this site
  • ORCID record for Leonid Mirny
  • Abstract
  • Full Text
  • Info/History
  • Metrics
  • Preview PDF
Loading

Abstract

Most existing coverage-based (epi)genomic datasets are one-dimensional, but newer technologies probing interactions (physical, genetic, etc.) produce quantitative maps with two-dimensional genomic coordinate systems. Storage and computational costs mount sharply with data resolution when such maps are stored in dense form. Hence, there is a pressing need to develop data storage strategies that handle the full range of useful resolutions in multidimensional genomic datasets by taking advantage of their sparse nature, while supporting efficient compression and providing fast random access to facilitate development of scalable algorithms for data analysis. We developed a file format called cooler, based on a sparse data model, that can support genomically-labeled matrices at any resolution. It has the flexibility to accommodate various descriptions of the data axes (genomic coordinates, tracks and bin annotations), resolutions, data density patterns, and metadata. Cooler is based on HDF5 and is supported by a Python library and command line suite to create, read, inspect and manipulate cooler data collections. The format has been adopted as a standard by the NIH 4D Nucleome Consortium. Cooler is cross-platform, BSD-licensed, and can be installed from the Python Package Index or the bioconda repository. The source code is maintained on Github at https://github.com/mirnylab/cooler.

Copyright 
The copyright holder for this preprint is the author/funder, who has granted bioRxiv a license to display the preprint in perpetuity. It is made available under a CC-BY 4.0 International license.
Back to top
PreviousNext
Posted February 22, 2019.
Download PDF
Email

Thank you for your interest in spreading the word about bioRxiv.

NOTE: Your email address is requested solely to identify you as the sender of this article.

Enter multiple addresses on separate lines or separate them with commas.
Cooler: scalable storage for Hi-C data and other genomically-labeled arrays
(Your Name) has forwarded a page to you from bioRxiv
(Your Name) thought you would like to see this page from the bioRxiv website.
CAPTCHA
This question is for testing whether or not you are a human visitor and to prevent automated spam submissions.
Share
Cooler: scalable storage for Hi-C data and other genomically-labeled arrays
Nezar Abdennur, Leonid Mirny
bioRxiv 557660; doi: https://doi.org/10.1101/557660
Digg logo Reddit logo Twitter logo Facebook logo Google logo LinkedIn logo Mendeley logo
Citation Tools
Cooler: scalable storage for Hi-C data and other genomically-labeled arrays
Nezar Abdennur, Leonid Mirny
bioRxiv 557660; doi: https://doi.org/10.1101/557660

Citation Manager Formats

  • BibTeX
  • Bookends
  • EasyBib
  • EndNote (tagged)
  • EndNote 8 (xml)
  • Medlars
  • Mendeley
  • Papers
  • RefWorks Tagged
  • Ref Manager
  • RIS
  • Zotero
  • Tweet Widget
  • Facebook Like
  • Google Plus One

Subject Area

  • Bioinformatics
Subject Areas
All Articles
  • Animal Behavior and Cognition (3510)
  • Biochemistry (7352)
  • Bioengineering (5328)
  • Bioinformatics (20276)
  • Biophysics (10024)
  • Cancer Biology (7749)
  • Cell Biology (11315)
  • Clinical Trials (138)
  • Developmental Biology (6438)
  • Ecology (9957)
  • Epidemiology (2065)
  • Evolutionary Biology (13332)
  • Genetics (9362)
  • Genomics (12591)
  • Immunology (7713)
  • Microbiology (19044)
  • Molecular Biology (7446)
  • Neuroscience (41060)
  • Paleontology (300)
  • Pathology (1231)
  • Pharmacology and Toxicology (2138)
  • Physiology (3163)
  • Plant Biology (6865)
  • Scientific Communication and Education (1274)
  • Synthetic Biology (1897)
  • Systems Biology (5317)
  • Zoology (1089)