Heavy-tailed prior distributions for sequence count data: removing the noise and preserving large differences

Abstract

In RNA-seq differential expression analysis, investigators aim to detect genes with changes in expression across conditions, despite technical and biological variability. A common task is to accurately estimate the effect size. When the counts are low or highly variable, the simple effect size estimate has high variance, leading to poor ranking of genes by effect size. Here we propose apeglm, which uses a heavy-tailed Cauchy prior distribution for effect sizes, resulting in lower bias than previous shrinkage estimators, while still reducing variance. apeglm is available at http://bioconductor.org/packages/apeglm, and can be used from within the DESeq2 software.