Correcting gradient-based interpretations of deep neural networks for genomics

ABSTRACT

Post-hoc attribution methods are widely applied to provide insights into patterns learned by deep neural networks (DNNs). Despite their success in regulatory genomics, DNNs can learn arbitrary functions outside the probabilistic simplex that defines one-hot encoded DNA. This introduces a random gradient component that manifests as noise in attribution scores. Here we demonstrate the pervasiveness of off-simplex gradient noise for genomic DNNs and introduce a statistical correction that is effective at improving the interpretability of attribution methods.