PT  - JOURNAL ARTICLE
AU  - Brandon Carter
AU  - Maxwell Bileschi
AU  - Jamie Smith
AU  - Theo Sanderson
AU  - Drew Bryant
AU  - David Belanger
AU  - Lucy Colwell
TI  - Critiquing Protein Family Classification Models Using Sufficient Input Subsets
AID  - 10.1101/674119
DP  - 2019 Jan 01
TA  - bioRxiv
PG  - 674119
4099  - http://biorxiv.org/content/early/2019/06/19/674119.short
4100  - http://biorxiv.org/content/early/2019/06/19/674119.full
AB  - In many application domains, neural networks are highly accurate and have been deployed at large scale. However, users often do not have good tools for understanding how these models arrive at their predictions. This has hindered adoption in fields such as the life and medical sciences, where researchers require that models base their decisions on underlying biological phenomena rather than peculiarities of the dataset introduced. In response, we propose a set of methods for critiquing deep learning models and demonstrate their application for protein family classification, a task for which high-accuracy models have considerable potential impact. Our methods extend the sufficient input subsets technique, which we use to identify subsets of features (SIS) in each protein sequence that are alone sufficient for classification. Our suite of tools analyzes these subsets to shed light on the decision-making criteria employed by models trained on this task. These tools expose that while deep models may perform classification for biologically-relevant reasons, their behavior varies considerably across choice of network architecture and parameter initialization. While the techniques that we develop are specific to the protein sequence classification task, the approach taken generalizes to a broad set of scientific contexts in which model interpretability is essential.