PT  - JOURNAL ARTICLE
AU  - Nambiar, Ananthan
AU  - Liu, Simon
AU  - Hopkins, Mark
AU  - Heflin, Maeve
AU  - Maslov, Sergei
AU  - Ritz, Anna
TI  - Transforming the Language of Life: Transformer Neural Networks for Protein Prediction Tasks
AID  - 10.1101/2020.06.15.153643
DP  - 2020 Jan 01
TA  - bioRxiv
PG  - 2020.06.15.153643
4099  - http://biorxiv.org/content/early/2020/06/16/2020.06.15.153643.short
4100  - http://biorxiv.org/content/early/2020/06/16/2020.06.15.153643.full
AB  - The scientific community is rapidly generating protein sequence information, but only a fraction of these proteins can be experimentally characterized. While promising deep learning approaches for protein prediction tasks have emerged, they have computational limitations or are designed to solve a specific task. We present a Transformer neural network that pre-trains task-agnostic sequence representations. This model is fine-tuned to solve two different protein prediction tasks: protein family classification and protein interaction prediction. Our method is comparable to existing state-of-the art approaches for protein family classification, while being much more general than other architectures. Further, our method outperforms all other approaches for protein interaction prediction. These results offer a promising framework for fine-tuning the pre-trained sequence representations for other protein prediction tasks.Competing Interest StatementThe authors have declared no competing interest.