RT Journal Article
SR Electronic
T1 A long-context language model for deciphering and generating bacteriophage genomes
JF bioRxiv
FD Cold Spring Harbor Laboratory
SP 2023.12.18.572218
DO 10.1101/2023.12.18.572218
A1 Shao, Bin
YR 2024
UL http://biorxiv.org/content/early/2024/02/07/2023.12.18.572218.abstract
AB Inspired by the success of large language models, we develop a long-context generative model for genomes. Our multiscale transformer model was pre-trained on unannotated bacteriophage genomes with byte-level tokenization. We demonstrate the foundational capabilities of our model including the prediction of essential genes, genetic variant effects, regulatory element activity and taxonomy of unannotated sequences. Furthermore, it generates de novo sequences up to 96K base pairs, which contain functional regulatory elements and novel proteins with phage-related functions.Competing Interest StatementThe authors have declared no competing interest.