Cracking the black box of deep sequence-based protein-protein interaction prediction

Abstract

Identifying protein-protein interactions (PPIs) is crucial for deciphering biological pathways and their dysregulation. Numerous prediction methods have been developed as a cheap alternative to biological experiments, reporting surprisingly high accuracy estimates. We systematically investigated how much reproducible deep learning models depend on data leakage, sequence similarities, and node degree information and compared them to basic machine learning models. We found that overlaps between training and test sets resulting from random splitting lead to strongly overestimated performances. In this setting, models learn solely from sequence similarities and node degrees. When data leakage is avoided by minimizing sequence similarities between training and test set, performances become random. Moreover, we find that baseline models directly leveraging sequence similarity and network topology show good performance at a fraction of the computational cost. Thus we advocate that any improvements are reported relative to baseline methods in the future. Our findings suggest that predicting protein-protein interactions remains an unsolved task for proteins showing little sequence similarity to previously studied proteins, highlighting that further experimental research into the dark protein interactome and better computational methods are needed.