Abstract
Identifying protein-protein interactions (PPIs) is crucial for deciphering biological pathways and their dysregulation. Numerous prediction methods have been developed as a cheap alternative to biological experiments, reporting surprisingly high accuracy estimates. We systematically investigated how much reproducible deep learning models depend on data leakage, sequence similarities, and node degree information and compared them to basic machine learning models. We found that overlaps between training and test sets resulting from random splitting lead to strongly overestimated performances. In this setting, models learn solely from sequence similarities and node degrees. When data leakage is avoided by minimizing sequence similarities between training and test set, performances become random. Moreover, we find that baseline models directly leveraging sequence similarity and network topology show good performance at a fraction of the computational cost. Thus we advocate that any improvements are reported relative to baseline methods in the future. Our findings suggest that predicting protein-protein interactions remains an unsolved task for proteins showing little sequence similarity to previously studied proteins, highlighting that further experimental research into the dark protein interactome and better computational methods are needed.
Competing Interest Statement
The authors have declared no competing interest.
Footnotes
(1) Incorporation of two topology-based baseline methods, the state-of-the-art methods D-SCRIPT and Topsy-Turvy, and their unbalanced dataset. (2) More discussion regarding other non-sequence-based approaches. (3) We emphasize the need for reporting improvements relative to baseline methods and underscore that predicting PPIs without relying on sequence similarity or degree information shortcuts is crucial for exploring the dark, unmapped part of the protein interactome.
https://github.com/biomedbigdata/data-leakage-ppi-prediction