RT Journal Article SR Electronic T1 Multi-modal digital pathology for colorectal cancer diagnosis by high-plex immunofluorescence imaging and traditional histology of the same tissue section JF bioRxiv FD Cold Spring Harbor Laboratory SP 2022.09.28.509927 DO 10.1101/2022.09.28.509927 A1 Jia-Ren Lin A1 Yu-An Chen A1 Daniel Campton A1 Jeremy Cooper A1 Shannon Coy A1 Clarence Yapp A1 Juliann B. Tefft A1 Erin McCarty A1 Keith L. Ligon A1 Scott J. Rodig A1 Steven Reese A1 Tad George A1 Sandro Santagata A1 Peter K. Sorger YR 2022 UL http://biorxiv.org/content/early/2022/09/30/2022.09.28.509927.abstract AB Precision medicine is critically dependent on better methods for diagnosing and staging disease and predicting drug response. Histopathology using Hematoxylin and Eosin (H&E) stained tissue - not genomics – remains the primary diagnostic modality in cancer. Moreover, recently developed, highly multiplexed tissue imaging represents a means of enhancing histology workflows with single cell mechanisms. Here we describe an approach for collecting and analyzing H&E and high-plex immunofluorescence (IF) images from the same cells in a whole-slide format suitable for translational and clinical research and eventual deployment in diagnosis. Using data from 40 human colorectal cancer resections (60 million cells) we show that IF and H&E images provide human experts and machine learning algorithms with complementary information. We demonstrate the automated generation and ranking of computational models, based either on immune infiltration or tumor-intrinsic features, that are highly predictive of progression-free survival. When these models are combined, a hazard ratio of ∼0.045 is achieved, demonstrating the ability of multi-modal digital pathology to generate high-performance and interpretable biomarkers.Competing Interest StatementPKS is a co-founder and member of the BOD of Glencoe Software, a member of the BOD for Applied Biomath, and a member of the SAB for RareCyte, NanoString, and Montai Health; he holds equity in Glencoe, Applied Biomath, and RareCyte. PKS is a consultant for Merck and the Sorger lab has received research funding from Novartis and Merck in the past five years. YC is a consultant for RareCyte. DC, JC, EM, SR, and TG are employees of RareCyte. The DFCI receives funding for KLL research from the following entities: Amgen, Travera, and X4. DFCI and KLL have patents related to molecular diagnostics of cancer. SJR receives research support from Bristol-Myers-Squibb and KITE/Gilead. SJR is on the Scientific Advisory Board for Immunitas Therapeutics. The other authors declare no outside interests.