GILEA: GAN Inversion-enabled latent eigenvalue analysis for phenome profiling and editing

Modeling heterogeneous disease states by data-driven methods has great potential to advance biomedical research. However, a comprehensive analysis of phenotypic heterogeneity is often challenged by the complex nature of biomedical datasets and emerging imaging methodologies. Here, we propose a novel GAN Inversion-enabled Latent Eigenvalue Analysis (GILEA) framework and apply it to phenome profiling and editing. As key use cases for fluorescence and natural imaging, we demonstrate the power of GILEA using publicly available SARS-CoV-2 datasets stained with the multiplexed fluorescence cell-painting protocol as well as real-world medical images of common skin lesions captured by dermoscopy. The quantitative results of GILEA can be biologically supported by editing latent representations and simulating dynamic phenotype transitions between physiological and pathological states. In conclusion, GILEA represents a new and broadly applicable approach to the quantitative and interpretable analysis of biomedical image data. The GILEA code and video demos are publicly available at https://github.com/CTPLab/GILEA.

the emergence of (unsorted) eigenvalues in the improved implementation of d FID , we recently 13 suggested comparing sorted 39 eigenvalues (d Eig ) as a simple alternative to d FID . For i = 1, 2, consider Z i := (z i 1 ,..., z i n i ) be a collection of n i p-dimensional 40 vectors, we have 41 Definition 1. Let S i = 1 n i Z i Z T i be the sample covariance matrix (SCM) of Z i , we define where j i is the j-th largest eigenvalue of S i .
address the infeasible deployment of d Eig to cases where we obtain imbalanced values of d Eig for k = 1,..., c, we propose 48 Definition 2. For i = 1, 2 and k = 1,..., c, let S i = ( 1 n i,1 Z i,1 Z T i,1 ,..., 1 n i,c Z i,c Z T i,c ) be the collection of SCMs of Z i,k , then we define where p 0 ⌧ p and S 1 is the reference SCM. 49 Similar to Principal Component Analysis (PCA) 15 , we only utilize the p 0 ⌧ p largest eigenvalues that reflect the largest 50 variances and the most critical information. As the 5 largest eigenvalues dominate > 95% of the overall values in the experiments, 51 we set p 0 = 5 throughout the article (please see 'Results' and 'Methods' for more detail).
To guarantee the consistency and to simulate clearly interpretable phenotypic transitions, image sequences w.r.t.  . Then, the learned features are utilized to analyze the e↵ectiveness of di↵erent drugs. b, In GILEA, we start the pipeline by pre-training the decoder on center-cropped single-cell images in an adversarial training manner for each fluorescent channel. Then, we learn robust latent representations with a residual-based encoder 20 for reconstructing these single-cell images. For quantifying the drug e↵ects, we compute the eigenvalues with learned single-cell representations and support our quantitative results with clearly observable phenotypic transitions.

3/18
Here, we report the application of GILEA to two large-scale fluorescence microscopy datasets RxRx19 (a,b) released by 76 Recursion 5 , which document the phenotypic e↵ects of more than 1800 drug candidates on Severe Acute Respiratory Syndrome is provided as percentage (⇥100) in the following plots for clearer visualization.

89
Definition 4. Following the specification of SCMs S Mock , S Infected , S Drug as Eq. 2, we define    In this study, we propose the novel GILEA approach for profiling and editing latent phenotypic representations of biomedical 160 imaging data. We demonstrate the practical application of GILEA on the multiplexed fluorescence microscopy SARS-CoV-2 161 datasets at single-cell and single-organelle granularity. Through application to well-defined biological settings, we achieve 162 refinement of high-throughput drug screening under the cell-painting protocol. Importantly, we verify the GILEA results in 163 direct comparison to the baseline method 5 , which is implemented with a fundamentally di↵erent approach. The drug candidates  Fig. 9), the association is less clear for drug candidates with 169 a positive baseline hit score (e.g., c-MET inhibitors in Fig. 9 (b)). This suggests that the current cell-level evidence is not Inversion-enabled computational 'scissor' will allow high-throughput and selective computational interventions on (latent) 185 genetic representations and may therefore serve as a promising proxy tool for targeted disease therapeutics.

187
In reference to Fig. 1(b), here we describe the architecture of GILEA in detail and provide insights into the evaluation of model the reference and comparing this to the malignant counterpart (e.g., malignant melanoma, mel), we randomly mix the images of 199 'nv' and 'mel' (e.g., x mel,1 ,..., x mel,n mel , x nv,1 ,..., x nv,n nv ) according to the interpolation weight w = n nv n nv +n mel . Then, we measure 200 the distribution di↵erence between nv and the mixed collection by d LEA . Ideally, we should observe that d LEA converges to 0 201 when w is shifted from 0 to 1 with the increasing inclusion of 'nv' (n nv ") and exclusion of 'mel' images (n mel #) (Fig. 5 (b)).

204
Model architecture 205 Motivated by the impressive achievements of GAN inversion 7 , we instantiate GILEA with the state-of-the-art GAN inversion 206 model ( Fig. 1(b)). Firstly, we learn the decoder (generator) under the StyleGAN 28, 29 framework, which has proved to 207 be successful in hallucinating high-quality natural images. Based on the training protocols suggested in the widely-used  and 'pixel2style2pixel' 31 (pSp) for comparison, both of which start with a ResNet backbone and then concatenate a feature 212 pyramid network 32 . Similar to the loss design of these studies that enable high-fidelity image reconstruction, we determine our 213 objective to be L = 1 L moco + 2 L 2 , where L moco is the contrastive loss that is superior in visual representation learning 33 , 214 and L 2 is the l 2 reconstruction loss. Eventually, we report the quantitative reconstruction results in Tab achieves better PSNR and SSIM scores than StyleGAN2_pSp, the qualitative di↵erence in image reconstruction appears 225 marginal ( Fig. 5 (a)). Furthermore, when examining the d LEA behavior with the increasing interpolation weight, we found 226 notable di↵erences between the two architectures. Compared to StyleGAN2, Fig. 5 (b) shows that d LEA computed with 227 StyleGAN3_e4e increases unexpectedly from w = 0.25 to w = 0.5 for the mixture data collection of nv and mel images, which 228 is in conflict with the fact that more inclusion of benign mole images should reduce the distance to the nv reference category.

229
With regards to nv, d LEA of StyleGAN3_pSp surprisingly suggests a larger data di↵erence of bkl (bcc) than mel. This is also 230 counter-intuitive as malignant neoplasms (mel) are well known to present distinct appearances in lesion size and pigmentation 231 heterogeneity, allowing these lesions to be clearly di↵erentiated from a benign mole (nv). Besides, we notice that the learned 232 representations of StyleGAN3 tend to be more convoluted and are thus less ideal to support clear biological interpretation (See 233 for example Appendix Fig. 11). Since StyleGAN3 is motivated by the texture-sticking drawback occurring in natural images

11/18
Next, we investigate the d LEA performance in regards to the amount p 0 of the largest eigenvalues utilized in Eq. 2. As we 241 can see in Fig. 5 (