Evaluating Cell Identity from Transcription Profiles

Induced pluripotent stem cells (iPS) and direct lineage programming offer promising autologous and patient-specific sources of cells for personalized drug-testing and cell-based therapy. Before these engineered cells can be widely used, it is important to evaluate how well the engineered cell types resemble their intended target cell types. We have developed a method to generate CellScore, a cell identity score that can be used to evaluate the success of an engineered cell type in relation to both its initial and desired target cell type, which are used as references. Of 20 cell transitions tested, the most successful transitions were the iPS cells (CellScore > 0.9), while other transitions (e.g. induced hepatocytes or motor neurons) indicated incomplete transitions (CellScore < 0.5). In principle, the method can be applied to any engineered cell undergoing a cell transition, where transcription profiles are available for the reference cell types and the engineered cell type. Highlights A curated standard dataset of transcription profiles from normal cell types was created. CellScore evaluates the cell identity of engineered cell types, using the curated dataset. CellScore considers the initial and desired target cell type. CellScore identifies the most successfully engineered clones for further functional testing.


Summary
Induced pluripotent stem cells (iPS) and direct lineage programming offer promising autologous and patient-specific sources of cells for personalized drug-testing and cellbased therapy. Before these engineered cells can be widely used, it is important to evaluate how well the engineered cell types resemble their intended target cell types.
We have developed a method to generate CellScore, a cell identity score that can be used to evaluate the success of an engineered cell type in relation to both its initial and desired target cell type, which are used as references. Of 20 cell transitions tested, the most successful transitions were the iPS cells (CellScore > 0.9), while other transitions (e.g. induced hepatocytes or motor neurons) indicated incomplete transitions (CellScore < 0.5). In principle, the method can be applied to any engineered cell undergoing a cell transition, where transcription profiles are available for the reference cell types and the engineered cell type.

Introduction
The discovery that a terminally differentiated cell type could be reverted to a pluripotent cell state capable of generating all possible cell types from the three germ cell layers (Takahashi and Yamanaka, 2006) has revolutionized the field of stem cell research and regenerative medicine. This concept of "reprogramming" one cell type to another has been further applied to lineage or direct reprogramming (Heinrich et al., 2015), in which cells are directly reprogrammed from one differentiated cell type to another, without first passing through a pluripotent state (Kim and Schöler, 2014). The development of these methods enable the production of patient-specific cells, which could be used for individualized drug testing and cell replacement therapy.
However, before these pluripotent stem cell-derived or other engineered cell types can be used for clinical applications, the safety and efficacy of the cells must be proven.
First and foremost comes the patient's safety with regard to oncogenicity and transplant rejection. For example, the process of creating iPS cells, including the reprogramming protocol and culture conditions, may introduce genomic aberrations (Lund et al., 2012), which could potentially render the iPS cells and their derivatives oncogenic. Indeed, the first phase 1 clinical trial using iPS cells was put on hold after gene mutations that were not present in the donor fibroblasts were found in the iPS cells used to treat age-related macular degeneration (Garber, 2015). Strict quality control of each patient-specific iPSC line for autologous transplantation also increases the cost and time for individualized treatment. In favor of a faster and more cost-effective way to generate clinical-grade cells, the suspended clinical trial aimed to resume (https://ipscell.com/2015/07/firstipscstop/) with the use of quality-controlled, banked, human leukocyte antigen (HLA)-typed iPS cells for allogeneic transplantation (Scudellari, 2016), which should reduce the risk of transplant rejection. In addition to 3 of 24 5 55 60 65 70 successful safety trials, the cell-based treatment must demonstrate effectiveness. But even before the process of human clinical trials starts, engineered cell types that are destined for cell therapy must be sufficiently characterized. We propose that characterization of any kind of derived cell type that is intended for clinical use, can be partly accomplished by analyzing the gene expression profiles, in comparison to their starting donor cells and desired target cells. The same applies to their use for disease modelling and drug testing. Before extensive time and effort is spent on functional testing of engineered cell lines, transcription profiles could already be used as an initial screen to identify the most promising clones.
Current protocols to reprogram cells from one cell type to another typically involve forced expression of key factors, modulation of pathways with inhibitors and/or agonists, and changes in growth conditions. These protocols evolve from informed guesses tested by trial and error strategies, and even then, the molecular mechanisms factorization models . In another method, CellNet quantifies the similarity of engineered cells and their target cell types, by extracting gene regulatory networks from transcriptome data of diverse cell types and further filtering the networks for regulatory interactions with ChIP-chip/-seq data (Cahan et al., 2014;Morris et al., 2014). Finally, KeyGenes uses a generalized linear regression model to extract classifier genes from a panel of transcriptional human fetal tissue, and then uses the "key genes" to classify query samples Takasato et al., 2015).
In the present study, we propose a method to generate the CellScore. Unlike other computational methods, the CellScore method evaluates the cell identity of an experimentally derived cell type by taking into account both the initial donor cell type and the target cell type. First, we manually curated a set of reference transcriptome profiles from various normal tissues or cell types ( Figure 1A). Then, we developed a method to assess cell identity of engineered cell types, using the expression profiles of the donor cell type and the desired target cell type as references ( Figure 1B). Our method is targeted towards researchers following the gene expression changes of cells whereby the cells are expected to change their identity from a starting donor cell type towards a modified (e.g. reprogrammed or differentiated) cell type. If the desired target cell type has appropriate gene expression data, our method can be used to evaluate the "proximity" between an engineered cell type and its desired target cell, in terms of a CellScore. We can also evaluate whether the changes in gene expression during the identity change are leading to the desired state, and which genes contribute to this successful reprogramming. Finally, the method and accompanying reference dataset have been implemented as R packages that offer functions to calculate and visualize the cell scoring results for a given cell transition dataset in multiple ways including a single function call to obtain a detailed summary report in PDF format. indicating that the normalization process was able to preserve sample characteristics, even though the samples originated from different studies.

Similarity scores based on quantitative expression values.
To further examine the similarity between donor, target, and engineered cell types, cells were sub-categorized according to their donor cell type to yield a cell subtype. From here on, we denote cell subtypes in the following format: derived cell type, "-", donor cell type. Using the expression values, cosine similarity was calculated between the centroids of each subtype ( Figure 3A). For the transitions to the ESC (embryonic stem cell) target cell type (i.e. iPS reprogramming), most of the iPS subtypes were highly similar to ESCs ( Figure 3B). One notable exception were the "iPS-GSC*" cells, which were originally claimed to be iPS lines derived from spermatogonial cells (GSC), but were later shown to be more consistent with a fibroblast cell type (Ko et al., 2010). This is clearly visible in  Figure 5A, with high target-like scores (>1.5) and low donor-like scores (< 1.5). The CellScore is simply the difference between the target-and donorlike scores, as shown in Figure 5B. The CellScore ranges between -1.2 and 1.2. A highly positive CellScore indicates that the experimental cell is very similar to its target cell type, while a highly negative score indicates that the experimental cell type has not successfully transitioned and has remained more donor-like. To visualize the CellScore and the metrics that contribute to the CellScore of an engineered cell type, a report can be generated with diagnostic plots as in Figure Figure 5A), one is a wildtype parent of the patients (#3) and the remaining iPS lines (#4,5,6) were previously established lines (iPS 19-9; hPSCreg identifier WAi001-A) from the Thomson laboratory (Yu et al., 2009). We also have calculated the CellScore for the iPS 19-9 lines in Supplementary Data S3 (see GSE15176). All iPS 19-9 lines from the original study (Yu et al., 2009)  platform with stromal fibroblasts to differentiate the cells to iHEP (Berger et al., 2015).
Though the CellScore for these two lines was only 0.57 and 0.48, the lines clearly had moderate target-like scores and low donor-like scores ( Figure 5C). In the microarray expression data, the iHEPs from this study expressed high levels of albumin, like the primary hepatocytes that were used as standards. However, a high level of alphafetoprotein gene in the iHEP cells shows that they were still immature hepatocytes and had not completely converted to the desired target cell type, primary hepatocytes.
Nevertheless, the authors demonstrated that their iHEP cells had hepatocyte morphology and polarity, as well as drug metabolizing ability (Berger et al., 2015).

Performance of the method.
To evaluate the performance of the CellScore method, higher FPR (0.14) at 0.95 sensitivity. This higher FPR was partly driven by chemically treated hepatocyte samples, which were counted as true negatives, but in fact were not significantly influenced by small molecule treatment.

Discussion
In the present study, we propose a new approach, called CellScore, to score experimental cell types undergoing a transition in cell identity. This is a particularly emerging current issue as it becomes more routine to engineer an essential cell type on-demand for disease modelling or patient-specific drug-testing. The CellScore alone is not intended to wholly replace other cellular and molecular techniques to confirm the desired cell function, but CellScore results can already provide some information about the success of engineered cell transition.
We applied the CellScore method to over 20 transitions in cell identity from data available from one particular microarray platform, including over 1500 samples in 86 studies. Some transitions were very popular and had much data from multiple studies (e.g. FIB →ESC, PSC → HEP, PSC → MYOC) while others, such as those deriving cells of the nervous system, only had data from a single study. In three transitions, for which additional public data was available from the same platform, we were able to show that the CellScore method was able to classify true positives with high specificity and low false positive rates. Overall, and probably not surprising, the highest scoring considered more fetal-like, rather than terminally differentiated, and due to ethical and practical considerations, there is a lack of sufficient numbers of human fetal tissue samples that could serve as standards. Therefore, the standard target cell type used to calculate the CellScore is a proxy, and in itself represents an ideal and unattainable goal. For example, in the PSC → HEP transition, primary hepatocytes were used as the target; however, the induced hepatocytes were still immature compared to primary hepatocytes. In spite of their immaturity, these cells already demonstrated sufficient functionality in terms of metabolic activity and liver marker expression, such that they could be used in some in vitro toxicity assays (Berger et al., 2015). It may not even be necessary to derive mature hepatocytes, depending on its downstream use. Immature hepatocytes could be used as a form of cell replacement therapy by injecting them into a liver, where the immature hepatocytes could fully differentiate into hepatocytes in situ (Si-Tayeb et al., 2010). In such cases, the CellScore could still be useful as a guide to choosing the most differentiated iHEPs and compare outcomes between distinctly scoring cells.
Beyond the CellScore as a single number to quantify a cell transition, the CellScore highlight genes responsible for an incomplete transition, which could be in fact driven by donor cell type-specific transcription networks (Nefzger et al., 2017) Functional enrichment analysis could highlight pathways or processes that may be suitable input for small molecule induction/repression prediction, in an effort to improve differentiation protocols (Siatkowski et al., 2013).
Other computational methods to evaluate cell identity exist, but mainly these are restricted in that they perform specific functions and rely on specific technologies.
Pluritest focuses on the scoring of human pluripotent stem cells and is restricted to using datasets from dedicated microarray platforms, as a means for standard testing on a web-based platform behind a login-wall . Keygenes is a specialized resource for scoring RNA-seq profiles to a fixed panel of human embryonic tissues, with freely available software . On the other hand, CellNet is applicable to microarray data of different platforms, provided that at least 60 microarrays are available for each tissue type. CellNet uses cell or tissue-specific gene regulatory networks to classify engineered cell types and its software is freely available (Cahan et al., 2014). None of these methods explicitly takes into consideration the donor cell type when evaluating the cell identity. We propose the CellScore method as a freely available method to evaluate any kind of cell identity transition, which could in principle score any type of cell transition for which donor cell and target cell data are available, regardless of the platform. The current study has been limited to data from one platform as a proof-of-principle, but in the future, we aim to leverage the data from additional microarray platforms, as well as RNA-seq datasets, for a comprehensive expression database (Mah et al., 2017) to evaluate cell identity.

Experimental Procedures
Data selection and processing. Transcriptome data was collected from published 13 of 24  Figure 1A and a complete list of all samples is given in Supplementary Data S1. Samples were manually annotated for their tissue origin and cell types using ontology terms (Malone et al., 2010;Stachelscheid et al., 2014). Raw microarray data were obtained from the public repositories Gene Expression Omnibus and ArrayExpress (Barrett et al., 2009;Rustici et al., 2013). Microarray data was normalized using the YuGene transform, which allows comparisons between experiments and is not dependent on the data distribution (Lê Cao et al., 2014). For present/absent calls, MAS5.0 detection p-values were calculated using the "affy" Bioconductor package for Affymetrix 3'IVT arrays (Gautier et al., 2004). Probesets in each sample were considered to be "present" if the detection p-value was less than 0.05. Annotation of probesets to genes was obtained from the BioC annotation package "hgu133plus2.db" (Carlson M, 2016). In the case that multiple probesets mapped to one gene, the probeset with the highest median across all samples was chosen to represent the expression for that gene.
Cell scoring method. Cell scores for a given cell transition starting from a specific cell type (defined by the donor cell type) towards a different cell type (defined by the target cell type) were calculated using two independent metrics, the on/off score and the cosine similarity score ( Figure 3A). Both metrics are composite, calculated once with the respect to the donor and once with the respect the target cell type ( Figures 1C). The first metric was based on the so-called on/off genes, i.e. cell type specific genes that were uniquely expressed (detected as "present") in either the donor cell type (donor markers) or the target cell type (target markers) of a particular cell transition. Then the on/off scores for a query sample i was defined by the fraction of lost donor markers (fD,i) and the fraction of gained of target markes (fT,i), such that: where nD,i and nT,i are the numbers of donor or target markers present in the sample i, and nD and nT are the numbers of donor and target markers, respectively, present in the standard cell types of a particular cell transition.
In the second metric, cosine similarity of transcription profiles was calculated between all target, donor and experimental cell types, based on a subset of the most variable genes from the standard cell types in the normalized expression matrix. By default, the most variable genes were defined to be those genes whose standard cell type group medians were in the top 10% of the expression interquartile range. Cosine similarity was calculated between each query sample and the mean centroid of each standard cell type. To get an intuitive impression of the cell identity of a query sample i, the donor-like score (sD,i) and the target-like score (sT,i,) were calculated as follows: where cosD,i is the cosine similarity between the query sample i and the centroid of the 15 of 24 Finally, the CellScore of a query sample i for a particular cell transition was defined to be: The CellScore values range between -2 and 2, with negative values indicating that the query sample is more donor-like (the cell transition was not successful), and positive values indicating that it is more like the target cell type (meaning a potentially successful cell transition). The cell scoring method and pre-processed reference and test data presented in this study is publicly available at https://github.com/nmah/CellScore.

Functional enrichment and data visualization. The Bioconductor package
"ReactomePA" was used to determine the enrichment of marker genes in Reactome pathways, using the entire list of genes in the dataset as the background (universe) set of genes (Yu and He, 2016). Enrichment results were plotted using the Bioconductor package "clusterProfiler" (Yu et al., 2012). Receiver Operating Characteristic curves were plotted by the R-package "pROC" (Robin et al., 2011).        carcinoma, induced hepatocytes, treated RPE, Wharton's jelly stem cells and cancer cell types (brain, bladder, colon, lung, prostate).

Tables
None.