ABSTRACT
Signal detection analyses are used to assess whether there is any evidence of signal within a large collection of hypotheses. For example, we may wish to assess whether there is any evidence of association with disease among a set of biologically related genes. Such an analysis typically treats all genes within the sets similarly, even though there is substantial information concerning the likely importance of each gene within each set. For example, deleterious variants within genes that show evidence of purifying selection are more likely to substantially affect the phenotype than genes that are not under purifying selection, at least for traits that are themselves subject to purifying selection. Here we improve such analyses by incorporating prior information into a higher-criticism-based signal detection analysis. We show that when this prior information is predictive of whether a gene is associated with disease, our approach can lead to a significant increase in power. We illustrate our approach with a gene-set analysis of amyotrophic lateral sclerosis (ALS), which implicates a number of gene-sets containing SOD1 and NEK1 as well as showing enrichment of small p-values for gene-sets containing known ALS genes.
INTRODUCTION
High-throughput sequencing (HTS) studies, both whole exome studies (WES) and, more recently, whole genome studies (WGS), have emerged as the primary approach to identifying genetic variation that may be associated with disease. Unlike genome-wide association studies which depend on linkage disequilibrium between tag SNPs and pathogenic variation, WES and WGS studies are able to assay pathogenic variation directly, and as a result, are able to directly interrogate the role of rare variation in disease. When the disease phenotype impacts the fitness of an individual, variants with a large effect on the phenotype will tend to be rare, as they will tend to be pruned out of the population before reaching appreciable frequency by purifying selection. This has been seen empirically (Park et al., 2011), and rare-variant focused disease mapping studies have successfully implicated a number of newly discovered disease genes, including studies in predominantly later onset diseases such as amyotrophic lateral sclerosis (ALS, Poppe et al, 2014; Cirulli et al. 2015) and idiopathic pulmonary fibrosis (IPF, Palmer et al, 2018). As each individual variant is rare, single-variant analyses are likely underpowered for detecting disease association. Therefore, rare-variant analyses usually integrate the effects of many variants across a gene or other genetic region. In these analyses, variants are often filtered by surrogates of the variant’s likely deleteriousness including the variant’s frequency in the general population as well as annotations of the variant’s likely functional impact on the ultimate protein product. Further, when such an analysis is restricted to rare variation, a gene that demonstrates an excess of deleterious variants in cases over controls provides strong evidence for the direct involvement of that gene in disease etiology as any indirect effects such as linkage disequilibrium are minimized.
A primary goal of such studies is signal identification, i.e., identifying individual genes that show significant differences in the burden of qualifying variants between cases and controls. For example, in the whole exome ALS sequencing study (Gelfman et al. 2018), the exomes of 18536 genes were sequenced across 3093 ALS cases and 8186 neurologically normal controls and tested for differences in the burden of rare variation, identifying SOD1 [MIM: 147450], NEK1[MIM: 604588], and TARDBP[MIM:605078] as being involved in ALS (Gelfman et al. 2018). Though these findings are undoubtedly an advance for ALS genetics, there is likely signal in this dataset that is below the identification threshold and, thus, remains latent. Our goal here is to enhance our understanding of disease etiology by uncovering this latent signal.
An alternate to signal identification is signal detection, i.e., the detection of whether any genes within a set of genes participating in a biologic process show significant differences in the burden of qualifying variants between cases and controls. Signal detection does not attempt to identify which genes in the set are non-null, just that there exists some subset of the genes within the gene-set that show signal. As a result, the signal detection problem can be addressed by a goodness of fit test that assesses whether the distribution of p-values, for the individual gene-level tests within the gene-set, follow a uniform distribution (i.e., all tests in the gene-set are null). Rejecting such a test implies that at least some of the genes in the gene-set have differences in the burden of qualifying variants between cases and controls. If the gene sets are chosen carefully, signal detection within the set can provide mechanistic insight into disease etiology, for example by emphasizing the importance of specific pathways or otherwise related gene sets. Higher Criticism (HC) is one such approach. Donoho and Jin (Donoho & Jin, 2004) showed that the HC statistic obtains the optimal detection limit for “rare-weak” alternatives where a small proportion of hypotheses weakly deviate from the null.
A limitation of such an approach is that it treats all genes within the focal gene-set equally, even though there is substantial external prior information about the relative importance of a gene within a gene-set. Genic intolerance (Petrovski et al, 2013), network centrality (Barabási & Albert, 1999; White & Smyth, 2003) and gene expression in disease relevant tissues are examples of sources of prior information that can be used to quantify the relative importance of genes within gene-sets. We discuss these sources of prior information in detail in the materials and methods section below.
Here we develop a novel HC statistic that incorporates prior information concerning the relative importance of genes within a gene-set into the analysis. We develop an asymptotic theory for the null distribution of our statistics and describe permutation procedures that can be used when the asymptotic approximation is likely to be poor. We conduct extensive simulation studies and show that when the prior information is correlated with those genes that are involved in disease our approach leads to a substantial increase in power. We illustrate our approach with a gene-set analysis of a whole exome sequencing study of amyotrophic lateral sclerosis.
MATERIALS AND METHODS
Overview of our approach
We develop a self-contained gene-set test based on a higher criticism statistic that explicitly weights the contribution of each gene in the set by prior information reflecting a genes likelihood of being involved in disease pathology. The higher criticism statistic our approach is based on can be thought of as a goodness of fit test. Specifically, consider a set of n statistics, X1, X2, …, Xn and empirical distribution function . In our application, these statistics represent tests of gene/phenotype association for each of the n genes in a gene-set selected on the basis of biological relationships. The higher criticism test assesses whether the observed distribution,
, is consistent with F0(x), the distribution of the test statistics under the global null. In most cases, the statistics we are working with are the p-values associated with each gene in the set. Therefore, as these p-values will be uniform under the null, we have that F0 (x) = x and we could assess whether the observed distribution of p-values is consistent with the global null using the Kolomorgorov-Smirnov (KS) test statistic,
However, Donoho and Jin (2004) showed that the power to detect small shifts in the distribution function due to a small number of weak signals can be improved by scaling the KS test by and restricting the domain over which the supremum is taken to the tail. These modifications of the KS test give rise to the higher-criticism (HC) test,
Incorporating prior information into higher-criticism statistics
To incorporate prior information into the HC framework, we assume that for the ith gene (ith Hypothesis being tested) there is affiliated a weight, wi ≥ 0, such that wi quantifies the relative importance of a gene within the gene-set. Let w = (w1, w2, …, wn). Let xi be ith gene’s p-value and let . Define
where
is the empirical distribution function of the x*s and F0 (x*) is cumulative distribution function of x* under the global null hypothesis. We can show that
.
It is not difficult to see that HC* is of the same form as the unweighted HC statistic studied by (Jaeschke, 1979) which was shown to converge in distribution to the Gumbel distribution as n goes to infinity. However, as noted by Barrett and Lin (2014), this convergence is extremely slow and unlikely to yield a good approximation in most cases. As a result, we use permutation to approximate the null distribution of HC*. When testing across a large number of gene-sets, we use the algorithm proposed by Ge, Dudoit, & Speed (2003) to account for testing multiple, potentially correlated, hypotheses.
Choosing the weights
The weights summarize prior information concerning the relative importance of a gene within a gene-set. We consider three main sources of this information: 1) Genic intolerance; 2) Network centrality; and 3) Gene expression in disease relevant tissues.
Genic Intolerance
Genic Intolerance quantifies the amount of purifying selection affecting a gene relative to a genome-wide average. Specifically, using standing human variation in large, publicly available, databases such as gnomAD (Lek et al, 2016), the number of common functional variants within a gene is regressed against the total number of variants within that gene. The Residual Variance Intolerance Score (RVIS) is the gene-level residual from that regression (Petrovski, Wang, Heinzen, Allen, & Goldstein, 2013). Thus, if a gene has less functional variation than expected given the total amount of variation within the gene, it will have a negative RVIS score. If it has more functional variation than expected, it will tend to have a positive score. RVIS has been shown to be strongly predictive of Mendelian disease genes, especially those that lead to early-onset severe disease phenotypes (Petrovski, Wang, Heinzen, Allen, & Goldstein, 2013).
Here, we calculate a gene’s intolerance-based weight, wri, as the gene’s intolerance percentile among all 18536 scored genes, scaled to be between 0 and 2. By rescaling, we ensure that those genes that have intolerance scores that are less than the mean, and hence are more likely to be important in disease etiology, are given more importance in the overall gene-set, by decreasing their p-values.
Connectivity of genes within biologic gene sets
Interactions between genes in a genetic gene set can be represented by a network. In such a representation, nodes denote genes and the edges connecting them represent gene-gene interactions. It is quite common in biologic networks for a few genes to have a much larger number of connections than other genes. These highly connected genes are referred to as “hub” genes, and it is reasonable to hypothesize that deleterious mutations within such genes might be more disruptive of the biologic process represented by the network than mutations falling within less connected, more distal, genes.
The connectivity of a node is captured in the graph theory concept of “centrality” (White & Smyth, 2003; Borgatti, 2005), which can be quantified in a number of ways. Here, we investigate 3 measures of network centrality: degree centrality (White & Smyth, 2003; Borgatti, 2005), Eigenvector centrality (Freeman, 1979; Stephenson and Zelen, 1989; Wasserman and Faust, 1994; White & Smyth, 2003) and PageRank centrality (Page et al, 1998, White & Smyth, 2003). Degree centrality is simply the number of edges from a given node, i.e., the number of genes that interact with a given gene. Eigenvector centrality of a node is a measure of the importance of the nodes it is connected to, i.e. a node is important if it is connected to other important nodes. (Freeman, 1979; Stephenson and Zelen, 1989; Wasserman and Faust, 1994; White & Smyth, 2003). PageRank centrality is defined by the pageRank algorithm used in web searches by Google (Page et al, 1998, White & Smyth, 2003). pageRank assumes that a web page (node) is more important if it receives more links (directed connections) from other high pageRank scored web pages (Page et al, 1998, White & Smyth, 2003). Unlike degree centrality and eigenvector centrality, pageRank considers the directionality of the connection.
For a given centrality measure, Let ci be the centrality for the ith gene. In order to generate weights that result in smaller p-values for more highly connected genes, we take wci = λ (aci + bc̅)−1, where c̅ is the mean centrality across the gene set, a and b are user-defined constants (here we take a = 0.95 and b = 0.05), and λ is a scaling factor so that the mean of the weights is one.
Gene expression in disease-related tissues
Genes that are important in disease etiology are more likely to be expressed in disease-related tissues during the developmental period leading to the disease. Therefore, for the ith gene we define a weight wei = λ (aei + be̅)−1, where ei is the expression level (appropriately normalized) in a disease-related tissue, e̅ is the mean expression across all genes in the gene set, a and b are user-defined constants (here we take a = 0.95 and b = 0.05), and λ is a scaling factor so that the mean of the weights is one.
Simulation study
We conduct a simple simulation study to evaluate the utility of our approach. For each scenario, we simulate 1e+4 datasets. For each simulated dataset we generate n independent statistics Xi, i = 1, …, n, associated with n hypotheses. Let π be the proportion of the Xi′s that are generated under the alternative. We assume Xi ∼ N(μ, 1) under the alternative and Xi ∼ N(0,1) under the null. Thus, marginally, Xi ∼ πN(μ, 1) + (1 − π)N(0,1). Note that π characterizes the sparsity of the alternatives among all the hypotheses tested while μ controls the location shift from null to alternative. Thus, in our simulations, we evaluate the power of our approach as π and μ vary and choose configurations that explore the detection boundary outlined by Donoho & Jin, 2014. Each Xi is converted to a p-value via pi = Φ(−|Xi|). Weights are generated from a truncated exponential distribution and then scaled to have mean one. We consider three different scenarios: 1) weights are randomly assigned to genes; 2) weights are negatively correlated with disease-associated genes so that their p-values in the HC* statistic are decreased, increasing their influence on the statistic; and 3) weights are positively correlated with disease-associated genes so that these genes will have less influence on the HC* statistic while the influence of genes that are not disease-associated will be increased. We generate a large number of simulated datasets under the global null (i.e., π = 0) and use these to calculate a rejection threshold for each scenario. Specifically, we take the top 5th percentile of HC and the HC* statistics (corresponding to the weighting scenarios) calculated using the global null simulated datasets and use them as rejection thresholds for the corresponding statistics under the various alternative hypotheses. Please see supplementary materials for complete details.
Amyotrophic lateral sclerosis whole exome study
We illustrate our approach through an analysis of data from a whole exome sequencing study comprised of 3093 ALS patients and 8186 controls of European ancestry (Gelfman et al, 2018). The sample information is available online (alsdb.org). The 18536 genes were sequenced and captured with the standard approach of Gelfman et al, 2018. Gene-level qualifying variant collapsing analyses were conducted according to two definitions of qualifying variants (table 1). Specifically, for a given qualifying variant definition and a given gene, we create an indicator variable of whether a given subject has a qualifying variant in that gene. We then test for association between the presence of a qualifying variant in the gene and case-control status using the Cochran–Mantel–Haenszel test, where strata are defined by the stratification score (Epstein et al, 2007). This results in two p-values per gene (corresponding to two definitions of qualifying variant); we take the minimum of the two to get a single gene-level p-value.
Different Genetic Models and qualifying criteria
We analyzed 4436 candidate gene sets extracted from the hallmark, GO biological process collections (c5.bp, version 6.1, Subramanian et al., 2005, Liberzon et al., 2015). Gene sets containing less than 10 genes were not analyzed.
We consider three different sources of prior information in developing gene-level weights: genetic networks from bioGRID (BioGRID Version 3.4.147, Stark et al., 2006, Chatraryamontri et al., 2017), genic intolerance (Petrovski, Wang, Heinzen, Allen, & Goldstein, 2013), and gene expression levels in disease-relevant tissue (brain) from GTex (GTEx Consortium, 2015). We used three different metrics for summarizing a genes importance within a genetic network: degree centrality (White & Smyth, 2003), eigenvector centrality (Freeman, 1979; Stephenson and Zelen, 1989; Wasserman and Faust, 1994; White & Smyth, 2003), and pageRank centrality (Page et al, 1998, White & Smyth, 2003).
For each gene-set, we conducted both weighted (denoted by w), using the weighting schemes highlighted above, and unweighted (denoted by u) analyses. Since we are interested in whether gene set analyses can help uncover specific gene sets that disease genes participate in, we also removed two genes that were found to be significantly associated with ALS: SOD1 (raw p-value: 4.1e-15, Bonferroni adjusted p-value: 7.6e-11) and NEK1 (raw p-value: 6.74e-10, Bonferroni adjusted p-value: 1.25e-05) from the gene set analyses. These analyses are denoted by wr and ur for weighted and unweighted analyses, respectively.
We obtained empirical null distributions of our statistics by randomly permuting case/control status (each distribution was generated using 2e+6 permutations). Because genes may participate in multiple gene sets, leading to correlation between tests, we used the step-down minP algorithm (Box 4: Ge, Dudoit, & Speed, 2003) to obtain multiplicity adjusted p-values across all the gene sets analyzed.
RESULTS
Simulation study
As expected, when the effect size, μ, is fixed, power increases with an increasing proportion of non-null hypotheses, π (figure 1). Similarly, when π is fixed, power increases with μ. More interestingly, we can see that weighting can substantially increase the power of the HC analysis if the values of weights are negatively correlated with non-null hypotheses, i.e. the weights tend to make p-values affiliated with non-null hypotheses smaller, so that those hypotheses have more influence on the final HC statistics (orange dashed lines). Further, the power is very similar to an unweighted analysis (solid black lines) when the weights are uncorrelated, i.e., are random noise (blue lines). It is only when the weights are positively correlated with the non-null hypotheses, so that null hypothesis are given more influence on the HC statistic, that we see a substantial negative effect on power when using weighting (red dash lines). However, in real applications one would expect that most weighting schemes would be somewhat informative of which genes would be disease-related. Thus, these results suggest that there is little downside to weighting individual hypotheses in HC analyses.
Figure 1 shows the selected simulation results of the power of weighted HC (mu=1.44, pi=0.447). The black lines (points) represent HC with no weights, the blue dashed lines(points) represent HC with uncorrelated weights. The red dashed lines (points) represent HC with weights that negatively correlated with non-null hypotheses (favors the alternative). The orange dashed lines (points) represent HC with weights that negatively correlated with non-null hypotheses (favors the null). 1a) gene set size:100, Fixed mu. 1b) gene set size:100, Fixed pi. 1c) gene set size:100. power vs. correlations between weights and non-null hypothesis. 1d) gene set size:1000, Fixed mu. 1e) gene set size:1000, Fixed pi. 1f) gene set size:1000. power vs. correlations between weights and non-null hypothesis. See support information for more results.
ALS data analysis
We found that marginally associated genes, had a strong effect on all HC analyses (weighted or not). For example, all gene-sets containing SOD1 (260) and NEK1 (21) are significantly associated with ALS after multiplicity adjustment, regardless of the HC statistic used (table 2). GSEA (Subramanian et al., 2005) fails to detect any significant gene sets. To investigate whether there is residual signal in gene sets after the marginally significant genes are removed, we conducted gene set analyses that excluded SOD1 and NEK1 from inclusion in any gene set. This analysis did not detect any significant gene sets after multiplicity adjustment, regardless of the method used.
The number of significant gene sets by method
We then investigated whether there were enrichment differences between methods for gene sets involving 51 known ALS disease genes highlighted in Cirulli et al. 2015 (Table S1). The results of these analyses are presented in table 3 and one can see that weighting based on pageRank centralities performs well. Since many of these gene sets are likely devoid of any signal, we repeated this analysis while further restricting the gene sets considered to those where there was at least one gene-set analysis approach yielding a marginally significant result (p<=0.05) (table 4). Once again, we find that pageRank centrality does well and that HC outperforms GSEA.
Method Comparison on ALS-related gene sets (Total gene sets: 899)
Method Comparison on core ALS-related gene sets (Total gene sets: 186)
DISCUSSION
We have presented a new gene-set based analysis that incorporates prior information into the analysis using a higher criticism approach. In both simulation studies and real data analyses, we showed that such an approach can lead to higher power. However, the choice of weights is important and consideration should be made for what information is most likely to be predictive of truly associated disease genes. For example, in our p-value enrichment analyses of known ALS genes, we found little enrichment when we used genic intolerance measures as our weights. As genic intolerance is indicative of purifying selection, this choice of weights may be less informative in a late-onset disorder such as ALS. Results would likely be different for earlier-onset disorders such as autism spectrum disorder, epilepsy, or schizophrenia. Further applications across a spectrum of diseases are needed before general recommendations can be made with respect to weighting schemes.
The higher criticism analysis can be thought of as a goodness of fit test that focuses on extreme deviations (by taking a max) from expectation under a global null that none of the genes within the gene set are associated with the disease. Though this approach has been shown to be optimal in detecting sparse signals within a large collection of hypotheses, it may be less sensitive to detecting signal that is more diffuse. In such a case, there may be an advantage in integrating over the tail of the distribution of deviations rather than taking a max. We are currently investigating this approach and plan to highlight it in a future manuscript.
DESCRIPTION OF SUPPLEMEMTAL DATA
Supplemental Data include two tables and 4 series of figures. Table S1 lists the ALS-related genes according to Cirulli et al, 2015. Table S2 describes the simulation procedure. The text describes profiling details in the simulation study. The 4 series of figures provides the complete results of the simulation under various condition settings.
DECLARATION OF INTERESTS
Andrew S. Allen and David B. Goldstein received research support from AstraZeneca, Inc. The remaining authors declare no competing interests.
WEB RESOURCES
An R package implementing the approach is available at https://github.com/mqzhanglab/wHC
ACKNOWLEDGEMENTS
We thank Chong Jin from University of North Carolina at Chapel Hill and Dr. Zhiguo Li from Duke University, for suggestions on the asymptotic distribution of Higher Criticism based on the p-values and weighted p-values under the null hypothesis.
Footnotes
EMAIL: andrew.s.allen{at}duke.edu