Large uncertainty in individual PRS estimation impacts PRS-based risk stratification

Large-scale genome-wide association studies have enabled polygenic risk scores (PRS), which estimate the genetic value of an individual for a given trait. Since PRS accuracy is typically assessed using cohort-level metrics (e.g., R2), uncertainty in PRS estimates at individual level remains underexplored. Here we show that Bayesian PRS methods can estimate the variance of an individual’s PRS and can yield well-calibrated credible intervals for the genetic value of a single individual. For real traits in the UK Biobank (N=291,273 unrelated “white British”) we observe large variance in individual PRS estimates which impacts interpretation of PRS-based stratification; for example, averaging across 13 traits, only 0.8% (s.d. 1.6%) of individuals with PRS point estimates in the top decile have their entire 95% credible intervals fully contained in the top decile. We provide an analytical estimator for individual PRS variance—a function of SNP-heritability, number of causal SNPs, and sample size—and observe high concordance with individual variances estimated via posterior sampling. Finally as an example of the utility of individual PRS uncertainties, we explore a probabilistic approach to PRS-based stratification that estimates the probability of an individual’s genetic value to be above a prespecified threshold. Our results showcase the importance of incorporating uncertainty in individual PRS estimates into subsequent analyses.

Introduction equations 30,33 or, if inversion is computationally prohibitive, approximated [34][35][36][37][38][39] . The uncertainty in other 57 biomarkers and non-genetic risk factors have also been well-studied 40 . For example, smoothing methods 58 and error-correction methods are performed before biomarkers and non-genetic risk factors are included in 59 the predictive model 41,42 . 60 Motivated by potential clinical applications of PRS in personalized medicine, where one of the main goals 61 is to estimate risk of a given individual, we focus on evaluating uncertainty in PRS estimates at the level of 62 a single target individual. Our goal is to quantify the statistical noise in individual PRS estimates ( $ ! ) 63 conditional on data used to train the PRS. We assess two metrics of individual PRS uncertainty: (1) the 64 standard deviation of the PRS estimate for individual i, denoted ( $ ! ); and (2) the -level credible 65 interval for the genetic value of individual i, defined as the interval that contains the genetic value of 66 individual i (GVi) with (e.g., 95%) probability, denoted ( GVi-CI). We extend the Bayesian framework and 99 th percentiles in the same testing sample after their 95% credible intervals are taken into account. 80 Finally, we explore a probabilistic approach to incorporating PRS uncertainty in PRS-based stratification 81 and demonstrate how such approaches can enable principled risk stratification under different cost scenarios.   Figure 3) and uncertainty (Methods).

145
First, we assess the calibration of the -level credible intervals for GV ! estimated by LDpred2. We   stratification. Since most traits analyzed here are not disease traits, we use "above-threshold" and "below- factors such as prevalence and the intended clinical application 1 . We then partition the above-threshold 209 individuals into two categories: individuals whose 95% GVi-CI are fully above the threshold t ("certain 210 above-threshold") and individuals whose 95% GVi-CI contain t ("uncertain above-threshold"). Similarly,

211
we classify individuals as below-threshold if their PRS point estimate lies below a prespecified threshold 212 ( PRS $ ! < ) and we partition these individuals into "certain below-threshold" and "uncertain below-  Table 1). Consistent with simulations, we find that uncertainty is higher for 218 traits that are more polygenic 45 (Table 1) with the average standard deviation of PRS $ ! ranging between 0.2 219 to 0.41 across the studied traits (Table S1). We assessed whether the standard practice of quantile 220 normalization of phenotypes impacts PRS and verify that for phenotypes with mildly skewed distributions,

221
GWAS marginal association statistics and PRS uncertainty are largely consistent with or without quantile 222 normalization ( Supplementary Figures 11 and 12).

223
For completeness, we investigated the impact of the threshold , and credible level , on PRS-based 224 stratification uncertainty, defined as the proportion of above-threshold individuals classified as "certain probability when the inferential variance in PRS estimation is taken into consideration (Table 2).

247
In contrast to current PRS-based stratification practices which compare an individual's PRS point estimate,

248
PRS $ 0 , to a given threshold without incorporating uncertainty, here we explore the use of the posterior 249 probability that GV for individual i is above the threshold (i.e. Pr(GV ! > )). We estimate Pr(GV ! > ) In this work, we demonstrate that uncertainty in PRS estimates at the individual level can have a large impact 286 on subsequent analyses such as PRS-based risk stratification. We note that this work focuses estimating 287 genetic value rather than predicting phenotype; uncertainty in predictions of phenotype will be larger than 288 the results reported here due to the additional uncertainty in unmeasured environmental factors 46  ), could also be investigated. Overall, our methods produce well-calibrated credible intervals in realistic 317 simulation parameter ranges, albeit slight mis-calibration when polygenicity is low and heritability is high. 318 We hypothesize that it is due to several approximations employed in LDpred2 for computational efficiency. 319 We leave investigation of the impact of approximation on calibration and further improvement for future 320 work.

321
Second, while we find broad evidence that both trait-specific genetic architecture parameters (e.g., there is strong evidence of autocorrelation, which otherwise will lead to underestimation of variance.

399
Although in this work we focus on LDpred2, the above described procedure is generalizable to a Under this assumption, following Appendix A in ref. 26 , the least squares estimate of the GWAS marginal Since the per-SNP heritability in this model, Finally, by multiplying a fixed genotype vector ! to both sides, we have Therefore, the prediction error variance is equal to the expectation of posterior variance under repeated 436 sampling of . Given large sample sizes, we expect that for each realization of , | L ! " Q will not 437 deviate much from ‰ | L ! " QŠ. Therefore, PEV and posterior variance will be approximately equal. 438 We also note that under infinitesimal model setting, the posterior variance of genetic value has the same 439 matrix form as the inversion of coefficient matrix of mixed model equation for BLUP 30,33 .          Table 1. PRS-based individual stratification uncertainty across 11 complex traits in UK Biobank. We quantified PRS-based stratification uncertainty in testing individuals for eleven complex traits at two stratification thresholds (t = 90 th and t = 99 th percentiles). The numbers of certain versus uncertain classifications are determined from the 95% credible intervals ( = 95%). For each trait, we report averages (and standard deviations) from five random partitions of the whole dataset.  Table 2. Average 95% posterior ranking credible intervals for individuals at two stratification thresholds for 11 traits. We estimated the 95% posterior ranking credible intervals for individuals at the 90 th and 99 th percentiles of the testing population PRS estimates. Mean and standard deviation are calculated from the 95% posterior ranking intervals of individuals whose point estimates lie within 0.5% of the stratification threshold (213 individuals between the 89.5 th and 90.5 th percentiles for t = 90 th and between the 98.5 th and 99.5 th percentiles for t = 99 th ).