Abstract
Sophisticated inferential tools coupled with the coalescent model have recently emerged for estimating past population sizes from genomic data. Accurate methods are available for data from a single locus or from independent loci. Recent methods that model recombination require small sample sizes, make constraining assumptions about population size changes, and do not report measures of uncertainty for estimates. Here, we develop a Gaussian process-based Bayesian nonparametric method coupled with a sequentially Markov coalescent model which allows accurate inference of population sizes over time from a set of genealogies. In contrast to current methods, our approach considers a broad class of recombination events, including those that do not change local genealogies. We show that our method outperforms recent likelihood-based methods that rely on discretization of the parameter space. We illustrate the application of our method to multiple demographic histories, including population bottlenecks and exponential growth. In simulation, our Bayesian approach produces point estimates four times more accurate than maximum likelihood estimation (based on the sum of absolute differences between the truth and the estimated values). Further, our method’s credible intervals for population size as a function of time cover 90 percent of true values across multiple demographic scenarios, enabling formal hypothesis testing about population size differences over time. Using genealogies estimated with ARGweaver, we apply our method to European and Yoruban samples from the 1000 Genomes Project and confirm key known aspects of population size history over the past 150,000 years.
1 Introduction
For a single non-recombining locus, neutral coalescent theory predicts the set of timed ancestral relationships among sampled individuals, known as a gene genealogy (Kingman 1982; Hudson 1983; Tajima 1983; Hudson 1990). In the coalescent model with variable population size, the rate at which two lineages coalesce, or have a common ancestor, is a function of the population size in the past. Here we denote the population size trajectory by N(t), where t is time in the past, and use the term local genealogy to describe ancestral relationships at one non-recombining locus. When analyzing multilocus sequences, a single local genealogy will not represent the full history of the sample. Instead, the set of ancestral relationships and recombination events among a sample of multilocus sequences can be represented by a graph, known as the ancestral recombination graph (ARG) which depicts the complex structure of neighboring local genealogies and results in a computationally expensive model for inferring N(t) (Griffiths and Marjoram 1997; Wiuf and Hein 1999).
Recent studies have leveraged computationally simpler approximations for the coalescent with recombination—the sequentially Markov coalescent (SMC) (McVean and Cardin 2005) and its variant SMC′ (Marjoram and Wall 2006; Chen et al. 2009)—both of which model local genealogies as a continuous time Markov process along sequences (Figure 1). The difference between the SMC and SMC′ is that the SMC models only the class of recombination events that alter local genealogies of the sample. In general, the SMC′ is a better approximation to the ARG than the SMC (Chen et al. 2009; Wilton et al. 2015). Because of these features, in this work we rely on the SMC′ to model local genealogies with recombination.
Under the coalescent and the sequentially Markov coalescent (SMC and SMC′) models, population size trajectories and sequence data are separated by two stochastic processes: i) a state process which describes the relationship between the population size trajectory and the set of local genealogies, and ii) an observation process which describes how the hidden local genealogies are observed through patterns of nucleotide diversity in the sequence data. The observation process includes mutation and genotyping error while the state process models coalescence. Sequence data are then used to make inferences of population size trajectories. In this paper, we restrict attention to the state process of local genealogies and show how inferences of population size trajectories can be made from them. We solve a number of key modeling and inference problems, and thus provide a basis for developing efficient algorithms to infer population parameters from sequence data directly.
Whole-genome inference of population size trajectories has been hampered by the enormous size of the state space of local genealogies when the sample size is large. The pioneering, pairwise sequentially Markov Coalescent (PSMC) method of Li and Durbin (2011) employed the SMC to make inferences from a sample of size two (n = 2). In this method, time is discretized and the population size trajectory is piece-wise constant, allowing pairwise genealogies also to be discretized. Subsequent methods for samples larger than two similarly rely on the discretization of time and genealogies. The natural extension of the PSMC to n > 2 is the multiple sequentially Markovian coalescent (MSMC) (Schiffels and Durbin 2014). However, the MSMC models only the most recent coalescent event of the sample, and hence its estimation of population sizes is limited to very recent times. Other recent methods propose efficient ways of exploring the state space of hidden genealogies for n > 2 (Sheehan et al. 2013; Rasmussen et al. 2014), yet also rely on discretizing the state space of local genealogies and assume a piece-wise constant trajectory of population sizes. We show that the a priori specification of change points for the piece-wise population size trajectory required by current approaches is problematic because estimates of N(t) are sensitive to this specification. Moreover, current methods do not generate interval estimates for N(t).
Gaussian Process-based Bayesian inference of population size trajectories has proven to be a powerful and flexible nonparametric approach when applied to a single local genealogy (Palacios and Minin 2013; Lan et al. 2015). The two main advantages of the GP-based approach are: (i) it does not require a specific functional form of the population size trajectory (such as constant or exponential growth) and (ii) it does not require an arbitrary specification of change points in a piece-wise constant or linear framework.
In this paper, we show the downstream effects of discretizing time, assuming a piecewise constant trajectory, and reporting only point estimates for past population sizes. We overcome previous limitations by introducing a Bayesian nonparametric approach with a Gaussian Process (GP) to model the population size trajectory as a continuous function of time. More specifically, we model the logarithm of the population size trajectory a priori as a Gaussian process (the log ensures our estimates are positive). As mentioned above, we assume that local gene genealogies are known. For our Bayesian model, we develop a Markov Chain Monte Carlo (MCMC) method to sample from the posterior distribution of population sizes over time. Our MCMC algorithm uses the recently developed algorithm Split Hamiltonian Monte Carlo (splitHMC) (Shahbaba et al. 2014; Lan et al. 2015). splitHMC updates all model parameters jointly and it can be extended to a full inferential framework that is directly applicable to sequence data. In order to compare our Bayesian GP-based estimation of population size trajectories with a piece-wise constant maximum likelihood-based estimation (e.g. Li and Durbin 2011; Sheehan et al. 2013; Schiffels and Durbin 2014), we implemented the Expectation-maximization (EM) algorithm within our framework and computed the observed Fisher information to obtain confidence intervals of the maximum likelihood estimates.
Lastly, we address a key problem for inference of population size trajectories under sequentially Markov coalescent models is the efficient computation of transition densities needed in the calculation of likelihoods. Here, we express the transition densities of local genealogies in terms of local ranked tree shapes (Tajima 1983) and coalescent times, and show that these quantities are statistically sufficient for inferring population size trajectories either from sequence data directly or from the set of local genealogies. The use of ranked tree shapes allows us to exploit the state process of local genealogies efficiently since the space of ranked tree shapes has a smaller cardinality than the space of labeled topologies (Sainudiin et al. 2014).
Methods: SMC′ Calculations
Following notation similar to Rasmussen et al. (2014) (Table 1) a realization of the embedded SMC′ chain consists of a set of m local genealogies (g0, g1,…,gm-1), m − 1 recombination break-points at chromosomal locations (b1, b2,…,bm-1), and m − 1 pruning locations (p1, p2,…,pm-1), where pi = (ui, wi) indicates the time of the recombination event ui and the branch wi where recombination happened in genealogy gi-1 (Figure 1). Genealogy g0 corresponds to the genealogy of n sequences that contains the set of timed ancestral relationships among the n individuals for the chromosomal segment (0, b1]. Genealogy gi corresponds to the genealogy of the same n sequences for the chromosomal segment (bi, bi+1] for i = 1, 2,…,m − 2. Finally, denotes the time when two of j lineages coalesce in genealogy gi, measured in units of generations before present.
Using capital letters to denote random variables, the evolution of the SMC′ process along chromosomal segments is governed by a point process B = {Bi}i∈ℕ that represents the random locations of recombination breakpoints. We use Si = Bi − Bi-1, for i = 1, 2,…, m, to denote the segment lengths for each local genealogy, with S0 = B0 = 0. Let G = {Gi}i∈ℕ be the chain which records the local genealogies, and let P = (U, W) = {(Ui, Wi)}i∈ℕ represent the chain which records the pruning locations (time and branch) on G. The sequence (Gi, Pi = {Ui, Wi}, Bi) has the following conditional independence relation:
Given a chain of local genealogies, pruning locations and recombination breakpoints, the joint transition probability to a new genealogy, pruning location and locus length can be expressed as the product of the locus length probability conditioned on the current genealogy (Expression 1, above), the pruning location probability conditioned on the current genealogy (Expression 2, above) and, the transition probability of the new genealogy conditioned on the current genealogy and pruning location (Expression 3, above).
2.1 Complete data transition densities
Consider the chain of local genealogies g = (g0, g1,…,gm-1) with recombination breakpoints at b = (0, b1,…,bm-1). According to the SMC′ process, the first local genealogy g0 follows the standard coalescent density: where and are the set of coalescent times in local genealogy g0. The piece-wise constant function Ai(t) denotes the number of ancestral lineages present at time t in genealogy gi, that is with .
Given a current local genealogy gi-1, the distribution of the length Si = Bi–bi-1 of the current locus depends on the current state of the SMC′ chain through the local genealogy’s total tree length li−1 (the sum of all branch lengths in gi-1) and the recombination rate per site per generation ρ. At recombination breakpoint bi, a new local genealogy gi is generated (Figure 1). This new local genealogy gi depends on the previous local genealogy gi-1 and the population size trajectory N(t). To generate gi we first randomly choose a pruning location pi (consisting of a pruning time ui and a lineage wi) uniformly along gi-1. At pruning location pi, we add a new lineage and coalesce it further in the past at time with some lineage, ci (Figure 2). We then delete the wi lineage’s segment from ui to (the coalescent time of lineage wi). The transition density to a new genealogy at recombination breakpoint bi is then where li-1 denotes the total tree length of gi-1.
This generative process of local genealogies can result in the two types of transitions depicted in Figure 2. A visible transition results in a genealogy gi which is different from gi-1 (Figure 2A), while an invisible transition makes gi identical to gi-1 (Figure 2B).
An invisible transition gi = gi-1, occurs when ci = wi. Given the pruning location pi = (ui, wi), a transition to an invisible event occurs when and Ci, the random variable indicating the lineage that coalesces with lineage , takes the value wi. The probability of an invisible transition is given by Thus, the joint transition probability to an invisible event with pruning location (ui, wi), given gi-1is:
4.2 Transition densities averaged over unknown pruning locations
Even though we will assume that local genealogies are known, in order to anticipate later applications to sequence data we do not wish to make the same assumption about pruning locations. Thus, we average over pruning locations to obtain marginal transition densities between genealogies, for both visible and invisible transitions.
To compute the marginal visible transition density to a new genealogy , we need to average over all possible pruning locations pi = (ui, wi) along gi-1. By comparing the two genealogies gi-1 and gi in Figure 2A, we know that pi corresponds to the lineage wi some time along , or equivalently, along . In general, comparison of gi-1 and gimay not provide complete information to identify the lineage that was pruned. When the children of the node corresponding to tdel and the children of the node corresponding to tnew are the same, pruning different branches can lead to the same transition. We enumerate all cases of incomplete information for visible transitions in Supporting Information Figure S1.
We introduce a function Ii-1(t), equal to the number of possible lineages at time t where the pruning location along gi-1 would produce a visible transition to gi. Ii-1(t) is a piece-wise constant function that takes the values in {0, 1, 2} depending on whether the pruning location pi can happen in 0, 1 or 2 branches at time t. In the example in Figure 2A, For a general Ii-1(t) piece-wise constant function that indicates the number of possible pruning branches at time t, the marginal visible transition density to a new genealogy is
Turning now to the computation of marginal transition probabilities for invisible events, we need to average over all possible pruning locations pi. Consider the example in Figure 2B and choosing a pruning time (ui) along gi-1. In order to have an invisible transition, the coalescing branch Ci must be the same pruning branch Wi. In Figure 2B the new coalescent time can happen along five lineages in the interval , three lineages in the interval , and two lineages in the interval . To generalize this calculation, we introduce the quantity with (n + 1) ≥ j ≥ k ≥ 2 which denotes the number of lineages in gi that are free (do not coalesce), in the time segment . The time interval includes the interval of pruning up to the interval of self-coalescence . Thus, if the pruning time happens at time, an invisible transition with new coalescent time can happen along free lineages.
In Figure 2B, ui happened in the time interval . If the new coalescent time happens in the interval along the same (unknown) pruning branch, then this invisible transition has probability with F5,5 = 5.
Now consider the same example of Figure 2B but with an unknown pruning time ui. The joint event where recombination occurs at pruning time and coalescent time occurs in the interval and this results in an invisible transition has probability: where denotes the double integral expression in Equation 9 for ease of notation.
An invisible transition would also result if and along the same (unknown) pruning branch; according to Figure 2B, this can happen along three lineages, so and this event has probability: If we continue considering the cases where and or we have and . Then, the joint probability of an invisible event and is For the cases when and the new coalescent time falls in another coalescent interval , we need to compute the following:
The joint probability of and no coalescence in the interval :
The probability of no coalescence in any of the intermediate coalescent intervals : and
The probability of coalescing at
Then, represents the probability that the pruning location is wi at time and the new lineage coalesces at time with lineage ci = wi. Overall, the marginal transition probability to an invisible event is:
2.3 The likelihood of the embedded SMC′ chain
Instead of having a complete realization of the embedded SMC′ chain of m local genealogies g0,…,gm-1 and pruning locations p1,…,pm-1 at recombination breakpoints b1,…,bm-1, we assume that our data (unless otherwise noted) consist only of m local genealogies at recombination breakpoints from a chromosomal segment of length L. Note that our observed data are not sequence data. More specifically, our observed data are Then, the observed data likelihood is where h(L − bm-1 | gm-1, ρ) is the survival function in state gm-1. Equation 13 is factored into terms that depend on N(t) alone and ones that depend on ρ alone. The terms that depend on ρ, given by Equation 5, depend on the data only through total tree lengths l0,…, lm−1 and locus lengths s1,…, sm-1, L − bm−1. By the factorization theorem for sufficient statistics, local tree lengths l0,…, lm−1 and locus lengths s1,…, sm−1, L − bm-1 are sufficient for inferring ρ.
Methods: Inference
Current coalescent-based methods that infer a population size trajectory N(t) from whole-genome data assume N(t) is a piece-wise constant function with change points x1 = 0 < x2 <… < xd (Li and Durbin 2011; Sheehan et al. 2013; Rasmussen et al. 2014; Schiffels and Durbin 2014). That is Equation 14 presents two challenges. The first challenge lies in the specification of the change points. The narrower an interval is, the higher the probability that we do not observe coalescent times in that interval. The fewer observed coalescent times in an interval, the greater the uncertainty of the estimate (if the estimate even exists). The second challenge lies in the specification of the time window (0, xd): if xd is set too far in the past, we might not have enough data to accurately estimate N(t) for xd≤t < ∞.
In order to solve the first challenge, Rasmussen et al. (2014) and Li and Durbin (2011) distribute the d change points evenly on a logarithmic scale: where κ is specified by the user. Schiffels and Durbin (2014) propose discretizing time according to the quantiles of the exponential distribution. where λ is the rate of an exponential distribution. Schiffels and Durbin (2014) model the time to the most recent coalescent event and set . However, Equation 16 is not directly applicable here because we use all coalescent events for inference.
In the following sections, we first present our Bayesian nonparametric method, then develop a maximum likelihood method under a piece-wise constant trajectory so we can directly compare an EM-based method (Li and Durbin 2011; Sheehan et al. 2013) to our Bayesian nonparametric method. The R code for all simulation studies and real data analysis conducted in this paper are publicly available at http://ramachandran-data.brown.edu/datarepo/.
3.1 Gaussian-Process-based Bayesian Nonparametric Estimation of N(t)
For our Bayesian methodology, we assume the following log-Gaussian Process prior on the population size trajectory, N(t): where 𝒢 𝒫(0, C(τ) denotes a Gaussian process with mean function 0 and inverse covariance function C-1(τ) = τ C-1 with precision parameter τ. For computational convenience, we use Brownian motion as our prior for f(t) since its inverse covariance matrix is sparse. We place a Gamma prior on the precision parameter τ, Assuming that recombination rate ρ is known, the posterior distribution of model parameters (Figure 3) is then The first two factors on the right side of Equation 18, detailed in Equations 8 and 11 involve integration over N(t), an infinite dimensional random function (Equation 17). We approximate the integral by the Riemann sum over a partition of the integration interval. That is, for xi < a < xi+1 <… < xk-1 < b < xk, Δi = xi+1 − a, Δk = b − xk-1 and Δj = xj+1 − xj for i < j < k. is a representative value of f(t) in the interval (xj, xj+1); in our implementation, we set with . This way, we discretize our time window in d evenly spaced segments x1 = 0 < x2 <… < xd, with , the maximum time to the most common ancestor observed in the sequence of local genealogies, and approximate N(t) by a piece-wise linear function evaluated at .
We condition on the set of m local genealogies g0,…,gm-1 to generate posterior samples for the vector and τ and use these posterior samples to infer N(t) at , where . Updating N(t) and τ separately is not recommended because of their strong dependency (Lan et al. 2015). Therefore, we update (N(t), τ) jointly in an MCMC sampling algorithm using Split Hamiltonian Monte Carlo (Shahbaba et al. 2014; Lan et al. 2015). Split Hamiltonian Monte Carlo relies on our ability to calculate the log-likelihood of the observed data and the gradient vector of the log-likelihood (i.e., the score function). The log-likelihood of the observed data is approximated via sums of the form in Equation 19. We approximate the score function ∇ℒobs(Y; f*) with respect to f* by applying Fisher’s identity: where, at each iteration in the MCMC, expectation is calculated using the current value of f*. We show the details of this calculation in the Appendix.
Alternatively, one can update N(t) in the MCMC algorithm using Elliptical Slice Sampler (Murray et al. 2010) with a fixed value of τ (perhaps estimated from previous studies or from a preliminary run from the Split Hamiltonian Monte Carlo algorithm). The advantage of using Elliptical Slice Sampler over the Split Hamiltonian Monte Carlo is purely computational since Elliptical Slice Sampler does not require calculation of the score function.
3.2 Maximum-likelihood estimation of N(t) with measures of uncertainty
We assume that the population size trajectory N(t) is defined as in Equation 14. The standard coalescent density (Equation 4) and the transition densities defined in Equations 11 and 8 are tractable, so calculation of the likelihood (Equation 13) is tractable. However maximization of the likelihood function cannot be performed analytically because pruning locations are missing. We rely on the Expectation-Maximization (EM) algorithm (Dempster et al. 1977) to find the maximum likelihood estimator of N = (N1,…,Nd). The complete data Yc for inferring N(t) are then the set of local genealogies g0,…,gm-1 and the set of pruning locations p1…, pm-1. For the invisible transitions, we also need to know the new coalescent times , where ℐ ⊂ {1, 2,…,m − 1} denotes the set of indices of invisible transitions (transition i is an invisible transition if gi = gi-1).
The complete data log-likelihood is then The EM algorithm starts by initializing the population size trajectory to a piece-wise constant function with change points x1,…, xd with arbitrarily chosen vector N0. At the kth iteration of the algorithm we set The conditional expectation in Equation 21 is conditional on the observed data Y defined in Equation 12. Let be the ordered set of time points corresponding to the change points x1,…,xd and the coalescent time points ti of local genealogy i. If the transition from gi to gi+1 is visible, we replace the jth time point by , where j corresponds to the index such that . For ease of notation, we will denote the number of time intervals |xi| by D = d + n − 2. Let be an indicator function that takes the value of 1 when the jth interval contains a coalescent time of the first genealogy g0. Then, the log density of the first genealogy is: Let be an indicator function that takes the value of 1 when the new coalescent time of genealogy i happens in the corresponding time interval , and let the adjusted interval length be
Then, the augmented transition density can be expressed as: where zi and Δ i are the vectors with and elements. For the EM algorithm we need to compute the conditional expected vectors and . The details of these calculation are in the Appendix.
We use the Fisher information matrix to compute approximate standard errors of and use these standard errors together with asymptotic normality of maximum likelihood estimators to produce confidence intervals for log population size piece-wise trajectories. We compute the observed Fisher information matrix following Louis (1982): where is the gradient and is the Hessian of the complete-data log-likelihood with respect to log N. This requires the calculation of conditional cross-product means and conditional second moments described in the Appendix.
Results
We simulated 1000 local genealogies of 2, 20 and 100 individuals from each of the three different demographic models described in Table 2 using MaCS (Chen et al. 2009); see Supporting Information for details of these simulations. We assumed that all individuals were sampled at time t = 0 under a demographic model in Table 2.
We compared the point estimates with the truth for each demographic model using the sum of relative errors (SRE): where is the estimated population size trajectory at time xi. We compute SRE at equally space time points x1,…,xK. Second, we compute the mean relative width (MRW) as follows: where corresponds to the 97.5% upper limit and corresponds to the 2.5% lower limit of . For EM estimates, corresponds to the 95% confidence interval estimated using the observed Fisher information; for Bayesian GP estimates, corresponds to the 95% Bayesian credible interval (BCI) of . To measure how well these intervals cover the truth, we compute the envelope measure (ENV) in the following way:
We compute SRE, MRW and ENV for K = 150 at equally spaced time points.
For our Bayesian GP estimates, we estimate N(xi) at d = 100 time points, unless stated otherwise. The parameters of the Gamma prior on the GP precision parameter τ were set to α = β = 0.001, reflecting our lack of prior information about the smoothness of the population size trajectory.
For our EM estimates, we used different discretizations based on Equation 15 and varying the number of change points d and κ over the fixed interval (0, xd) with xd set to be the maximum observed coalescent time. For the cases where we only consider one genealogy (m = 1), the EM approach becomes standard maximum likelihood estimation. We summarize our posterior inference and compare our Bayesian GP method to the EM method. The population size trajectory is logtransformed for ease of visualization and for direct comparison with other methods (Minin et al. 2008; Palacios and Minin 2013).
4.1 Sensitivity of EM estimates of N (t) to discretization
In Figure 4, we show our Bayesian GP and EM estimates of a constant population size trajectory from a single genealogy of 100 individuals with different discretizations. We find that our Bayesian GP point estimates depicted in Figure 4A recover the truth (dashed line) almost perfectly with less uncertainty than the EM (Figure 4B-C). Comparing our Bayesian GP estimates with different discretizations: 50, 100 and 200 equally spaced time points (Figure 4A), we find that increasing the number of time points improves inference (Table 4) but that the differences between estimates among the three discretizations are marginal (Figure 4A). In contrast, we show that different grid definitions alter the EM estimates (Figure 4B). It is not clear how to define a good strategy for the definition of the grid for the EM method, even for the simple model of constant population size. For example, increasing κ from 100 to 500 with 5 change points (Figure 4B), does not improve estimation. Increasing the number of change points does not necessarily improve the estimates either; for example, increasing the the number of change points from 5 to 10 for κ = 10 (Figures 4B-C). EM grid sensitivity is persistent even when the number of genealogies increases; Figure S2 in Supplementary Information shows that the best definition of change points when our data consist of 1000 local genealogies of 100 individuals has 10 change points evenly distributed.
4.2 Comparing Methods of Estimating N (t)
Figure 5 shows the estimated population size trajectories when the number of samples is 2 for the three different demographic scenarios and varying the number of local genealogies (100, 500 and 1000 local genealogies). For constant and exponential growth, our EM method assumes a piece-wise constant trajectory of 10 change points (d = 10) and κ = 1 using Equation 15 (similar to Li and Durbin (2011) and Rasmussen et al. (2014)). For the bottleneck scenario, some of the intervals did not have coalescent events; hence, for this case we assumed a piece-wise constant trajectory of 5 change points (d = 5) and κ = 1 for constructing our EM estimates. We show the boxplots of the time to the most recent common ancestor (TMRCA) at the bottom of each plot in Figure 5. The distribution of the TMRCA serves as an indicator of the uncertainty expected of our estimates. Both approaches, EM and Bayesian GP show narrower confidence and credible intervals at the center of the distribution of the TMRCA, particularly during the bottleneck in Figure 5C.
For the constant population demographic model in Figure 5A, our Bayesian GP outperforms our EM estimates considerably. This is not surprising since a priori log N(t) has mean 0 in our Bayesian approach (Equation 17). Moreover, EM confidence intervals only cover the truth constant population size around 30% of the time, while the GP method covers 100% of the truth (Table 4A). Despite placing a mean-0 prior on logN(t), the Bayesian GP method accurately recovers sudden changes as shown in the bottleneck example. Our Bayesian GP prior on log N(t) is Brownian motion which is not differentiable at any point; yet, our Bayesian GP recovers smooth curves (Figure 5B).
Table 4A shows the performance statistics for the estimates of N(t) in Figure 5. In general, our Bayesian GP has wider credible intervals than the EM confidence intervals but these credible intervals cover the true trajectory better than the EM confidence intervals in all the cases (MRW and ENV in Table 4). Our Bayesian GP estimates also generally have smaller sums of relative errors (SRE in Table 4). Under the bottleneck scenario, our Bayesian GP produces greater sums of relative errors than does the EM, but our Bayesian GP estimates recover the truth more accurately than the EM during the bottleneck.
Figures 6 and 7 show our estimates when n = 20 and n = 100. The performance statistics of the estimates displayed are shown in Table 4(B) and (C). In general, our GP-based estimates have smaller SRE and larger ENV than the EM-based estimates and hence, the MRW is usually wider in the GP-based estimates, accurately reflecting the uncertainty of the estimates. As expected, increasing the number of loci (m) generally decreases the width of the confidence and credible intervals of our estimates (MRW). Although this is generally true for EM estimates as well, EM estimates have very low coverage of the truth (MRE in Table 4) when the number of loci increases.
4.3 Sampling more individuals versus sequencing more loci
Figures 5-7 show our estimates for n = 2, 20 and 100 sampled individuals across varying numbers of loci. Since performance of EM estimates depends strongly on the definition of the grid, we base what follows on the Bayesian GP estimates. We find that increasing the number of loci, decreases uncertainty of our estimates and allows us to infer N(t) further back in time. Increasing the number of samples does not necessarily increase the performance of our GP estimates. For example, under the bottleneck scenario, we are able to detect the bottleneck phase fairly accurately even for two samples with m = 1000 local genealogies. Increasing the number of samples to n = 20 and n = 100 does not improve estimation of the features of the bottleneck. This is because most TMRCAs observed under the bottleneck scenario occur during the bottleneck (Figures 5,6 and 7), regardless of the number of individuals sampled. In contrast, in our exponential growth example, increasing the number of samples from n = 2 to n = 100 improves accuracy (point estimates are closer to the truth, see SRE in Tables 4A-C) and credible intervals cover the truth completely (ENV of 100%).
4.4 Sequential Tajima’s genealogies are sufficient statistics under the SMC ′
Under the SMC′ process, marginally at each locus along the chromosome, a local genealogy is a realization of Kingman’s n-coalescent (Kingman 1982), a continuous-time Markov chain taking its values in the set 𝒦n of partitions of the label set {1, 2,…, n}. A local genealogy g of n individuals includes labeled topology 𝒦n and coalescent times t = (tn,…,t2). The state space of a local genealogy is then 𝒢 = 𝒦n ⊗ R+n−1, and the cardinality of the set 𝒦n is n!(n − 1)!/2n−1. However, only the set of ordered coalescent times carry information about N(t). For a single locus, the set coalescent times are sufficient statistics for inferring N(t) (proof is in the Appendix). A natural question that follows is whether the coalescent times corresponding to the set of local genealogies are sufficient statistics for inferring N(t) under the SMC′ model. We find that the sufficient statistics for inferring N(t) under the SMC′ model, are the coalescent times, when taken together with local ranked tree shapes. For a single locus, the set of coalescent times together with the ranked tree shape correspond to a realization of Tajima’s n-coalescent. Tajima’s n-coalescent (Tajima 1983) is a continuous-time Markov chain taking its values in the set Hn of ranked tree shapes also called histories, evolutionary relationships or vintaged and sized coalescent (Sainudiin et al. 2014). The state space of Tajima’s local genealogy is then 𝒢T = ℋn ⊗ ℝ+n-1, and the cardinality of the set ℝn corresponds to the sequence of Euler zigzag numbers whose first ten elements are 1, 1, 1, 2, 5, 16, 61, 272, 1385, 7936 (Disanto and Wiehe 2013). The probability of getting a particular type of ranked tree shape Hn of n samples (Tajima 1983) is given by where c is the number of cherries, defined as branching events that lead to exactly two leaves.
In the Methods section, we defined transition densities in terms of coalescent times and Fi,j quantities. The set of all Fi,j quantities from a local genealogy form a triangular matrix: F-matrix. In the Appendix, we show that (i) F-matrices are in bijection with ranked tree shapes and (ii) the set of local Tajima’s genealogies are sufficient statistics for inferring N(t) under the SMC′ model. These observations are crucial for inferring N(t) from sequence data directly. Coalescent-based inference from sequence data rely on marginalization over the hidden state space of genealogies. In the Appendix, we show that the state space needed is the space of local Tajima’s genealogies, as opposed to the space of local Kingman’s genealogies. For n = 10 sequences, there are 2, 571, 912, 000 possible labeled topologies while only 7, 936 possible ranked tree shapes.
4.5 Application to human data
We applied our method to a 2-Mb region on chromosome 1 (187,500,000-189,500,000) with no genes from five Yorubans from Ibadan, Nigeria (YRI) and five Utah residents of central European descent (CEU) from the 1000 Genomes pilot project (1000 Genomes Project Consortium 2012) and previously analyzed for the same purpose (Sheehan et al. 2013). We used ARGweaver (Rasmussen et al. 2014) to obtain a sample path of local genealogies for the two populations (YRI and CEU). The parameters used are 200 change points, a mutation rate of μ = 1.26×10−8 and a recombination rate of ρ = 1.6 × 10-8 (Rasmussen et al. 2014, details regarding parameters used can be found in Supplementary Information). We note that ARGweaver assumes the SMC process and our method assumes the SMC′ process. Moreover, our inference is based on a single sample of the SMC process with known pruning times. Our ARGweaver set of local genealogies are discretized at 200 time points and our GP-based inference is influenced by this discretization. In Figure 8 we show our estimates of past Yoruban (in blue) and European population sizes (in green). The two population size trajectories experience a series of bottlenecks and overlap until about 100 YKA, assuming a diploid reference population size of N0=10,000 and a generation time of 25 years. In Figure 8 we recover an out-of-Africa bootleneck that starts about 100 KYA and ends about 30 KYA in the European population. These results are consistent with previously published results (Li and Durbin 2011; Gronau et al. 2011; Rasmussen et al. 2011; Sheehan et al. 2013; Schiffels and Durbin 2014). In Supplementary Information Figure S4 we show the estimates of logN(t) instead of logN(t) and time measured in units of N0 generations (same scaling as with simulations in Figures 5-7).
Discussion
In this paper, we propose a Gaussian-process based Bayesian nonparametric method for estimating effective population size trajectories N(t) from a sequence of local genealogies, accounting for recombination. Under a variety of simulated demographic scenarios and sampling designs, our method recovers the truth with better precision and accuracy than a maximum likelihood approach (Figures 5-7). We apply our method to genealogies estimated from human genomic data ARGweaver (Rasmussen et al. 2014) and conduct inference of the human population size trajectory for European and African populations; this application to real data recover the known features of the out of Africa bottleneck (Figure 8).
Several recent approaches have emerged for inference of population size trajectories from multiple whole-genome sequences using the sequentially Markov coalescent (SMC) (Li and Durbin 2011; Sheehan et al. 2013; Schiffels and Durbin 2014). However, current SMC-based methods rely on maximum likelihood inference (EM) of both a discretized parameter space and a discretized state space in order to gain computational tractability, and incur the costs of reduced accuracy and biased estimates. Although in principle the EM approach and the Bayesian nonparametric approach approximate N(t) similarly — by either a piece-wise constant or a piece-wise linear function — the Bayesian nonparametric approach is not affected by increasing the number of parameters (or change points) in the estimation of N(t). For comparison with existing methods, we implemented an EM approach to infer population size trajectories from a sequence of local genealogies and we note that increasing the number of loci may actually increase the bias of the EM estimates (Figures 5-7). For example, in simulation, our EM approach incorrectly detects the initial period of the simulated bottleneck (around 0.8N0 instead of 0.5N0 generations ago) with narrow confidence intervals (Figure 7C).
There are many advantages to using Bayesian GP over EM for inference of population size trajectories. Similar to Palacios and Minin’s (2013) approach to inference from a single genealogy, we a priori assume that N(t) follows a log Brownian Motion process. This allows us to model N(t) as a continuous positive function. The main advantage of using a Brownian Motion process is that its inverse covariance function is a sparse matrix that allows for fast computations. Since the likelihood function involves integration over N(t), this integral is approximated by the Riemann sum over a regular grid of points. The finer the grid is, the better the approximation. We find that our method performs well for inferring N(t) at 100 change points in all our examples and, more importantly, results are not sensitive to the number of change points used in the analysis (Figure 4). Our Bayesian approach relies on MCMC for inference from the posterior distribution of model parameters. Because population sizes at different grid points are correlated, we adapt the recently developed MCMC technique Split Hamiltonian Monte Carlo (splitHMC) for jointly sampling all model parameters (Shahbaba et al. 2014; Lan et al. 2015). splitHMC is a Metropolis sampling algorithm that efficiently proposes states that are distant from current states with high acceptance rates. It has been shown to be more efficient in inferring N(t) from a single genealogy than elliptical slice sampling or regular Hamiltonian Monte Carlo sampling(Lan et al. 2015). However, splitHMC relies on calculating the score function at every single iteration. Because pruning time in each local genealogy is unknown, we calculate the score function via Fisher’s formula.
In simulations, we find that our algorithm scales well with hundreds of individuals; our computational bottleneck is in the number of local genealogies. We envision that extending the current methodology to inference from sequence data directly will require a strategy for sampling shorter genomic segments. This would be a probabilistic alternative to arbitrarily choosing segment lengths (Sheehan et al. 2013; Rasmussen et al. 2014).
Under the SMC model, every recombination event along the genome translates to a new coalescent event for the sample under study, so increasing the number of loci results in more realizations of the coalescent process. The longer the segments are and the larger the number of samples taken, the greater the chance of observing variation due to recombination. This fact makes it hard to define a sampling strategy: longer genomes or larger sample sizes? We show that increasing the number of local genealogies improves precision of our Bayesian GP estimates (Figures 5-7). However, resolution into the past from contemporaneous sequences highly depends on the actual population size trajectory N(t).
We use ARGweaver (Rasmussen et al. 2014) to generate two samples of contiguous local genealogies corresponding to a 2-Mb region of chromosome 1 for five Europeans (CEU) and five Africans (YRI) from the 1000 Genomes Project; this genomic region is free of genes and was also analyzed in Sheehan et al. (2013). Taking these two samples of local genealogies as our data (4186 local genealogies for CEU and 6247 local genealogies for YRI), we were able to use our Bayesian GP method to infer Yoruban and and European effective population size trajectories (Figure 8). We find an out-of-Africa bottleneck that began ∼ 100 KYA and ended ∼ 30 KYA in the European population consistent with Li and Durbin (2011); Rasmussen et al. (2011); Gronau et al. (2011); Sheehan et al. (2013) and Schiffels and Durbin (2014). We note that our estimates are based on a single sample of local genealogies and thus ignore genealogical uncertainty. Moreover, we generated our data from the posterior distribution of local genealogies using ARGweaver at 200 time intervals so our GP-based approach cannot fully detect sudden changes that may occur between the discretized times. In addition, ARGweaver assumes an SMC prior model on local genealogies and our GP-based method assumes the SMC′ process; the lack of invisible recombination events in ARGweaver ‘s genealogies will bias inference.
The natural next extension for our method presented in this study is to infer N(t) from sequence data directly and not from the set of local genealogies. Our MCMC approach allows us extend the current methodology in a Bayesian hierarchical framework where the SMC′ process would be used as a prior distribution over local genealogies. The work we present here suggests a combination of ARGweaver accommodating SMC′ and GP priors would result in an efficient method for inferring population size trajectories from sequence data directly. In addition, our model can be easily modified to model a variable recombination rate along chromosomal segments and to jointly infer variable recombination rates and N(t).
Finally, we show that, under the SMC′ model, local ranked tree shapes and coalescent times correspond to a set of local Tajima’s genealogies; these Tajima’s genealogies are the sufficient statistics for inferring N(t). Under the SMC′ model, the state space needed for inferring population size trajectories from sequence data is that of a sequence of local Tajima’s genealogies. This lumping, or reduction of the original SMC′ process, will allow more efficient inference from sequence data directly.
Current methods for inferring population size trajectories make tradeoffs to analyze whole genomes that limit both biological understanding of sudden population size changes and the ability to test hypotheses regarding population size changes. This work represents a critical set of theoretical results that lay the groundwork for efficient estimation of detailed histories from sequence data with measures of uncertainty.
Acknowledgements
We thank Amandine Veber for her valuable suggestions and comments. We thank Shiwei Lan for his suggestions for speeding up the MCMC sampling scheme and Sara Sheehan and Melissa J. Hubisz for helpful discussions. J.A.P. acknowledges scholarship from CONACyT Mexico to pursue her research work. This research is supported in part by NSF CAREER Award DBI-1452622 (to S.R.). S.R. is a Pew Scholar in the Biomedical Sciences, supported by The Pew Charitable Trusts, and an Alfred P. Sloan Research Fellow.
Appendix A
Discretization
For both our Bayesian method and our EM method, we assume that N(t) is a piece-wise linear (or piece-wise constant) function with d change points. Let be the ordered set of time points corresponding to the change points x1,…,xd and the coalescent time points ti of local genealogy i. Then, we calculate all the factors needed for the observed data likelihood (Equation 13) and the complete data likelihood (Equation 20).
Let denote the discretized version of F i that represents the number of branches in gi that do not coalesce with any other branch in the time interval . Note that the indices here are in increasing order, k ≤ j. Similarly, let denote the probability that Ui (the time along genealogy i), occurs in and the self-coalescing event occurs at time in .That is where is the joint probability of pruning time and not coalescing back to the same branch in the time interval , and
Expectation-Maximization Algorithm
E-step: Equations 22 and 23 show that for the E-step, the only expectations we need are and . We compute these expression as follows:
For i ∈ ℐ and for i ∈ ℐc, let
Then where
M-step. Now, for the kth iteration of the algorithm and maximizing the complete data loglikelihood (Equation 20) we have where is an indicator function that takes the value of 1 when (xl, xxl+1) covers the interval .
Observed score function for Split Hamiltonian Monte Carlo
Our Bayesian approach relies on Split Hamiltonian Monte Carlo (splitHMC) to sample from the posterior distribution of model parameters. This method requires the calculation of the observed score function. We use Fisher’s identity and calculate the observed score function as the conditional expected complete score function. The lth element of ∇ℒobs is
Fisher Information Calculation
The calculation of the Fisher information needed to estimate confidence intervals of a piece-wise constant trajectory of population sizes, requires the following expected values:
For k < j and i ∈ ℐ and for k < j and i ∈ ℐc
For j < k and i ∈ ℐ and
For i ∈ ℐc and j < k where is as defined in Equation 30, and where and
For i ∈ ℐ and for i ∈ ℐc
The gradient vector of the complete data log-likelihood has lth element
With and Next, differentiating Equation (32), we have = 0 for all l ≠ m, so the Hessian is a diagonal matrix with (l, l)th element and where and
Also, where and for l < 0
Sufficient statistics under SMC′
Proposition 1.
For a single locus, the set of coalescent times are sufficient statistics for inferring N(t).
Proof. This can be proved using the factorization theorem. The marginal density of a local genealogy (Equation 3) has a unique factor that depends on N(t) and g only through tn,…,t2. The values of A(t) are induced by the natural order of the coalescent times.
Let F denote a lower triangular matrix of size n × n with the Fi,j entry the number of lineages that do not coalesce in the time interval (ti+1, tj), as defined in the methods section and with the following properties:
Fi,1 = 0 for all i = 1,…,n (The first column contains 0s for completion)
Fi,j = 0 for all j > i (Lower triangular matrix)
Fi,i = i for all i ≥ 2 (The diagonal corresponds to the number of lineages at each intercoalescent interval)
Fi,i−1 = i − 2 for all i ≥ 2 (At each intercoalescent interval, we loose two free lineages, so the second diagonal correspond to the number of lineages minus two)
For j < n − 1, the last row of F is defined according to:
Let c denote the number of cherries, then
For i < n and j < i − 1, if Fn,j−1 = Fn,j − 2, then Fi,j−1 = Fi,j − 2.
Let vi denote the set of lineages in the intercoalescent interval (ti, ti-1) with direct descendant internal nodes. The lineage labels correspond to the label of the coalescent time, when the direct descendant internal node was created. That is, the lineage created at tn has label n: vn = {n}; the lineage created at ti has label i. Let |vi| denote the size of the set vi. Note that 1≤ |vi| ≤c and
For i < n and j < i − 1, if Fn,j−1 = Fn,j − 1, then at time tj, there is a coalescence between a singleton and a lineage in the set vj. Let aj be the lineage selected uniformly at random
For i < n and j < i – 1, if Fn,j-1 = Fn,j, then at time tj, there is a coalescence between two lineages and from the set vj. Let denote the minimum and the maximum of the two lineages selected, then
We show the correspondence between a ranked tree shape and the F-matrix in the example of Figure A1. The first row and the first column are set to 0, the first two diagonals are known with probability 1: Fi,i = i and Fi,i-1 = Fi,i − 2 for i > 1. In our example, n = 5 and so, the first diagonal corresponds to (0, 2, 3, 4, 5) and the second diagonal corresponds to (0, 1, 2, 3). The last row F5, contains 0, followed by the number of branches that do not coalesce in the time intervals (t6, t2), (t6, t3), (t6, t4) and (t6, t5) corresponding to (0, 0, 2, 3, 5).
Proposition 2.
There is a bijection between the set of ranked tree shapes ℋn and ℱ, the set of F-matrices.
Proof. The probability of the F matrix can be expressed as the product of the conditional probabilities of the columns of the F matrix, that is: since the first and last column of F are known with probability 1. Note F.,j represents the j-th column vector of the F matrix.
Let di = Fn,i − Fn,i-1 for i = 3,…,n, and d2 = Fn,2 then That is, the conditional probability of the (n − j)th column of F given the (n − j + 1)th column of F is the product of the conditional probability of the last element of the (n − j)th column and the conditional probability of the rest of the (n − j)th column. When dn-j = 2 the rest of the column is known with probability 1 (property 7 of the F-matrix). When dn-j = 1, the rest of the n − jth column has probability 1/|vn-j+1| (property 9 of the F-matrix) and when dn-j = 2, the rest of the n − jth column has probability (property 10 of the F-matrix). Then re-writing Equation 33, we have since , and , then and Since dn = 2 and |vn| = 1, for j = n − 1, then dn-1 is either 1 or 2, then If we continue expanding the expressions, we get:
Note that the entries of the F matrix correspond to the same quantity needed to express the transition density of an invisible event (Equation 11). We claim that the sequence of coalescent times sets t0, t1,…,tm-1 and F 0,F 1,…,Fm-1 matrices corresponding to the ranked tree shapes of local genealogies g0, g1,…,gm-1 are sufficient statistics to infer N(t) under the SMC′ process. We prove this through the following propositions.
Proposition 3.
The probability density of Tajima’s genealogy is proportional, up to a combinatorial factor, to the probability density of Kingman’s genealogy.
Proof.
Proposition 4.
The marginal visible transition density from a local Kingman’s genealogy gi-1 to Gi is proportional to the marginal visible transition density from the corresponding local Tajima’s genealogy to
Proof. When the labeled topology of gi-1 is the same as the labeled topology of gi, then a transition from gi-1 to gi contains the same information about pruning location as a transition from to (Supplementary Information, Figures S1A and S2D). In fact, the Ii-1(t) function defined in section 2.1.2 (Equation 8) can be defined in terms of the F i-matrix and the coalescent times ti-1 and ti. In this case, for some j ∈ and . Then Hence, if Ki-1 = Ki, the labeled topologies of gi-1 and gi, then When the labeled topologies of gi-1 and gi are different, but the children of and the children of are the same, we cannot exactly identify the pruning branch and the new coalescing branch (Supplementary Information, Figure S1B) and then a transition from gi-1 to gi contains the same information about pruning location as a transition from to . Let and , since the children of and are the same, it is enough to consider Fi-1. Then and
Now, when the deleted node corresponding to tdel is a cherry and the new node corresponding to tnew is also a cherry, there are four possible topologies Ki that lead to the same ranked tree shape F i, then
Proposition 5
The marginal invisible transition density from a local Kingman’s genealogy gi-1 to Gi is equal to the marginal invisible transition density from the corresponding local Tajima’s genealogy to .
Proof. since all needed to compute the transition probability are the coalescent times and the Fi-1 matrix. Since the topology does not change, the proof follows.
Proposition 6.
The Likelihood of partially observed embedded SMC′ chain of local Kingman’s genealogies is proportional, up to a combinatorial factor, to the likelihood of partially observed embedded SMC′ chain of the corresponding local Tajima’s genealogies.
Proof. The proof follows from propositions 3, 4 and 5 needed to express the likelihood of partially observed embedded SMC′ chain (Equation 13).