## Abstract

High mutation rates select for the evolution of mutational robustness where populations inhabit flat fitness peaks with little epistasis, protecting them from a mutational meltdown. Recent evidence suggests that a different effect protects small populations from extinction via the accumulation of deleterious mutations. In drift robustness, populations tend to occupy peaks with steep flanks and positive epistasis between mutations. However, it is not known what happens when mutation rates are high and population sizes are small at the same time. Using a simple fitness model with both variable epistasis and variable mutational effect size, we show that the equilibrium fitness has a minimum as a function of the parameter that tunes epistasis, implying that this critical point is an unstable fixed point for evolutionary trajectories. In agent-based simulations of evolution at finite mutation rate, we demonstrate that when mutations can change epistasis, trajectories with a subcritical value of epistasis evolve to decrease epistasis, while those with supercritical initial points evolve towards higher epistasis. These two fixed points can be identified with mutational and drift robustness, respectively.

## Introduction

When a population is in evolutionary balance, it is able to maintain its fitness while at the same time exploring ways to increase its fit to the environment via adaptive mutations [1]. However, this balance between the evolutionary forces of selection and mutation can sometimes be precarious. When mutation rates become too high, for example, mutations can overpower selection leading to the extinction of populations via mutational meltdown [2]. Similarly, when population size dwindles, selection can become so weak that deleterious mutations cannot be eliminated anymore, leading to extinction via Muller’s ratchet [3]. It turns out that populations can adapt to these threats, for example by evolving mutational robustness [4], or by evolving resistance to the effects of drift [5, 6, 7]. Mutational robustness is achieved by a genetic architecture that implies a flat fitness peak, so that a large fraction of mutations are either neutral or have a small fitness effect [8, 9], while giving up wild-type fitness in return. Robustness to drift, on the other hand, appears to involve favoring fitness peaks that have steep flanks, enabled by mutations that are synergistic in their deleterious effect [5, 6], while *reducing* (rather than increasing) the likelihood of mutations with small effect, and increasing the fraction of mutations that are lethal. Interestingly, a recent re-analysis of the “survival-of-the-flattest” effect (that enables mutational robustness) has shown that an increase in the fraction of lethal mutations is also seen in the response to high mutation rates (see also [7]), suggesting that resistance to drift and to mutational meltdown are intertwined.

Here we study the impact of epistasis on both drift and mutational robustness in a simple model fitness landscape and show that whether a population predominantly displays drift or mutational robustness is largely determined by the average value of directional epistasis: populations occupying a peak with synergistic epistasis above a critical value will tend to evolve towards drift-robust peaks (by moving towards peaks with increased positive epistasis), while those inhabiting peaks with sub-critical epistasis will respond by lowering epistasis until mutations are mostly neutral, consistent with mutational robustness. Thus, evolutionary trajectories for populations under evolutionary stress will bifurcate towards drift-robust or mutationally-robust fixed points.

## Model

We study a simple fitness landscape in which peaks have unit wild-type fitness, and the fitness of a *k*-mutant is determined by
where *s* = − log *f* (1) is the mean effect of a deleterious mutation, *q* determines the degree of directional epistasis, and genotypes have a finite number of binary loci *L* (see, e.g., [10])^{1}. In such a model *q* = 1 signals absence of epistasis (a multiplicative landscape), *q >* 1 describes a peak with synergy between deleterious mutations, while *q* < 1 is indicative of buffering mutations (antagonistic epistasis). When *q >* 1 we sometimes speak of *negative* epistasis (because the combined two-mutant fitness is lower than the multiplicative expectation), while *q* < 1 indicates *positive* epistasis (the double-mutant is higher in fitness than expected on the basis of the single-mutation effect).

The equilibrium fitness of a population in mutation-selection balance can be calculated in the weak mutation limit *Nµ* ≪ 1, where *k*-mutants either go to fixation or are lost one-by-one, by averaging over the stationary distribution of *k*-mutants

The stationary distribution *p*_{k} is easily calculated using the fixation formula of Sella and Hirsh [12] (which is a more accurate estimate than Kimura’s celebrated formula) as

In the expression above, *N* is the population size, and *Z* is the partition function
and we used the Sella-Hirsh fixation probability appropriate for a haploid Wright-Fisher process. For *q* = 1 we can obtain a closed-form expression for the equilibrium fitness,
that shows clearly the steep fitness drop with decreasing population size that is due to genetic drift. But different values for *q* affect the fitness drop differently. In Fig. 1A, we can see the dependence of *f*_{eq} on the population size for the multiplicative model (*q* = 1), a model with positive epistasis (*q* = 2.0), as well as the case of negative epistasis (*q* = 0.5), evaluated at *L* = 100. The model suggests that while positive epistasis protects from a fitness drop for moderate population sizes (higher mean equilibrium fitness), the drop becomes severe once populations dwindle below 100. In fact, plotting *f*_{eq} against *q* as in Fig. 1B reveals a fitness *minimum* as a function of *q*, suggesting that fitness loss via drift can be prevented in two different ways: high positive epistasis or high negative epistasis, while populations with weak or no epistasis appear to be the most vulnerable.

### Two regimes: selection and neutral drift

The minimum in mean equilibrium fitness apparent in Fig. 1B can be seen as interpolating between two regimes: the *neutral drift* regime defined here by the limit of small *q* ≪ 1, and the *selection* regime *q* ≫ 1. In the selection regime, an organism’s fitness declines rapidly with increasing number of mutations, and this rapid decline effectively limits the maximum number of mutations an organism can carry. By contrast, in the neutral drift regime even organisms with a very large number of mutations have a fitness close to the maximum possible value of 1.

When *q* ≪ 1, the fitness landscape is essentially flat, and dynamics are dominated by neutral drift on that landscape. We can estimate the mean equilibrium fitness in the neutral regime by using
that is, the distribution Eq. (3) but with *f*_{k} ≡ 1. The result is the dashed line in Fig. 2, which agrees with the full model for *q* < 1. Note that in this derivation we set *f*_{k} ≡ 1 only in Eq. (3) but not in Eq. (2). The idea is that in the drift regime fitness differences are sufficiently small that they have no influence on the mutant distribution *p*_{k}. This does not mean, however, that all organisms have a fitness of 1.

On the other hand, when epistasis between deleterious mutations is synergistic (*q* ≫ 1), the number of mutations that a population can sustain before it goes extinct is limited to some number *k*_{max}. We can model this limitation by imposing a maximum number of mutations *k*_{max},
where *k*_{max} *< L*. This truncated mutation model agrees with the full solution for large *q* (Fig. 2, dotted line).

One of the most striking features of the interplay between the neutral regime and the selection regime is the appearance of a minimum mean fitness (as a function of epistasis) where the drop of fitness is largest. The location of this minimum *q*^{⋆} (reflecting the amount of directional epistasis that leads to the largest fitness loss) depends on the population size, the mean deleterious effect of mutations, as well as the number of loci (Fig. 3).

To estimate the epistasis coefficient at which the steady-state fitness is at its minimum, we analyze the stationary distribution of fitness (3), which is determined by the combined effects of neutral drift (represented by the term ) and selection, via the fitness . The binomial coefficient of neutral drift for any *k* is typically much larger than 1, whereas the selection term is much smaller than 1. To estimate the amount of epistasis where these two components balance each other we maximize neutral drift by setting *k* = *L/*2, so that the condition for maximal fitness loss (where drift maximally balances selection) becomes

We can solve this equation for *q* by using the Stirling approximation to expand the binomial coefficient, and obtain for the minimum *q** that

We can test this estimate by comparing it to the numerically inferred minimum obtained via numerical simulation, described below. Figure 3 shows that our estimate of *q** is very close to the numerically determined minimum for various population sizes and selection strengths.

### Increased mutation rate exacerbates fitness loss in neutral regime

The theoretical results shown above were derived in the weak mutation limit where every mutation is either lost or goes to fixation before another mutation occurs in the population. In this section we study how finite mutation rates modify those results.

We simulate finite populations on a single-peak fitness landscape at finite mutation rates *µ* using stochastic simulation. The population evolves asexually, and the population size is held constant over time for all simulations. For each combination of mutation rates and epistasis parameters, we simulated populations sizes *N* = 10 and *N* = 100, as well as selection coefficients *s* = 0.01 and *s* = 0.001. We recorded the mean fitness of the population over a period of time after a population reached steady-state (see Methods), as a proxy for this equilibrium fitness *f*_{eq}. The simulations of the evolutionary process on the fitness landscape defined by Eq. (1) recover the theoretical results well for small mutation rates, as expected. As the mutation rate increases, we see notable departures from the weak mutation limit for the selection regime (larger *q*), while the neutral (drift) regime is largely unaffected by the increased rates (Fig. 4).

In particular, we notice that the minimum of the equilibrium fitness shifts towards higher *q* (Fig. 4). Furthermore, while for small mutation rates an increased epistasis protects from the loss of fitness due to genetic drift (mean fitness does not drop appreciably), it is clear that higher mutation rates negate this effect, and instead exacerbate the loss of fitness. Indeed, the increased mutation rate mimics the effect of a smaller population size (see Fig. 1B), which is expected as the effective population size decreases with mutation rate.

While the depressed equilibrium fitness suggests that there are two routes to withstand genetic drift at small population sizes, it is not clear whether evolutionary trajectories could indeed bifurcate.

### Bifurcation analysis of survival strategies

The minimum in *f*_{eq} at *q*^{⋆} suggests that if *q* were a dynamical variable, then *q*^{⋆} represents an unstable fixed point of the evolutionary dynamics. While *q* is not a dynamical variable in the usual sense, we can simulate it by endowing each genotype with a particular value of *q* that can be changed via mutation. In such a simulation, the statistics of the mutational process affecting *q* (the rate of change *µ*_{q} as well as the mean change per mutation Δ*q*) matter, so we test multiple different values for each.

It is worth pointing out that a genotype-dependent *q* appears to contradict the idea of fitness optimization in a landscape with a fixed fitness function such as Eq. (1). Such a function suggests that as a population climbs this peak, the parameters *q* and *s* are unaffected by this climb. While this is true for such a simple fitness function, it does not hold for more realistic evolutionary landscapes (for example in digital life [13, 14, 10, 15]), where the mean effect of mutations *s* and the directional epistasis *q* are not fixed properties of the landscape, but instead emerge as properties of the local neighborhood in genetic space. As a consequence, moving in this space (via mutations) will affect both *s* and *q*. We attempt to simulate part of that dynamics by allowing *q* to adapt (while keeping *s* fixed). If selection favors a particular value of epistasis, we should see a gradual change in the mean epistasis of a population.

In Fig. 5, we show how the mean epistasis parameter (averaged over sequences in the population) changes over time when populations are seeded with different seed organisms with fixed initial *q*. A bifurcation is indicated when trajectories move to different future fixed points given different initial states. While we can see clear signs of a bifurcation when plotting the mean trajectory in *q*-space over time (Fig. 5), viewing each trajectory separately reveals significant variation among them. In particular, for trajectories that are initialized with a *q* above the fixed point, some trajectories still move towards the low-*q* fixed point, which results in the mean of trajectories to appear constant (for example, in Fig. 5C). We discuss this phenomenon below.

## Discussion

The dynamics of evolution in asexual population is well-understood in the common population-genetic limits, namely vanishingly small mutation rate and large population (weak mutation and strong selection). When mutation rates are high and selection is weak, the classic theoretical results are undermined by new effects such as mutational robustness (effect of large mutation rate) and drift robustness (effect of small population size), as anticipated by generalized population-genetic models such as “free-fitness” evolution [16, 17, 12, 18].

In most realistic populations, we expect both effects to contribute. For example, when the mutation rate is large, the effective population size is diminished, so that both mutational and drift robustness are bound to be intertwined. The mean directional epistasis between mutations plays a role in both effects. While fitness peaks for mutationally robust populations tend to be flatter with little epistasis between mutations, we also observe something akin to truncation selection [8, 9]. In drift robustness, we observe both an increase in neutral mutations as well as an increase in strongly deleterious and lethal mutations, mediated by strong negative epistasis (*q >* 1).

Here we calculated the mean equilibrium fitness of a population in the limit of small mutation rates using a simple fitness function with variable epistasis and tunable mutation effect-size, and found a minimum as a function of the mean directional epistasis parameter *q* that depends on population size. Stochastic simulations of adaptation on this landscape suggest that the minimum also depends on mutation rate. The model further suggests that there are two attractive fixed points for evolutionary dynamics, namely small *q* where mutations become nearly neutral, and large *q* where deleterious mutations interact synergistically. The low-*q* fixed point^{2} can be identified with mutational robustness (*q* ≈ 0). In contrast, the *q >* 1 fixed point is reminiscent of drift robustness. While the existence of a minimum in equilibrium fitness is suggestive of an unstable fixed point *q*^{⋆} at which evolutionary trajectories bifurcate towards a low-*q* and a high-*q* fixed point, an agent-based simulation of such trajectories in a bit-string fitness model implementing Eq. (1) but with variable *q* paints a more complicated picture.

It is clear from inspection of Eq. (1) that for every single sequence, a reduction of *q* while keeping *k* constant increases fitness (as *∂f/∂q* < 0), no matter what the mean *q* of the population. This means that the population will sense an evolutionary pressure to reduce *q* independently of the mean population epistasis. However, if *q > q*^{⋆}, a secondary selective pressure appears that acts via the fitness distribution of a sequence’s offspring. For sequences with *q > q*^{⋆}, sequences with higher *q* have on average off-spring with higher fitness than those with lower *q*, leading to a second-order selective pressure to increase *q*. However, in any particular fitness trajectory, there is a chance that a sequence with *q < q*^{⋆} is among the offspring. Such a sequence may then go to fixation and abrogate the evolutionary trajectories leading towards a *q > q*^{⋆}, even though the selective pressure towards higher *q* is still present. We also expect that the likelihood of mutations that create sequences with *q < q*^{⋆} in the off-spring distribution depends on *µ*_{q} as well as Δ*q*. This is precisely what we observe in Figs. 5 and 6: while for small *q < q*^{⋆} the trend towards the mutationally robust fixed point is evident, at *q > q*^{⋆} the mean epistasis across 10 replicate experiments often shows a decrease (or remains constant) even though theoretically we expect an approach towards the drift-robust fixed point. The distribution of fitness trajectories shown in Fig. 6 shows that while some trajectories indeed move towards higher *q*, the possibility of mutating towards *q < q*^{⋆} leads to trajectories in which the secondary selective pressure towards higher *q* is muted. Indeed, trajectories towards *q > q*^{⋆} are absent among the replicates with initial *q < q*^{⋆}, reinforcing the conclusion that a critical amount of epistasis separates a population’s response to evolutionary stress either in a mutationally-robust, or a drift-robust manner.

While it is difficult to extrapolate results obtained using the abstract fitness function (1) to more complex landscapes in which many different peaks with different effect sizes and directional epistasis exist at the same time, our results support the notion that mutational robustness and drift robustness are indeed two different effects, which are likely to be intertwined in realistic scenarios.

## Methods

### Evolutionary model with fixed epistasis

For all simulations, we implemented an individual-based bit string model. Each individual was represented with a number of deleterious mutations it possesses. The number of deleterious mutations is *k* and *k* = 0, 1, *…, L. L* is the maximum number of deleterious mutations allowed in a population. A population was represented as a vector *V* of a length *L* + 1. Each bin *V*_{k} within a vector corresponded to the number of individuals with deleterious mutations *k*. The fitness of an individual could be determined with equation , where *s* is the selection coefficient, and *q* is the epistasis coefficient. At each time point, a population replicates and mutates. A replication event increases or decreases the number of individuals within each bin. The number of offspring was drawn from a multivariate distribution, and the probability of replication for each bin was determined by the number of individuals within a bin and a bin’s fitness. For all replication events, the population size was held constant. After replication, a mutation event moves individuals up or down a bin. For most simulations, the maximum number of moves up or down a bin was set to 3. When mutation rate was set to 1, the maximum number of moves was set to 4. The probability of mutating was calculated with

Here, *u* is per site mutation rate, *k* is number of mutation an individual has at time *t*, and *j* is the number of mutations an individual has at time *t* + 1. Using the probabilities calculated from the equation above, the number of individuals that would move or stay were drawn from a multivariate distribution. We simulated population sizes of 100 and We set selection coefficients to 0.01 and 0.001. We use the mutation rates of 1, 0.1, 0.01, 0.001, and 0.0001 (mutation rate is defined as the expected number of mutations per genome per duplication). For each combination of population size, selection coefficient, and mutation rate, we simulated 10 replicates. After a population has reached an equilibrium at *t* = 1, 500, 000, we calculated equilibrium fitness by taking the mean of population fitness over the next 1,000,000 time steps.

### Evolutionary model with evolving epistasis

Similarly to simulations with fixed epistasis, we implemented an individual-based bit-string model to simulate populations with evolving epistasis. A population was represented with two vectors. The length of each of the vectors corresponded to the size of the population, and each element within a vector represented an individual. The first vector contained a number of mutations (*k*) an individual possesses, and the second vector contained an epistasis coefficient (*q*) an individual’s genome experiences. The fitness of an individual could be determined with equation . At each time point, a population replicated and mutated. We replicated a population by creating a new generation with Wright-Fisher model. We mutated a population with a two step process. First, an individual either gained a mutation (*k* + 1), lost a mutation (*k −* 1), or did not mutate. The probability of mutating was calculated with equation 10. Second, each individual’s epistatis mutated. If epistasis was set to change, it could equally likely increase or decrease by a fixed amount (Δ*q*). We set Δ*q* to remain the same across generations per one trajectory of an evolving population. However, epistasis was only set to evolve after a population has reached an equilibrium (after *t* = 200, 000).

## Acknowledgements

This work was supported in part by the National Science Foundation’s BEACON Center for the Study of Evolution in Action, under contract No. DBI-0939454.

## Footnotes

↵

^{1}The present model in which fitness declines as a function of genetic distance from the wild-type (modulated by epistasis) gives rise to conclusions similar to what Fisher’s geometric model would predict, even though in Fisher’s model the distance from wild-type is phenotypic rather than genetic [11].↵

^{2}Note that while technically the low-*q*fixed point is*q*= 0, this value cannot be attained in any realistic population as such a landscape is completely neutral (*w*= 1) in this limit.