Abstract
The outcome of a major evolutionary transition is the aggregation of independent entities into a new synergetic level of organisation. Classical models involve either pairwise interactions between individuals or a linear superposition of these interactions. However, major evolutionary transitions display synergetic effects: their outcome is not just the sum of its parts. Multiplayer games can display such synergies, as their payoff can be different from the sum of any collection of two-player interactions. Assuming that all interactions start from pairs, how can synergetic multiplayer games emerge from simpler pairwise interaction? Here, we present a mathematical model that captures the transition from pairwise interactions to synergetic multiplayer ones. We assume that different social groups have different breaking rates. We show that non-uniform breaking rates do foster the emergence of synergy, even though individuals always interact in pairs. Our work sheds new light on the mechanisms underlying a major evolutionary transition.
1 Introduction
Major evolutionary transitions share cooperation as a common theme: simple units aggregate to form a new level of organisation in which individuals benefit others bearing a cost to themselves (1, 2). However, from a Darwinian perspective, cooperation is difficult to explain, as natural selection promotes selfishness rather than cooperation (3–7). The evolution of cooperation has often been approached through the lens of simple two-player games that depict social dilemmas (8–10). The study of games such as the Prisoner’s Dilemma, or alternative situations such as the Stag-Hunt game (11), have provided insightful views on which mechanisms are likely to promote cooperation — e.g., spatial reciprocity, direct reciprocity, indirect reciprocity, kin selection and group selection (12–14). However, the simplicity of two-player games is a double-edged sword, as these pairwise games may fail to capture the intricacies of complex interactions in real social and biological systems. Evolutionary transitions typically involve multiple interaction partners at the same time rather than a collection of pairwise interactions. For instance, when cells interact to form a multicellular organism, a superposition of pairwise interactions is insufficient to capture the intricacies of the complex organism. This is because an interaction among all the cells is not just a sum of pairwise interactions. Synergetic interactions — the whole being more than the sum of its parts — may be necessary to allow a high level of selection unit to emerge. Synergetic interactions could then pave the way for the emergence of complex phenomena such as division of labour or multicellularity. Therefore, understanding how synergetic interactions emerge is an important part of our understanding of evolutionary transitions.
General multiplayer games, which cannot be decomposed into pairwise interactions, can represent such synergy effects. They can display broader and richer dynamics than their traditional two-player counterparts (15–18). In particular, multiplayer games can exhibit payoff non-linearities and can thus account for the synergetic effects that are intrinsic to major evolutionary transitions. Although the emergence of synergetic interactions among multiple players is key to all major evolutionary transitions as aforementioned (19), we lack fundamental understanding on how such complex synergetic interactions occur in the first place. Here, we present a mathematical model that captures the emergence of synergetic multiplayer games from simple pairwise interactions.
2 Results
2.1 Model description
We consider a structured population of N individuals, assorted into l sets (20), each consisting of m individuals. Individuals can have a different number of set memberships and play one of two strategies, A or B. An individual accumulates the payoff through interactions within all the sets it belongs to. These interactions are always pairwise and the payoff depends on the set configuration, i.e., the number of individuals playing A and B in the set. At every time-step, either the strategy of an individual or the set structure is updated. With probability w, the strategy of an individual is updated. Two individuals are randomly chosen and one imitates the other’s strategy with a probability that increases with the payoff difference. Individuals with higher payoffs are more likely to be imitated (21, 22). With probability 1 – w, a set is randomly chosen. This set may break up with probability ki, — where i is the number of strategy A individuals in the set, ranging from 0 to m. As a consequence, ki determines the fragility of a set, which in turn, depends on the set composition (23). If the set breaks, a randomly chosen individual — which belongs to at least one other set — is expelled. In order to keep the size of the set constant, another random individual is then incorporated into the set (Fig.1).
Although our model is simple, it captures two fundamental aspects. First, the set structure mimics the social interactions which is an intrisic characteristic of biological systems and human societies. For instance, the set size could be based on the diffusion rate of the public goods secreted by cooperative cells (24, 25). The overall structure also allows to consider an organisation of arbitrary size, from a small family to a large assembly. Second, individuals only interact in pairs and payoffs are additive. In this case, the payoff of an individual is nothing but sum of the corresponding pairwise interactions. Thus, there are no imposed synergetic effects via the payoff accumulation process. Instead, it can only emerge from the dynamics of the population structure.
Pairwise games between two strategies
If the size of the sets is two, m = 2, the population structure is equivalent to a network, where a “set” becomes a “link” (Fig. 1). In this case, the accumulated payoffs can still be captured by a pairwise interaction, hence there is no synergetic multiplayer interactions (26, 27). Although our analytical framework is general enough to allow the study of any set size, we focus on the case where m = 3, that is, when the sets contain three individuals. Let us assume that for two individuals playing strategy A (B), each one obtains a payoff of a(d). Similarly, for two individuals playing different strategies, the individual using strategy A obtains the payoff b and the individual using strategy B obtains the payoff c. Given that there are three individuals in every set, the payoff of an individual within a set is determined by two pairwise interactions. Therefore, the payoff of an individual playing strategy A (B) in a set with j other individuals playing A is given by aj = aj + b(2 − j) (bj = cj + d(2 − j)). Note that for m = 3, j = 0, 1, 2. Given that the breaking probability of a specific set may depend on the number of A individuals within the set, the set dynamics allows for non-uniform breaking probabilities across the sets.
To demonstrate that our simple model can indeed capture the emergence of synergy, we consider two aspects: the accumulated payoff of both types and the evolutionary dynamics of the two strategies. We find that non-uniform breaking probabilities across the sets foster the emergence of synergetic multiplayer interactions. In other words, when the fragilities of the sets are non-uniform we find that (i) the expected accumulated payoff of both strategies is consistent with the one of a typical multiplayer game that cannot be decomposed into a pairwise game, and (ii) the evolutionary dynamics of the strategies exhibit two internal equilibria of selection, which is impossible in a two-player game.
The calculation of the average accumulated payoff in the general case is challenging, even though the model is simple. We overcome this problem by assuming that the probability with which the strategy is updated is small, w ≪ 1. Asa consequence, the set structure can reach its stationary state—which determines the accumulated payoffs—before a strategy update occurs. Importantly, the average accumulated payoffs for both strategies are consistent with the payoffs of the following 3-player game in a well mixed population, up to a positive rescaling factor (see SI Appendix, Section 2.1): Here ai/ki+1 is the payoff for an individual using strategy A when it meets i opponents using strategy A. Equivalently, bi/ki is the payoff for an individual using strategy B when it meets i opponents using strategy A. The payoff table in Eq.(1) has two important features. First, the derived multiplayer game is of the same size as that of the set. Second, the payoff entries are proportional to the product of the accumulated payoff in a set and its lifetime.
The evolutionary outcome of both strategies can be predicted by the replicator equation for a large class of microscopic imitation rules, if the population size is sufficiently large (see SI Appendix, Section 2.2). The replicator equation is given by where are the payoffs for strategy A and B of the 3-player game based on Eq. (1), and xA is the fraction of individuals using strategy A. In other words, the dynamics of the pairwise game under active set dynamics can be captured by a multiplayer game in a well-mixed population. The internal equilibria of this equation are the roots of the equation fA − fB = 0. Based on the initial fraction of individuals using strategy A, these equilibria determine where an infinite population would end up (28).
The above results on the accumulated payoffs and the evolutionary dynamics of strategies hold for any set fragilities (k0, k1, k2, k3). In the following, we apply these results to homogenous and heterogeneous set fragilities to address when and how synergetic interactions emerge.
Whenever the fragility of the sets is homogenous, k0 = k1 = k2 = k3, Eq. (1) is identical to the one of the original pairwise game, even though it is a 3-player game (see SI Appendix, Section 2.2). Therefore there is no synergy effect in the payoffs. Given that the replicator equation is equivalent to the one of the pairwise game, there is at most one internal equilibrium with the same position and stability. The upper panel of Fig. 2 shows the agreement between the analytical approximation and a simulation of the full model.
However, when fragilities are not homogeneous across the sets, Eq.(1) becomes a 3-player game, which cannot be decomposed into additive pairwise interactions (lower panel of Fig. 2). In this case, the payoff of an individual interacting with two opponents is not equal to the sum of the two pairwise interactions (Fig. 3). Consequently, the presence of non-uniform set breaking probabilities generates synergetic payoffs. Synergy emerges exclusively as a result of the evolutionary dynamics of the set structured population. In addition to this, the replicator equation has two internal equilibria, which is not possible in pairwise interactions (see lower panel of Fig. 2). Static random networks display similar effects (29). A necessary condition for the emergence of two equilibria is that the sign of the effective payoff difference ai/ki+1 − bi/ki changes twice with the increase of the number of opponents using strategy A, i (30, 31). A more detailed analysis on the conditions that lead to two internal equilibria can be found in SI Appendix, Section 2.2. If one of the two equilibria is stable, the other has to be unstable. Given this, our model can explain both the maintenance of biodiversity and phenotypic dominance within the same framework.
Pairwise games between n strategies
The model can be extended to account for an arbitrary number of strategies, n, instead of only two. In the pairwise interactions with n strategies or an n × n game, the non-uniform breaking probabilities also generate synergetic multiplayer interactions. Although the analytical calculations are more intricate due to the increased number of set configurations, we find that the payoff matrix of the emergent multiplayer game is consistent with the one of an n-strategy m-player game (SI Appendix, Section 3.1). Interestingly, the intuition behind these payoff entries is similar to the ones of the two-strategy case, as they still represent the product of the additive payoffs via pairwise interactions and the duration of that set. In addition to this, the n-strategy m-player game has, at most, (m − 1)n−1 isolated internal equilibria, where as the original n × n pairwise game has at most one equilibrium (SI Appendix, Section 3.2). The dynamics in our model are thus rich enough to capture complex phenomena exhibited by social and biological systems.
3 Discussion
Synergy refers to the idea that the whole is greater than the sum of its parts. Interestingly, synergy is identical to “cooperation” in ancient Greek. Synergy can be observed in a plethora of different contexts such as in genes (32), microbial populations (33), and even social and economic systems. From an evolutionary perspective, synergy is a cornerstone of all major evolutionary transitions. These evolutionary milestones involve the aggregation of simple units into a new entity which becomes a higher-level Darwinian unit of selection (1). With this in mind, we present a minimalistic model that shows how synergy can actually emerge. Our model allows to treat the emergence of synergetic interactions from simple additive pairwise ones analytically. We assume that the strategy of the individuals and the set structure evolve in time. The results prove that non-uniform set breaking rates, which depend exclusively on the configuration of these sets, lead to payoff non-linearities. These are consistent with the dynamics of a multiplayer game, even though individuals always play a two-player game and no group selection effects are present. These findings rely on two conditions: i) sets must contain more than two individuals, and ii) the breaking rates must depend on the configuration of the sets and, hence be non-uniform. As a consequence, our model may be useful as a starting point for the investigation of the evolution of more complex phenomena, e.g., synergetic interactions within the group. It shows how the aggregation of individuals can lead to complex interactions that cannot be disentangled into simpler interactions.
Methods
The Fermi updating rule. We use the Fermi update rule, given by the following algorithm:
Randomly select an individual, a* and denote its payoff as πa*;
Randomly select another individual, b*, among all the individuals in the sets individual a* is in and denote the payoff of b* as -πb*;
a*switches to the strategy of b* with probability (1 + exp[−β(πb* − πa*)])−1.
Accumulated payoffs. Each data point is the average of 100 independent realisations. Every realisation takes 106 generations. In each realisation, for the first 104 generations, only set dynamics occur. For the rest generations, at every step, with probability w = 10−3 we compute the average accumulated payoff of each strategy. Otherwise, with probability 1 − w, set dynamics happens. At the end of each realisation, we compute the mean value of all the average accumulated payoff. Selection gradient. Each data point is the average of 100 independent realisations. Every realisation takes 107 generation. For the first 107 generations of each realisation, only set dynamics occur. After that, with a probability of w = 10−3 two individuals are chosen randomly from the entire population. The first individual is the “focal” one which is the one that may imitate the strategy of the second one based on the Fermi rule. We keep track of the transition without implementing them. We denote y and z as the number of times that an individual playing strategy A and B changes its strategy. is the estimator of the selection gradient , where Q is the number of strategy updating events in this realisation.
Accumulated group payoffs. Each data point is the average of 100 independent realisations. Every realisation takes 106 generations. In each realisation, for the first 104 generations, only set dynamics occur. For the rest generations, at every step, with probability w = 10−3 we compute the average accumulated payoff of each strategy induced by the set with 0, 1 and 2 strategy A opponents, respectively. Otherwise, with probability 1 − w, set dynamics happens. At the end of each realisation, we compute the mean value of all the average accumulated payoff.
Acknowledgements
We thank Chaitanya Gokhale for inspiring discussions.