Abstract
Interpretations of values of the FST measure of genetic differentiation rely on an understanding of its mathematical constraints. Previously, it has been shown that FST values computed from a biallelic locus in a set of multiple populations and FST values computed from a multiallelic locus in a pair of populations are mathematically constrained by the frequency of the allele that is most frequent across populations. We report here the mathematical constraint on FST given the frequency M of the most frequent allele at a multiallelic locus in a set of multiple populations, providing the most general description to date of mathematical constraints on FST in terms of M. Using coalescent simulations of an island model of migration with an infinitely-many-alleles mutation model, we argue that the joint distribution of FST and M helps in disentangling the separate influences of mutation and migration on FST. Finally, we show that our results explain puzzling patterns of microsatellite differentiation, such as the lower FST values in interspecific comparisons between humans and chimpanzees than in the intraspecific comparison of chimpanzee populations. We discuss the implications of our results for the use of FST.
Subject Areas statistical genetics
Competing Interest Statement
The authors have declared no competing interest.