PT - JOURNAL ARTICLE AU - Diala Abu Awad AU - Denis Roze TI - Epistasis, inbreeding depression and the evolution of self-fertilization AID - 10.1101/809814 DP - 2020 Jan 01 TA - bioRxiv PG - 809814 4099 - http://biorxiv.org/content/early/2020/02/16/809814.short 4100 - http://biorxiv.org/content/early/2020/02/16/809814.full AB - Inbreeding depression resulting from partially recessive deleterious alleles is thought to be the main genetic factor preventing self-fertilizing mutants from spreading in outcrossing hermaphroditic populations. However, deleterious alleles may also generate an advantage to selfers in terms of more efficient purging, while the effects of epistasis among those alleles on inbreeding depression and mating system evolution remain little explored. In this paper, we use a general model of selection to disentangle the effects of different forms of epistasis (additive-by-additive, additive-by-dominance and dominance-by-dominance) on inbreeding depression and on the strength of selection for selfing. Models with fixed epistasis across loci, and models of stabilizing selection acting on quantitative traits (generating distributions of epistasis) are considered as special cases. Besides its effects on inbreeding depression, epistasis may increase the purging advantage associated with selfing (when it is negative on average), while the variance in epistasis favors selfing through the generation of linkage disequilibria that increase mean fitness. Approximations for the strengths of these effects are derived, and compared with individual-based simulation results.