Extracting diamonds: Identifiability of 4-node cycles in level-1 phylogenetic networks under a pseudolikelihood coalescent model

Abstract

Here, we investigate the (generic) identifiability of the position of the hybrid node in a 4-node hybridization cycle in a semi-directed level-1 phylogenetic network. While generic identifiability is easily attained under non-restrictive assumptions such as t in (0,∞) for all branches and γ in (0,1) for the inheritance probability of the hybrid edges, simulations show that accurate detection of these cycles can be complicated by inadequate sampling, small sample size or gene tree estimation error. We identify practical advice for evolutionary biologists on best sampling strategies to improve the detection of this type of hybridization cycle.