RT Journal Article SR Electronic T1 Estimating probabilistic dark diversity based on the hypergeometric distribution JF bioRxiv FD Cold Spring Harbor Laboratory SP 636753 DO 10.1101/636753 A1 Carlos P. Carmona A1 Robert Szava-Kovats A1 Meelis Pärtel YR 2019 UL http://biorxiv.org/content/early/2019/05/15/636753.abstract AB The biodiversity of a site includes the absent species from the region that are theoretically able to live in the site’s particular ecological conditions. These species constitute the dark diversity of the site. Unlike present species, dark diversity is unobservable and can only be estimated. Most existing methods to designate dark diversity act in a binary fashion. However, dark diversity is more suitably defined as a fuzzy set—in which the degree of certainty about species membership is expressed as a probability.We present a new method to estimate probabilistic dark diversity based on the hypergeometric distribution. The method relies on co-occurrences to infer the strength of the association between pairs of species and assign probabilistic adscription to dark diversity to absent species. We compare it with two established methods to estimate dark diversity (Beals index and favorability correction). To test the methods, we created simulations based on individual agents in which the suitability of each species in each site is known. We compared the ability of the methods to accurately predict suitability and the size of dark diversity, and compared their sensitivity to data availability. Further, we assessed the methods in two real datasets with nested sampling designs.Our simulations revealed that predictions of the Beals method were extremely sensitive to species frequency, and predicted suitability poorly. The Favorability transformation corrected this relationship, but did still predicted extremely low probabilities for species with very little information. The Hypergeometric method outperformed the Beals and Favorability methods in all considered aspects in the simulations and displayed better characteristics in the real datasets.Probabilistic consideratiosn of biodiversity will help to acknowledge the uncertainty associated with ecological information. Although the Beals method has been described as the best estimator of dark diversity, it should be preferred only when the goal is to predict future apperances of species. However, studies on dark diversity should focus on the ecological affinities of species. The Hypergeometric method is the most promising method to estimate probabilistic dark diversity and species pool composition based on co-occurrences.