Original ArticlesA Monte Carlo method to account for sampling error in multi-species indicators
Introduction
In order to realize the international ambition to slow and eventually halt the ongoing global decline in biodiversity, as expressed in the context of the Convention on Biological Diversity (Butchart et al., 2010, Secretariat of the Convention of Biological Diversity, 2014), it is indispensable to have reliable instruments to measure progress towards set targets. Biodiversity indicators are increasingly used to monitor trends in biodiversity at various habitats and scales (Biala et al., 2012, Butchart et al., 2010, Szabo et al., 2012, Van Strien et al., 2016), the most popular being the combined population trends of individual species (Brereton et al., 2011, Freeman et al., 2001, Gregory et al., 2005, Loh et al., 2005). Such multi-species indicators (MSI) have the advantage of being relatively insensitive to the fluctuations of individual species, thus helping scientists, conservationists and decision makers to better understand the dominant factors influencing biodiversity in a region, country, continent or the entire biosphere. Until now the development of MSIs has mainly focused on methods to calculate the mean index of species, of which the geometric mean of species indices appears one of the most appropriate to use (Buckland et al., 2005, Buckland et al., 2011, Lamb et al., 2009, Van Strien et al., 2012). Popular examples of MSIs include the global Living Planet Index (Collen et al., 2009, Loh et al., 2005), the European Grassland Butterfly Indicator (Van Swaay et al., 2013), and the European Wild Bird Indicators (Gregory et al., 2005, Gregory and Van Strien, 2010).
The usefulness of MSIs and trends in MSIs is strongly increased if accompanied by proper measures of uncertainty. Without these, it becomes problematic to test whether changes in the indicator are statistically significant and/or to test the found trend against other indicators. The main sources of uncertainty in MSIs are sampling error and process noise. Sampling error refers to the uncertainty of the species indices, which in most monitoring programmes must be considered as “sampling error in a broad sense”: the “pure sampling error” caused by sampling only part of the population, complemented by sources of variation like measurement bias, imperfect detection and missing values. This part of the variation in time series is also called “observation error” (e.g. Dennis et al., 2006). Process noise refers to the interannual variation between indices, the “process” being the trend in population numbers which usually is the main objective of a monitoring programme. Surprisingly, although the sources of uncertainty of MSIs are theoretically well-known it often proves a challenge to construct confidence intervals (CIs) for both MSIs and trends therein that take into account both sampling error and process noise. We know of three common methods, none of which is completely satisfying:
- (1)
CI based on bootstrapping across species
In this approach (for instance Collen et al., 2009, Craigie et al., 2010, Eaton et al., 2016) the trend of each species is considered as a replicate of the MSI. This approach is useful to assess the robustness of the MSI against species selection, but it neglects sampling error in the species indices. In addition, it suffers from a conceptual drawback: it is questionable to include interspecific variation in the confidence intervals of MSIs. The rationale of testing against variation between species is that the species are randomly sampled from a large group, but this rationale is unjustified as species to represent an MSI are typically deliberately selected. In addition, bootstrapping species may yield wide confidence intervals if the trend of even a single species deviates from the trend of the other selected species for the MSI. Consequently, even evident shifts in the mean of the MSI may remain statistically insignificant.
- (2)
CI based on interannual variation
This approach is used for the European Wild Bird Indicators and the Living Planet Index (Butchart et al., 2010, Gregory and Van Strien, 2010, Loh et al., 2005), amongst others. Again, in these indicators sampling error is neglected and confidence intervals for trends in MSIs only include the interannual variation. For the European Wild Bird Indicators (Gregory et al., 2005) an analytical approach is presented to calculate CIs for the MSI that takes into account sampling error. However, this approach cannot be extended to trend assessments and it fails whenever a species index is missing for a particular year. Thus, as is the case for other indicators, sampling error is neglected in the trend assessment for European Wild Bird Indicators, even when available. The latter is inevitable, as the TrendSpotter software used for trend calculation cannot include standard errors of yearly MSIs (Soldaat et al., 2007, Visser, 2004). TrendSpotter can efficiently model flexible trends and their CIs by applying the Kalman filter. Unfortunately, only relative weighting factors can be attached to the MSIs. Absolute weighting factors like the standard errors of the MSI would not lead to proper CIs for the calculated trends.
- (3)
CI based on bootstrapping of sites
This approach properly takes into account sampling error and can be applied in a randomized monitoring scheme like the British Farmland Bird Indicator (Freeman et al., 2001). Bootstrapping on the site level, however, cannot be applied if sites are not a random sample of the population, as in many volunteer-based monitoring programmes. Obviously, bootstrapping of sites can also not be applied when data are not available on the site level, for example when MSIs are constructed using time series obtained from the literature (as in the Living Planet Index) or from national reports (as in the European Wild Bird Indicators).
An approach to take into account sampling error in MSIs that, to our knowledge, has not been explored so far is the use of standard errors of the species indices. In this paper we describe Monte Carlo procedures to generate confidence intervals for MSIs and trends in MSIs based on the standard errors of species indices. The method overcomes the abovementioned conceptual and practical obstacles, and offers several opportunities for testing and comparing trends in MSIs. Here, we first use conventional approaches to calculate an MSI with confidence intervals from an ideal simulated data set, without missing values. Subsequently, we apply our method to the same simulated data, and compare the outcome to the results of the conventional approaches for validation. Thereafter we illustrate our method using Dutch breeding bird data. Finally, we show how the method can be used to test for change-points in the MSI and trend differences between MSIs and some additional possibilities for trend assessment.
Section snippets
Calculating MSIs and confidence intervals by Monte Carlo simulation
The starting point of the Monte Carlo (MC) method is a data set with species indices and standard errors, for instance calculated with the TRIM software (Pannekoek and Van Strien, 2005). The index value in some pre-defined base year is set to 100 with standard error zero (step 1 in Fig. 1). The indices in the other years are expressed as percentage of the base year and their standard errors are a function of the variance in the specific year and the base year. Our method assumes that the
Method validation
Fig. 2 shows the MSIs of the computer-generated annual indices of the three fictitious species and their CIs derived by the Monte Carlo approach and by two alternative procedures: the analytical approach as advocated by Gregory et al. (2005) (Fig. 2a) and bootstrapping of sites (Fig. 2b). As expected, the three approaches yield practically the same MSIs and CIs. The CIs of the bootstrapping method are slightly smaller than for the Monte Carlo method.
Illustration on Dutch breeding bird data
Fig. 3 shows the standard output of the MC
Validation and applications
The Monte Carlo method we describe is a straight-forward and conceptually sound method to estimate confidence intervals around multi-species indicators. The method produces almost exactly the same results as both the analytical approach and bootstrapping, as we demonstrated for the simulated data set. Small differences are caused by the stochastic nature of both the MC approach and bootstrapping. The validation test could only be performed in an ideal simulated data set with random sites and
Conclusion
Our Monte Carlo simulation approach is a straightforward, easy to apply and conceptually sound method to take into account sampling error in multi-species indicators. Unlike bootstrapping of sites, it does not require the raw abundances per species for each of the surveyed plots: it can be applied whenever standard errors of the year indices of individual species are available. As a consequence, different approaches for index calculation between species are allowed. Contrary to analytical
Acknowledgements
We thank Hans Visser and Reinier Bikker for stimulating discussions that sharpened our view on the risks of combining species indices. Marnix de Zeeuw is acknowledged for performing the bootstrapping on simulated data sets. The authors also wish to thank Jonathan Loh and Louise McRae for kindly providing details on the methodology of the Living Planet Index. The comments of two anonymous reviewers were very helpful to increase the accessibility of the paper. The Breeding Bird Monitoring
References (32)
- et al.
Large mammal population declines in Africa’s protected areas
Biol. Conserv.
(2010) - et al.
Indices for monitoring biodiversity change: are some more effective than others?
Ecol. Indic.
(2009) - et al.
Adapting global biodiversity indicators to the national scale: a red list index for Australian birds
Biol. Conserv.
(2012) - et al.
Desirable mathematical properties of indicators for biodiversity change
Ecol. Indic.
(2012) - et al.
Modest recovery of biodiversity in a western European country: the Living Planet Index for the Netherlands
Biol. Conserv.
(2016) - et al.
Life-history and ecological correlates of population change in Dutch breeding birds
Biol. Conserv.
(2010) Estimation and detection of flexible trends
Atmos. Environ.
(2004)Categorical Data Analysis
(1990)- Biala, K., Condé, S., Delbaere, B., Jones-Walters, L., Torre-Marín, A., 2012. Streamlining European biodiversity...
- et al.
Bird Census Techniques
(2000)
The development of butterfly indicators in the United Kingdom and assessments in 2010
Journal of Insect Conservation
Monitoring change in biodiversity through composite indices
Phil. Trans. R. Soc. B: Biol. Sci.
The geometric mean of relative abundance indices: a biodiversity measure with a difference
Ecosphere
Global biodiversity: indicators of recent declines
Science
Visualising Data
Monitoring change in vertebrate abundance: the living planet index
Conserv. Biol.
Cited by (38)
Well known indicator groups do not predict the decline of insects
2024, Ecological IndicatorsIn the past 100 years dragonflies declined and recovered by habitat restoration and climate change
2023, Biological ConservationCitation Excerpt :Posterior means are reported as point estimators, associated Bayesian standard deviations as standard errors (SE) and 1.96 * SE as 95 % confidence intervals (CI). Period effects were converted into species indices with the first period as the base period as recommended by Soldaat et al. (2017). This means that the estimate for the first period is set at 100 and all estimates of other periods are expressed relative to that.
Long-term trends in forest bird populations reflect management changes in Central European forests
2022, Ecological IndicatorsCitation Excerpt :In the next step, we used the species-specific annual indices with SEs and calculated MSIs (and their confidence intervals, CIs) for respective bird groups. Annual values of the MSI are geometric means of the annual indices of the species included into a given MSI (Gregory et al., 2007, Soldaat et al., 2017). We followed the Monte Carlo method of Soldaat et al. (2017) for generating CIs of MSIs and testing whether the MSI significantly increases or decreases over the focal period.
Simple methods for improving the communication of uncertainty in species’ temporal trends
2022, Ecological IndicatorsMulti-species population indices for sets of species including rare, disappearing or newly occurring species
2022, Ecological IndicatorsCitation Excerpt :For calculating the geometric mean MSI, we treated zero values by truncating the time series at 1% of the reference population size as described in Soldaat et al. (2017). We used the Monte Carlo method to propagate the errors from the population indices or estimates to the MSI (Soldaat et al., 2017). To do so, we draw 1000 values from a log-normal distribution with mean yit and Nit respectively and the corresponding standard error as scale parameter.