Skip to main content
bioRxiv
  • Home
  • About
  • Submit
  • ALERTS / RSS
Advanced Search
New Results

Evaluating species distribution models with discrimination accuracy is uninformative for many applications

View ORCID ProfileDan L. Warren, Nicholas J. Matzke, Teresa L. Iglesias
doi: https://doi.org/10.1101/684399
Dan L. Warren
1Senckenberg Biodiversity and Climate Research Centre, Frankfurt, Germany
2Biodiversity and Biocomplexity Unit, Okinawa Institute of Science and Technology, Okinawa, Japan
3Department of Biology, Macquarie University, Sydney, Australia
  • Find this author on Google Scholar
  • Find this author on PubMed
  • Search for this author on this site
  • ORCID record for Dan L. Warren
  • For correspondence: dan.l.warren@gmail.com
Nicholas J. Matzke
4Department of Biological Sciences, University of Auckland, Auckland, New Zealand
  • Find this author on Google Scholar
  • Find this author on PubMed
  • Search for this author on this site
Teresa L. Iglesias
3Department of Biology, Macquarie University, Sydney, Australia
5Physics and Biology Unity, Okinawa Institute of Science and Technology
  • Find this author on Google Scholar
  • Find this author on PubMed
  • Search for this author on this site
  • Abstract
  • Full Text
  • Info/History
  • Metrics
  • Supplementary material
  • Data/Code
  • Preview PDF
Loading

Abstract

Aim Species distribution models are used across evolution, ecology, conservation, and epidemiology to make critical decisions and study biological phenomena, often in cases where experimental approaches are intractable. Choices regarding optimal models, methods, and data are typically made based on discrimination accuracy: a model’s ability to predict subsets of species occurrence data that were withheld during model construction. However, empirical applications of these models often involve making biological inferences based on continuous estimates of relative habitat suitability as a function of environmental predictor variables. We term the reliability of these biological inferences “functional accuracy.” We explore the link between discrimination accuracy and functional accuracy.

Methods Using a simulation approach we investigate whether models that make good predictions of species distributions correctly infer the underlying relationship between environmental predictors and the suitability of habitat.

Results We demonstrate that discrimination accuracy is only informative when models are simple and similar in structure to the true niche, or when data partitioning is geographically structured. However, the utility of discrimination accuracy for selecting models with high functional accuracy was low in all cases.

Main conclusions These results suggest that many empirical studies and decisions are based on criteria that are unrelated to models’ usefulness for their intended purpose. We argue that empirical modeling studies need to place significantly more emphasis on biological insight into the plausibility of models, and that the current approach of maximizing discrimination accuracy at the expense of other considerations is detrimental to both the empirical and methodological literature in this active field. Finally, we argue that future development of the field must include an increased emphasis on simulation; methodological studies based on ability to predict withheld occurrence data may be largely uninformative about best practices for applications where interpretation of models relies on estimating ecological processes, and will unduly penalize more biologically informative modeling approaches.

Footnotes

  • https://github.com/danlwarren/sim-code-Warren-et-al-2019

Copyright 
The copyright holder for this preprint is the author/funder, who has granted bioRxiv a license to display the preprint in perpetuity. It is made available under a CC-BY 4.0 International license.
Back to top
PreviousNext
Posted June 27, 2019.
Download PDF

Supplementary Material

Data/Code
Email

Thank you for your interest in spreading the word about bioRxiv.

NOTE: Your email address is requested solely to identify you as the sender of this article.

Enter multiple addresses on separate lines or separate them with commas.
Evaluating species distribution models with discrimination accuracy is uninformative for many applications
(Your Name) has forwarded a page to you from bioRxiv
(Your Name) thought you would like to see this page from the bioRxiv website.
CAPTCHA
This question is for testing whether or not you are a human visitor and to prevent automated spam submissions.
Share
Evaluating species distribution models with discrimination accuracy is uninformative for many applications
Dan L. Warren, Nicholas J. Matzke, Teresa L. Iglesias
bioRxiv 684399; doi: https://doi.org/10.1101/684399
Reddit logo Twitter logo Facebook logo LinkedIn logo Mendeley logo
Citation Tools
Evaluating species distribution models with discrimination accuracy is uninformative for many applications
Dan L. Warren, Nicholas J. Matzke, Teresa L. Iglesias
bioRxiv 684399; doi: https://doi.org/10.1101/684399

Citation Manager Formats

  • BibTeX
  • Bookends
  • EasyBib
  • EndNote (tagged)
  • EndNote 8 (xml)
  • Medlars
  • Mendeley
  • Papers
  • RefWorks Tagged
  • Ref Manager
  • RIS
  • Zotero
  • Tweet Widget
  • Facebook Like
  • Google Plus One

Subject Area

  • Ecology
Subject Areas
All Articles
  • Animal Behavior and Cognition (4381)
  • Biochemistry (9581)
  • Bioengineering (7086)
  • Bioinformatics (24844)
  • Biophysics (12597)
  • Cancer Biology (9952)
  • Cell Biology (14345)
  • Clinical Trials (138)
  • Developmental Biology (7944)
  • Ecology (12101)
  • Epidemiology (2067)
  • Evolutionary Biology (15984)
  • Genetics (10921)
  • Genomics (14735)
  • Immunology (9869)
  • Microbiology (23645)
  • Molecular Biology (9477)
  • Neuroscience (50838)
  • Paleontology (369)
  • Pathology (1539)
  • Pharmacology and Toxicology (2681)
  • Physiology (4013)
  • Plant Biology (8655)
  • Scientific Communication and Education (1508)
  • Synthetic Biology (2391)
  • Systems Biology (6427)
  • Zoology (1346)