Skip to main content
bioRxiv
  • Home
  • About
  • Submit
  • ALERTS / RSS
Advanced Search
New Results

Countering reproducibility issues in mathematical models with software engineering techniques: A case study using a one-dimensional mathematical model of the atrioventricular node

View ORCID ProfileChristopher Schölzel, View ORCID ProfileValeria Blesius, Gernot Ernst, View ORCID ProfileAlexander Goesmann, View ORCID ProfileAndreas Dominik
doi: https://doi.org/10.1101/2021.02.19.431951
Christopher Schölzel
1THM University of Applied Sciences, Giessen, Germany
4Justus Liebig University Giessen, Germany
  • Find this author on Google Scholar
  • Find this author on PubMed
  • Search for this author on this site
  • ORCID record for Christopher Schölzel
  • For correspondence: christopher.schoelzel@mni.thm.de
Valeria Blesius
1THM University of Applied Sciences, Giessen, Germany
  • Find this author on Google Scholar
  • Find this author on PubMed
  • Search for this author on this site
  • ORCID record for Valeria Blesius
Gernot Ernst
2Vestre Viken Hospital Trust, Kongsberg, Norway
3University of Oslo, Norway
  • Find this author on Google Scholar
  • Find this author on PubMed
  • Search for this author on this site
Alexander Goesmann
4Justus Liebig University Giessen, Germany
  • Find this author on Google Scholar
  • Find this author on PubMed
  • Search for this author on this site
  • ORCID record for Alexander Goesmann
Andreas Dominik
1THM University of Applied Sciences, Giessen, Germany
  • Find this author on Google Scholar
  • Find this author on PubMed
  • Search for this author on this site
  • ORCID record for Andreas Dominik
  • Abstract
  • Full Text
  • Info/History
  • Metrics
  • Preview PDF
Loading

Abstract

One should assume that in silico experiments in systems biology are less susceptible to reproducibility issues than their wet-lab counterparts, because they are free from natural biological variations and their environment can be fully controlled. However, recent studies show that only half of the published mathematical models of biological systems can be reproduced without substantial effort. In this article we examine the potential causes for failed or cumbersome reproductions in a case study of a one-dimensional mathematical model of the atrioventricular node, which took us four months to reproduce. The model features almost all common types of reproducibility issues including missing information, errors in equations and parameters, a lack in available data files, non-executable code, missing or incomplete experiment protocols, and missing semantic information about the rationale behind equations. Many of these issues seem similar to problems that have already been solved in software engineering using techniques such as unit testing, regression tests, continuous integration, version control, archival services, and a thorough modular design with extensive documentation. Applying these techniques, we reimplement the examined model using the modeling language Modelica. The resulting workflow can be applied to any mathematical model. It guarantees methods reproducibility by executing automated tests in a virtual machine on a server that is physically separated from the development environment. Additionally, it facilitates results reproducibility, because the model is more understandable and because the complete model code, experiment protocols, and simulation data are published and can be accessed in the exact version that was used in this article. While the increased attention to design aspects and documentation required considerable effort, we found it justified, even just considering the immediate benefits during development such as easier and faster debugging, increased understandability of equations, and a reduced requirement for looking up details from the literature.

Author summary Reproducibility is one of the cornerstones of the scientific method. In order to draw reliable conclusions, an experiment must yield the same results when it is repeated using the same methods. However, biological systems are complex, which makes experiments cumbersome. It is therefore desirable to build a mathematical representation of the biological system, which captures its essential behavior in a set of variables and equations and allows for easier and faster experimentation. Unfortunately, recent studies have shown that half of the published mathematical models are not immediately reproducible due to missing information, mathematical errors, and incomplete documentation. These issues are similar to those faced in software engineering: A single missing file or a buggy line of code can render any kind of software useless. Software engineering has turned to rigorous software testing, automated development pipelines, and version control systems to overcome these challenges, but these techniques are not yet widely applied to mathematical modeling. In this paper we demonstrate their benefit for the reproducibility of a large mathematical model of the atrioventricular node. The software engineering solutions that we employ can be applied to any mathematical model and could therefore facilitate scientific progress by encouraging and simplifying model reuse.

Copyright 
The copyright holder for this preprint is the author/funder, who has granted bioRxiv a license to display the preprint in perpetuity. It is made available under a CC-BY 4.0 International license.
Back to top
PreviousNext
Posted February 19, 2021.
Download PDF
Email

Thank you for your interest in spreading the word about bioRxiv.

NOTE: Your email address is requested solely to identify you as the sender of this article.

Enter multiple addresses on separate lines or separate them with commas.
Countering reproducibility issues in mathematical models with software engineering techniques: A case study using a one-dimensional mathematical model of the atrioventricular node
(Your Name) has forwarded a page to you from bioRxiv
(Your Name) thought you would like to see this page from the bioRxiv website.
CAPTCHA
This question is for testing whether or not you are a human visitor and to prevent automated spam submissions.
Share
Countering reproducibility issues in mathematical models with software engineering techniques: A case study using a one-dimensional mathematical model of the atrioventricular node
Christopher Schölzel, Valeria Blesius, Gernot Ernst, Alexander Goesmann, Andreas Dominik
bioRxiv 2021.02.19.431951; doi: https://doi.org/10.1101/2021.02.19.431951
Digg logo Reddit logo Twitter logo CiteULike logo Facebook logo Google logo Mendeley logo
Citation Tools
Countering reproducibility issues in mathematical models with software engineering techniques: A case study using a one-dimensional mathematical model of the atrioventricular node
Christopher Schölzel, Valeria Blesius, Gernot Ernst, Alexander Goesmann, Andreas Dominik
bioRxiv 2021.02.19.431951; doi: https://doi.org/10.1101/2021.02.19.431951

Citation Manager Formats

  • BibTeX
  • Bookends
  • EasyBib
  • EndNote (tagged)
  • EndNote 8 (xml)
  • Medlars
  • Mendeley
  • Papers
  • RefWorks Tagged
  • Ref Manager
  • RIS
  • Zotero
  • Tweet Widget
  • Facebook Like
  • Google Plus One

Subject Area

  • Systems Biology
Subject Areas
All Articles
  • Animal Behavior and Cognition (2639)
  • Biochemistry (5231)
  • Bioengineering (3653)
  • Bioinformatics (15748)
  • Biophysics (7226)
  • Cancer Biology (5604)
  • Cell Biology (8060)
  • Clinical Trials (138)
  • Developmental Biology (4748)
  • Ecology (7477)
  • Epidemiology (2059)
  • Evolutionary Biology (10535)
  • Genetics (7708)
  • Genomics (10093)
  • Immunology (5168)
  • Microbiology (13844)
  • Molecular Biology (5361)
  • Neuroscience (30629)
  • Paleontology (213)
  • Pathology (873)
  • Pharmacology and Toxicology (1520)
  • Physiology (2236)
  • Plant Biology (4992)
  • Scientific Communication and Education (1039)
  • Synthetic Biology (1382)
  • Systems Biology (4135)
  • Zoology (808)