Toward Reproducible Computational Research: An Empirical Analysis of Data and Code Policy Adoption by Journals

PLoS One. 2013 Jun 21;8(6):e67111. doi: 10.1371/journal.pone.0067111. Print 2013.

Abstract

Journal policy on research data and code availability is an important part of the ongoing shift toward publishing reproducible computational science. This article extends the literature by studying journal data sharing policies by year (for both 2011 and 2012) for a referent set of 170 journals. We make a further contribution by evaluating code sharing policies, supplemental materials policies, and open access status for these 170 journals for each of 2011 and 2012. We build a predictive model of open data and code policy adoption as a function of impact factor and publisher and find higher impact journals more likely to have open data and code policies and scientific societies more likely to have open data and code policies than commercial publishers. We also find open data policies tend to lead open code policies, and we find no relationship between open data and code policies and either supplemental material policies or open access journal status. Of the journals in this study, 38% had a data policy, 22% had a code policy, and 66% had a supplemental materials policy as of June 2012. This reflects a striking one year increase of 16% in the number of data policies, a 30% increase in code policies, and a 7% increase in the number of supplemental materials policies. We introduce a new dataset to the community that categorizes data and code sharing, supplemental materials, and open access policies in 2011 and 2012 for these 170 journals.

Publication types

  • Research Support, U.S. Gov't, Non-P.H.S.

MeSH terms

  • Codes of Ethics*
  • Computational Biology / ethics*
  • Models, Theoretical*
  • Periodicals as Topic
  • Reproducibility of Results

Grants and funding

This research was supported by NSF award number 1153384 “EAGER: Policy Design for Reproducibility and Data Sharing in Computational Science.” See http://www.nsf.gov/awardsearch/showAward?AWD_ID=1153384. The funders had no role in study design, data collection and analysis, decision to publish, or preparation of the manuscript.