Comparing quality of reporting between preprints and peer-reviewed articles in the biomedical literature

Clarissa F D Carneiro; Victor G S Queiroz; Thiago C Moulin; Carlos A M Carvalho; Clarissa B Haas; Danielle Rayêe; David E Henshall; Evandro A De-Souza; Felippe E Amorim; Flávia Z Boos; Gerson D Guercio; Igor R Costa; Karina L Hajdu; Lieve van Egmond; Martin Modrák; Pedro B Tan; Richard J Abdill; Steven J Burgess; Sylvia F S Guerra; Vanessa T Bortoluzzi; Olavo B Amaral

doi:10.1186/s41073-020-00101-3

Comparing quality of reporting between preprints and peer-reviewed articles in the biomedical literature

Res Integr Peer Rev. 2020 Dec 1;5(1):16. doi: 10.1186/s41073-020-00101-3.

Authors

Clarissa F D Carneiro¹, Victor G S Queiroz², Thiago C Moulin², Carlos A M Carvalho^{3

4

5}, Clarissa B Haas⁶, Danielle Rayêe⁷, David E Henshall⁸, Evandro A De-Souza², Felippe E Amorim², Flávia Z Boos⁹, Gerson D Guercio¹⁰, Igor R Costa², Karina L Hajdu⁷, Lieve van Egmond¹¹, Martin Modrák¹², Pedro B Tan⁷, Richard J Abdill¹³, Steven J Burgess¹⁴, Sylvia F S Guerra^{5

15

16}, Vanessa T Bortoluzzi⁶, Olavo B Amaral²

Affiliations

¹ Institute of Medical Biochemistry Leopoldo de Meis, Federal University of Rio de Janeiro, Rio de Janeiro, RJ, 21941-902, Brazil. clarissa.franca@bioqmed.ufrj.br.
² Institute of Medical Biochemistry Leopoldo de Meis, Federal University of Rio de Janeiro, Rio de Janeiro, RJ, 21941-902, Brazil.
³ Seção de Arbovirologia e Febres Hemorrágicas, Instituto Evandro Chagas, Ananindeua, Pará, Brazil.
⁴ Departamento de Patologia, Universidade do Estado do Pará, Belém, Pará, Brazil.
⁵ Centro Universitário Metropolitano da Amazônia, Instituto Euro-Americano de Educação, Ciência e Tecnologia, Belém, Pará, Brazil.
⁶ Departamento de Bioquímica, Instituto de Ciências Básicas da Saúde, Universidade Federal do Rio Grande do Sul, Porto Alegre, Rio Grande do Sul, Brazil.
⁷ Biomedical Sciences Institute, Federal University of Rio de Janeiro, Rio de Janeiro, Brazil.
⁸ University of Edinburgh Medical School, Edinburgh, Scotland, UK.
⁹ Programa de Pós-Graduação em Psicobiologia, Universidade Federal de São Paulo, São Paulo, Brazil.
¹⁰ Department of Psychiatry, University of Minnesota, Minneapolis, MN, USA.
¹¹ Department of Neuroscience, Uppsala University, Uppsala, Sweden.
¹² Institute of Microbiology of the Czech Academy of Sciences, Prague, Czech Republic.
¹³ Department of Genetics, Cell Biology, and Development, University of Minnesota, Minneapolis, MN, USA.
¹⁴ Carl R Woese Institute for Genomic Biology, University of Illinois at Urbana-Champaign, Champaign, IL, USA.
¹⁵ Seção de Virologia, Instituto Evandro Chagas, Ananindeua, Pará, Brazil.
¹⁶ Departamento de Morfologia e Ciências Fisiológicas, Universidade do Estado do Pará, Belém, Pará, Brazil.

Abstract

Background: Preprint usage is growing rapidly in the life sciences; however, questions remain on the relative quality of preprints when compared to published articles. An objective dimension of quality that is readily measurable is completeness of reporting, as transparency can improve the reader's ability to independently interpret data and reproduce findings.

Methods: In this observational study, we initially compared independent samples of articles published in bioRxiv and in PubMed-indexed journals in 2016 using a quality of reporting questionnaire. After that, we performed paired comparisons between preprints from bioRxiv to their own peer-reviewed versions in journals.

Results: Peer-reviewed articles had, on average, higher quality of reporting than preprints, although the difference was small, with absolute differences of 5.0% [95% CI 1.4, 8.6] and 4.7% [95% CI 2.4, 7.0] of reported items in the independent samples and paired sample comparison, respectively. There were larger differences favoring peer-reviewed articles in subjective ratings of how clearly titles and abstracts presented the main findings and how easy it was to locate relevant reporting information. Changes in reporting from preprints to peer-reviewed versions did not correlate with the impact factor of the publication venue or with the time lag from bioRxiv to journal publication.

Conclusions: Our results suggest that, on average, publication in a peer-reviewed journal is associated with improvement in quality of reporting. They also show that quality of reporting in preprints in the life sciences is within a similar range as that of peer-reviewed articles, albeit slightly lower on average, supporting the idea that preprints should be considered valid scientific contributions.

Keywords: Peer review; Preprint; Publication; Quality of reporting; Scientific journal; bioRxiv.