Assessment of whether published non-Cochrane systematic reviews of nursing follow the review protocols registered in the International Prospective Register of Systematic Reviews (PROSPERO): A comparative study

Purpose We compare published non-Cochrane reviews of nursing with their pre-registered protocols on PROSPERO to quantify the prevalence of differences and the extent to which the differences were explained. Methods We searched for protocols and their corresponding reviews in PROSPERO’s nursing group that were “completed and published” from inception to September 8th, 2019. Two authors independently identified differences and classified the difference as none, partial, or complete, and determined if the existed differences had been explained. Frequency (n), percentage (%), median, and inter-quartile ranges were used to analyze the extent of differences and explanations. Results We identified 22 pre-registered protocols and their reviews. All 22 pairs (100%) exhibited differences. Eighteen pairs (82%) showed differences in at least six methodological sections, while 21 pairs (95%) involved completed difference in at least one section. The median number of differences per review was 8.00 (upper quartile = 6.00, lower quartile = 9.75). The differences involved all 13 compared methodology-related sections. Only 5 (3%) of all differences were explained in the systematic reviews. Conclusions We observed widespread differences between non-Cochrane reviews of nursing and their protocols recorded in PROSPERO, with relatively few explanations for the changes. Measures including establishing a new item in the reporting guideline of systematic reviews to guide reporting and explaining the reasons for differences between protocols and systematic reviews or even requiring authors to do so at the Journal’s author guideline are recommended to improve transparency.


47
Systematic reviews are recognized as being vitally important for evidence-based health care 48 and guide clinical decision-making [1]. However, due to the nature of retrospective design, 49 selective inclusion and reporting of outcomes must be addressed at the level of systematic 50 review [2][3][4]. 51 To ensure transparency in the assembly and writing of systematic reviews, some documents the methods and planning of the review. However, these organizations produce only 55 a relatively small proportion of published systematic reviews [1], and mandatory registration 56 for most systematic reviews is lacking. In 2010, PRISMA issued a statement advocating 57 registration of systematic review protocols and this was followed by the establishment of the 58 International Prospective Register of Systematic Reviews (PROSPERO) by the United 59 Kingdom Centre for Reviews and Dissemination in 2011 [6]. Their objectives are to effectively 60 fill the vacant position of the systematic review registration and provide a new registration 61 platform for non-Cochrane systematic reviews with which they expect to improve the quality 62 of these reviews [6][7]. For registration on PROSPERO, authors are required to submit introducing bias into the study [3][4]. Wherever possible, protocol changes should be avoided 72 in order to ensure that research methodology is not changed in response to unexpected results

73
[9]. It is particularly important that these changes be avoided during the data gathering and 74 analysis steps of the study [9-10].

75
The number of systematic reviews indexed by MEDLINE has tripled over the past decade, 76 with more than 8,000 published in 2016 alone [1]. It is also noticeable that there are differences 77 between the final published reviews and the previously published protocols. Silagy platforms for non-Cochrane systematic reviews, it is important to assess whether the protocols 86 registered on PROSPERO have improved the transparency of non-Cochrane systematic 87 reviews, specifically considering reviews in nursing. As such, our primary objective was to 88 investigate and quantify differences in methodology-related sections between non-Cochrane 89 systematic reviews and their pre-registered protocols on PROSPERO's nursing group. Our 90 secondary objective was to determine the extent to which these changes were reported and 91 explained in the published systematic reviews.  "pairs" to exclude Cochrane reviews, JBI reviews, and reviews included non-clinical studies. 103 We did not limit the language of published systematic reviews.  (e.g., when we observed a partial and a complete difference, we classified the compared section 139 as having a complete difference). If there were more than two partial differences concerning 140 the one section, we also classified it as having a complete difference.

157
All differences between the systematic reviews and their protocols are listed in S3 Appendix.

158
These differences were seen in all the included systematic reviews and in all 13 of the 159 methodology-related sections. We observed that many protocols were deficient in key 160 information such as details of participants, intervention/exposure, comparator, and outcomes.

161
More alarmingly, key information was often missing in mandatory fields such as "Search 162 strategy", "Study selection", "Data extraction", "Risk of bias assessment", and "Data

235
The strengths and limitations of this study 236 This is the first study to compare non-Cochrane systematic reviews with their protocols 237 registered on PROSPERO's nursing group regarding differences in all methodology-related 238 sections, as opposed to only differences in predefined outcomes. We acknowledge several 239 limitations to our study. Firstly, we identified the published systematic reviews by using the 240 final publication detail of PROSPERO's record. It is therefore possible that the reviews could 241 be misses if the information was not updated. Additionally, although we did not restrict the 242 number of reviews based on language or date of protocol registration, the sample size was still 243 relatively small. Secondly, we did not verify the differences with the authors due to limited 244 resources. Our study was based solely on the reporting of the systematic reviews and their 245 protocols, resulting in decisions on differences being affected by the quality of the reporting. should also be investigated.

258
Differences between the non-Cochrane systematic reviews of nursing and their protocols 259 recorded on PROSPERO's nursing group were widespread, with relatively few explanations. 260 We recommend measures to improve transparency of non-Cochrane systematic reviews of 261 nursing, specifically, establishing a new item in PRISMA, MOOSE, and ENTREQ to guide 262 reporting and explaining the reasons for differences between protocols and systematic reviews 263 or even requiring authors to do so at the Journal's author guideline.