Multicenter preclinical studies as an innovative method to enhance translation: a systematic review of published studies

Multicenter preclinical studies have been suggested as a method to improve reproducibility, generalizability and potential clinical translation of preclinical work. In these studies, multiple independent laboratories collaboratively conduct a research experiment using a shared protocol. The use of a multicenter design in preclinical experimentation is a recent approach and only a handful of preclinical multicenter studies have been published. Here, we systematically identify, assess and synthesize published preclinical multicenter studies investigating interventions using in vivo models. Synthesized data included study methods/design, basic characteristics, outcomes, and barriers and facilitators. Study risk of bias, completeness of reporting and the degree of collaboration were evaluated using established methods. The database searches identified 3095 citations and 12 studies met inclusion criteria. The multicenter study design was applied across a diverse range of diseases including stroke, heart attack, traumatic brain injury, and diabetes. The median number of centers was 4 (range 2-6) and the median sample size was 135 (range 23-384). Most studies had lower risk of bias and higher completeness of reporting than typically seen in single-centered studies. Only four of the twelve studies produced results consistent with previous single-center studies, highlighting a central concern of preclinical research: irreproducibility and poor generalizability of findings from single laboratories. Our review suggests that multicenter preclinical studies may provide a method to robustly assess therapies prior to considering clinical translation. Registered with PROSPERO CRD42018093986.


Introduction
The translation process of preclinical findings into clinical practice is fraught with time lags, steep 67 costs, and considerable failure rates [1][2][3][4][5][6][7]. It has been suggested that one of the problems 68 contributing to translational failures lies outside of clinical research itself, and instead originates 69 in the preclinical stage of research [8][9][10]. Translational barriers associated with preclinical 70 research include poor study design and reporting that make reproducibility difficult; biased 71 selection of animal models and small sample sizes which reduces inferential strength; and 72 publication bias which may distort evidence and justification to proceed to first-in-human trials [5, 73 6, 11]. In order to increase the chance of 'bench-to-bedside' translation success, various measures 74 to improve the state of preclinical research have been suggested [12,13]. One measure is the 75 application of multicenter experimentation in preclinical studies, analogous to what is commonly 76 done in clinical trials [14,15]. In both clinical and preclinical research, multicenter studies can 77 assess external validity and inherently test reproducibility, while also increasing efficiency in 78 meeting enrolment numbers [9]. In addition, rigorously designed and reported multicenter studies 79 offer the opportunity to enhance internal validity and increase transparency [16]. 80

81
To improve the process of translation, multiple calls from the biomedical science community have 82 been made to adopt the multicenter preclinical approach [6, 12-14, 16, 17]. Some recent examples 83 have been published that exemplify the successful implementation of this approach [18][19][20]. As 84 interest in multicenter preclinical studies grows, and to demonstrate their value (if any), it is 85 imperative that a systematic evaluation should be performed of the studies conducted to date. This 86 will inform and optimize future multicenter preclinical studies by identifying knowledge gaps and 87 producing an evidence map of current practices and outcomes [5][6][7]. 88

89
The objective of this systematic review was to identify and qualitatively summarize the preclinical 90 multicenter study literature. All multicenter in vivo preclinical studies of interventions were 91 included. We compared and contrasted the methods and organization of these experiments. Quality 92 of reporting, risk of bias, degree of collaboration, and barriers/enablers to multicenter study 93 conduct were assessed. Finally, we considered how results of these studies and the use of the 94 multicenter study design informed the translation of biomedical research. 95

96
Search results and study characteristics 97 The database searches identified a total of 3095 papers after duplicates were removed (Fig 1). 98 Two additional papers were identified through a search of references of included papers. After 99 title, abstract, and full-text screening twelve articles met eligibility criteria (Tables 1 and 2). The identified studies fell into six clinical domains: traumatic brain injury (n = 5), myocardial 106 infarction (n = 2), stroke (n = 2), diabetes (n = 1), traumatic injury (n = 1), and effects of stimulate 107 exposure (n = 1 (n = 2) results; the three studies that recommended to proceed with human clinical trials had mixed 132 (n = 1) and positive results (n = 2), and the five studies that suggested that there should be no 133 further testing (clinical or preclinical) all had null results (Fig 2). Brief synopses of the twelve 134 studies can be found in supporting information (S1 Text), along with sample statements of their 135 future recommendations (S2 Table). 136

137
None of the 12 studies (0%) were considered low risk of bias across all ten domains (Table 3). Ten 138 studies randomized animals to experimental groups and two of these reported the method of 139 random sequence generation. Nine studies had a low risk of detection bias by blinding of outcome 140 assessors. Eight studies were at low risk of performance bias by blinding personnel administering 141 interventions. All but one study was unclear if animals were randomly housed during the 142 experiments. Five studies from the same research consortium (Operation Brain Trauma Therapy) 143 had high risk of bias for other sources of bias due to potential industry-related influences (Table   144 3). The four 'other sources' of risk of bias assessments for each study is found in the supporting 145 information (S3 Table).  brain injury studies included preclinical in the paper title. Reporting assessment for all twenty-162 nine items across the twelve studies can be found in the supporting information (S4 Table).   Degree of collaboration 173 Overall, the twelve studies scored medium to high in the degree of collaboration (      Table) with details on the sources for each item.    1. In a study by Reimer et al. (1985) [26], three independent laboratories collaborated to 608 develop models to test potential ischemic myocardium protection therapies, using two 609 standardized, well-characterized canine models of myocardial infarction. Using the two

Spoerke, 2009
The species-specific differences in factor activities will require ongoing investigation to ensure full safety and efficacy. Our future investigations will include a comprehensive evaluation of the effects of the lyophilization process on coagulation properties of the LP.

Jones, 2015
other investigators can adopt the protocols [for measuring infarct size in mice, rabbits, and pigs in a manner that is rigorous, accurate, and reproducible] in their own laboratories.

Llovera, 2015
future clinical trials testing immunotherapeutic drugs for stroke will need to ensure that the included study population feature a substantial neuroinflammatory reaction to the brain injury Maysami, 2015 interleukin 1 receptor antagonist should be evaluated in more extensive clinical stroke trials

Bramlett, 2016
Although we cannot rule out the possibility that other doses or more prolonged treatment could show different effects, the lack of efficacy of EPO reduced enthusiasm for its further investigation in OBTT.

Browning, 2016
…need for OBTT to study LEV further. This includes studies of dose response, therapeutic window, mechanism, and testing in our large animal FPI model in micropigs… consider a randomized controlled trial examining early administration in patients

Dixon, 2016
Our findings reduce enthusiasm for further investigation of this therapy in OBTT and suggest that if this strategy is to be pursued further, alternative CsA analogs with reduced toxicity should be used.

Gill, 2016
…pause in proceeding with clinical trials without further preclinical testing.

Mountney, 2016
the current findings do not support the beneficial effects of simvastatin… it will not be further pursued by OBTT.

Shear, 2016
The marginal benefits achieved with nicotinamide, however, which appeared sporadically across the TBI models, has reduced enthusiasm for further investigation by the OBTT Consortium.  Table   703 on potential conflict of interests. Contamination: Low risk = No treatment or drug other than the study drug used. Unclear = Possibility of contamination from other treatments or drugs. High risk = Animals receive additional treatment/drugs other than the intervention. Authors report this could influence the results. Unit of analysis errors: Low risk = Animals were analyzed individually as one replicate. Unclear: unclear if animals were analyzed individually and treated as one replication. High risk = Animals were not analyzed individual (ex. all animals in one cage) or not treated as one replicate (ex. Same animal: one eye intervention, one eye control). *financial disclosure, no statement of other conflicts provided Provide an explicit statement of questions being addressed with reference to participants, interventions, comparisons, outcomes, and study design (PICOS).

Protocol and registration 5
Indicate if a review protocol exists, if and where it can be accessed (e.g., Web address), and, if available, provide registration information including registration number.

& 21
Eligibility criteria 6 Specify study characteristics (e.g., PICOS, length of follow-up) and report characteristics (e.g., years considered, language, publication status) used as criteria for eligibility, giving rationale.

21-22
Information sources 7 Describe all information sources (e.g., databases with dates of coverage, contact with study authors to identify additional studies) in the search and date last searched.

22-23, 37
Search 8 Present full electronic search strategy for at least one database, including any limits used, such that it could be repeated.

22-23, 37
Study selection 9 State the process for selecting studies (i.e., screening, eligibility, included in systematic review, and, if applicable, included in the meta-analysis). 23 Data collection process 10 Describe method of data extraction from reports (e.g., piloted forms, independently, in duplicate) and any processes for obtaining and confirming data from investigators.

23-25
Data items 11 List and define all variables for which data were sought (e.g., PICOS, funding sources) and any assumptions and simplifications made. 23

12
Describe methods used for assessing risk of bias of individual studies (including specification of whether this was done at the study or outcome level), and how this information is to be used in any data synthesis.