PT  - JOURNAL ARTICLE
AU  - David Augustin
AU  - Ken Wang
AU  - Antje-Christine Walz
AU  - Ben Lambert
AU  - Michael Clerx
AU  - Martin Robinson
AU  - David Gavaghan
TI  - Treatment response prediction: Is model selection unreliable?
AID  - 10.1101/2022.03.19.483454
DP  - 2022 Jan 01
TA  - bioRxiv
PG  - 2022.03.19.483454
4099  - http://biorxiv.org/content/early/2022/03/21/2022.03.19.483454.short
4100  - http://biorxiv.org/content/early/2022/03/21/2022.03.19.483454.full
AB  - Quantitative modelling has become an essential part of the drug development pipeline. In particular, pharmacokinetic and pharmacodynamic models are used to predict treatment responses in order to optimise clinical trials and assess the safety and efficacy of dosing regimens across patients. It is therefore crucial that treatment response predictions are reliable. However, the data available to fit models are often limited, which can leave considerable uncertainty about the best model to use. Common practice is to select the model that is most consistent with the observed data based on the Akaike information criterion (AIC). Another popular approach is to average the predictions across the subset of models consistent with the data. In this article, we argue that both approaches can lead to unreliable predictions, as treatment responses typically display nonlinear dynamics, so models can be consistent with the observed dynamics, whilst predicting incorrect treatment responses. This is especially the case when predicting treatment responses for either times or dosing regimens that go beyond the observed dynamics. Across a range of experiments on both real laboratory data and synthetically derived data on Neisseria gonorrhoeae response to ciprofloxacin, we show that probabilistic averaging of models results in more reliable treatment response predictions.Competing Interest StatementKW and ACW are employees and shareholders of F. Hoffmann-La Roche Ltd.