PopED: An extended, parallelized, nonlinear mixed effects models optimal design tool

doi:10.1016/j.cmpb.2012.05.005

Computer Methods and Programs in Biomedicine

Volume 108, Issue 2, November 2012, Pages 789-805

https://doi.org/10.1016/j.cmpb.2012.05.005 Get rights and content

Abstract

Several developments have facilitated the practical application and increased the general use of optimal design for nonlinear mixed effects models. These developments include new methodology for utilizing advanced pharmacometric models, faster optimization algorithms and user friendly software tools. In this paper we present the extension of the optimal design software PopED, which incorporates many of these recent advances into an easily useable enhanced GUI. Furthermore, we present new solutions to problems related to the design of experiments such as: faster and more robust FIM calculations and optimizations, optimizing over cost/utility functions and diagnostic tools and plots to evaluate design performance. Examples for; (i) Group size optimization and efficiency translation, (ii) Cost/constraint optimization, (iii) Optimizations with different FIM approximations and (iv) optimization with parallel computing demonstrate the new features in PopED and underline the potential use of this tool when designing experiments.

Introduction

The idea of designing scientific experiments in an optimal way dates back to the beginning of the 20th century [1]. R.A. Fisher first demonstrated that the information contained in experimental data is limited and dependent on the experimental setup [1]. A formal mathematical inequality was established by Cramer and Rao (independently from each other) stating that the inverse of the Fisher information matrix (FIM) is a lower bound for the covariance of any unbiased estimator [2], [3]. Therefore, the Cramer–Rao bound provides the essential basis for the optimized planning of an experiment, suggesting that the experimental design variables should be chosen such that the FIM is maximal. For linear and nonlinear regression models without mixed effects parameters the FIM can be analytically calculated as demonstrated in the work by Atkinson and Donev [4]. In contrast to that, the calculation of the FIM for nonlinear mixed effect models (NLMEM) has to rely on approximations since analytic solutions do not exist. A closed form solution for the first-order approximated FIM for NLMEM was initially derived by Mentré et al. [5]. Later on this approach was extended and implemented in a computer program by Retout et al. [6]. Today, several software packages for the optimization of population experimental designs (designs for nonlinear mixed effects models) are available and have been compared by Mentré et al. [7]. The programs differ in their implementations of the FIM calculation, the programming language and in the set of available features. In this paper we discuss new features of the program PopED (Population Experimental Design) initially developed by Foracchia et al. [8].

This paper consists of three parts. First, the theoretical basis of population optimal design is presented in Section 2. In the second part the implementation of PopED is described in detail. This includes (i) a listing of the different FIM approximations available (Section 3.1 and in more detail in the appendix), (ii) a description of the numerical methods used for the calculation of the FIM and the optimization of the designs (Sections 3.2 Design criteria, 3.3 Design optimization, 3.4 Calculation of derivatives, 3.5 Solution of differential equations) and (iii) a presentation of the software architecture used for the graphical user interface (GUI) and the calculation engine (Sections 4.1 Graphical user interface, 4.2 Calculation engine, 4.3 Parallelization of PopED). Section 4 also includes a description of how computations can be parallelized in PopED. The third part of the article presents four examples selected to demonstrate the strengths of the software (Sections 5.1 Example 1: Group size optimization, 5.1.1 Results, 5.1.2 Conclusions, 5.2 Example 2: Constraint/cost optimization, 5.2.1 Results, 5.2.2 Conclusions, 5.3 Example 3: Optimization with different FIM approximations, 5.3.1 Results, 5.3.2 Conclusions, 5.4 Example 4: Parallel execution). Even though all examples consider pharmacometric applications of optimal design it should be pointed out that the application of the software is not restricted to this area as demonstrated for example in the work by Sjögren et al. [9].

Section snippets

Optimal design

The fundamental role of the Cramer–Rao inequality for the theory of optimal design has been mentioned above. Formally, the relationship between the inverse of the FIM and the variance-covariance matrix (COV) of any unbiased estimator, as established by the Cramer–Rao inequality, can be formulated as ${FIM}^{- 1} (q, Θ) \leq COV (q, y, Θ)$ where y is the observed data, q are the design specific variables and Θ the model parameters. It is clear from this equation that a maximal FIM (or minimal inverse of the FIM) is

Fisher information matrix

In PopED the user can choose among an extensive collection of different FIM approximation methods. In this section, the different approximations are listed and discussed briefly. A more mathematical description is given in the appendix.

First-order approximation: An approximation of the FIM for NLMEMs was first proposed by Mentré et al. [5] and further developed and improved by Retout et al. [6], [10]. The first-order (FO) approximation available in PopED is based on the approach described by

Graphical user interface

The graphical user interface (GUI) (Fig. 1) is written in C# .NET Framework 2.0 and communicates with the calculation engine, written in MATLAB, via xml files (Fig. 2). The GUI enables usage of PopED without extensive knowledge of MATLAB or programming in general. However, the GUI only presents design- and optimization settings with a more user-friendly interface and hence all settings enabled via the GUI can also be entered directly in the PopED MATLAB configuration file (function_input.m). A

Example 1: Group size optimization

This example illustrates the application of the group size optimization, the differences between the GS and FA algorithm and the efficiency translation feature. The pharmacometric model for this example was a simple dose–response model, described by the following equation: $y = (θ_{1} e^{η_{1}} + \frac{θ_{2} e^{η_{2}} \cdot D_{g}}{θ_{3} e^{η_{3}} + D_{g}}) (1 + ε)$ where D_g is the vector of doses given to group g, ɛ is the residual variability random variable $(ε \sim N (0, σ^{2}))$ , θ_1,2,3 are the fixed and η_1,2,3 the associated random effect variables $(η_{x} \sim N (0, ω_{x}^{2}))$ .

Discussion

This paper shows several new features in optimal design for NLMEM all implemented in the optimal design tool PopED. The tool is open source and freely accessible online (http://poped.sf.net). Most of the features in PopED can be used by the free software FreeMat [30] but some additional functionality are limited to MATLAB [24] users (see Table 2 for details).

Several other tools in NLMEM optimal design are available, written in MATLAB or R and with graphical user interfaces [7]. However, the

Conflict of interest

The authors declare no conflict of interest.

Acknowledgements

The parallel computations were performed on resources provide by SNIC through Uppsala Multidisciplinary Center for Advanced Computational Science (UPPMAX) under Project p2010057.

References (47)

S. Retout et al.
Development and implementation of the population Fisher information matrix for the evaluation of population pharmacokinetic designs
Computer Methods and Programs in Biomedicine
(2001)
M. Foracchia et al.
POPED, a software for optimal experiment design in population kinetics
Computer Methods and Programs in Biomedicine
(2004)
M. Tod et al.
Implementation of OSPOP, an algorithm for the estimation of optimal sampling times in pharmacokinetics by the ED, EID and API criteria
Computer Methods and Programs in Biomedicine
(1996)
L. Pronzato et al.
Robust experiment design via stochastic approximation
Mathematical Biosciences
(1985)
K. Ogungbenro et al.
The use of a modified Fedorov exchange algorithm to optimise sampling times for population pharmacokinetic experiments
Computer Methods and Programs in Biomedicine
(2005)
I. Gueorguieva et al.
A program for individual and population optimal design for univariate and multivariate response pharmacokinetic–pharmacodynamic models
Computer Methods and Programs in Biomedicine
(2007)
J.F. Box
R. A. Fisher and the design of experiments, 1922–1926
The American Statistician
(1980)
H. Cramér
Methods of estimation
Mathematical Methods of Statistics
(1946)
C.R. Rao
Information and accuracy attainable in the estimation of statistical parameters
Bulletin of the Calcutta Mathematical Society
(1945)
A.C. Atkinson et al.
Optimum Experimental Designs
(1992)

F. Mentré et al.

Optimal design in random-effects regression models

Biometrika

(1997)

F. Mentré, J. Nyberg, K. Ogungbenro, S. Leonov, A. Aliev, S. Duffull, C. Bazzoli and A. C. Hooker, Comparison of...

E. Sjögren et al.

Optimal experimental design for assessment of enzyme kinetics in a drug discovery screening environment

Drug Metabolism and Disposition

(2011)

S. Retout et al.

Further developments of the Fisher information matrix in nonlinear mixed effects models with evaluation in population pharmacokinetics

Journal of Biopharmaceutical Statistics

(2003)

J. Nyberg, S. Ueckert, A.C. Hooker, Approximations of the population Fisher information matrix – differences and...

T. Mielke et al.

Some considerations on the Fisher information in nonlinear mixed effects models

mODa 9 – Advances in Model-Oriented Design and Analysis

(2010)

V.V. Fedorov, S.L. Leonov, Optimization of sampling times for PK/PD models: approximation of elemental Fisher...

A. Dokoumetzidis et al.

Bayesian optimal designs for pharmacokinetic models: sensitivity to uncertainty

Journal of Biopharmaceutical Statistics

(2007)

M.G. Dodds et al.

Robust population pharmacokinetic experiment design

Journal of Pharmacokinetics and Pharmacodynamics

(2005)

M.D. McKay et al.

A comparison of three methods for selecting values of input variables in the analysis of output from a computer code

Technometrics

(1979)

J. Nyberg et al.

Simultaneous optimal experimental design on dose and sample times

Journal of Pharmacokinetics and Pharmacodynamics

(2009)

V.V. Fedorov et al.

Model-Oriented Design of Experiments

(1997)

C.G. Broyden

The convergence of a class of double-rank minimization algorithms 1. General considerations

IMA Journal of Applied Mathematics

(1970)

Cited by (63)

The effect of cold storage and cooking on the viability of Clostridioides difficile spores in consumer foods
2023, Food Microbiology
The increased detection of clinical cases of Clostridioides difficile coupled with the persistence of clostridial spores at various stages along the food chain suggest that this pathogen may be foodborne. This study examined C. difficile (ribotypes 078 and 126) spore viability in chicken breast, beef steak, spinach leaves and cottage cheese during refrigerated (4 °C) and frozen (−20 °C) storage with and without a subsequent sous vide mild cooking (60 °C, 1 h). Spore inactivation at 80 °C in phosphate buffer solution, beef and chicken were also investigated to provide D_80°C values and determine if PBS was a suitable model system for real food matrices. There was no decrease in spore concentration after chilled or frozen storage and/or sous vide cooking at 60 °C. Non-log-linear thermal inactivation was observed for both C. difficile ribotypes at 80 °C in phosphate buffer solution (PBS), beef and chicken. The predicted PBS D_80°C values of 5.72±[2.90, 8.55] min and 7.50±[6.61, 8.39] min for RT078 and RT126, respectively, were in agreement with the food matrices D_80°C values of 5.65 min (95% CI range from 4.29 to 8.89 min) for RT078 and 7.35 min (95% CI range from 6.81 to 7.01 min) for RT126. It was concluded that C. difficile spores survive chilled and frozen storage and mild cooking at 60 °C but may be inactivated at 80 °C. Moreover thermal inactivation in PBS was representative of that observed in real food matrices (beef and chicken).
Optimizing study design in LPS challenge studies for quantifying drug induced inhibition of TNFα response: Did we miss the prime time?
2022, European Journal of Pharmaceutical Sciences
In this work we evaluate the study design of LPS challenge experiments used for quantification of drug induced inhibition of TNFα response and provide general guidelines of how to improve the study design. Analysis of model simulated data, using a recently published TNFα turnover model, as well as the optimal design tool PopED have been used to find the optimal values of three key study design variables – time delay between drug and LPS administration, LPS dose, and sampling time points – that in turn could make the resulting TNFα response data more informative. Our findings suggest that the current rule of thumb for choosing the time delay should be reconsidered, and that the placement of the measurements after maximal TNFα response are crucial for the quality of the experiment. Furthermore, a literature study summarizing a wide range of published LPS challenge studies is provided, giving a broader perspective of how LPS challenge studies are usually conducted both in a preclinical and clinical setting.
Use of prior knowledge and extrapolation in paediatric drug development: A case study with deferasirox
2019, European Journal of Pharmaceutical Sciences
The characterisation of pharmacokinetics, pharmacodynamics and dose-exposure-response relationships requires data arising from well-designed study protocols and a relatively large sample from the target patient population. Such a prerequisite is unrealistic for paediatric rare diseases, where the patient population is often vulnerable and very small. In such cases, different sources of data and knowledge need to be considered to ensure trial designs are truly informative and oncoming data can be analysed efficiently. Here, we use clinical trial simulations to assess the contribution of historical data for (1) the analysis of sparse samples from a limited number of children and (2) the optimisation of study design when an increase in the number of subjects is not feasible. The evaluation of the pharmacokinetics of deferasirox in paediatric patients affected by haemoglobinopathies was used as case study. Our investigation shows that the incorporation of prior knowledge increases parameter precision and probability of successful convergence from only 12% with no priors to 56% and 75% for weakly and highly informative priors, respectively. In addition, results suggest that even when only one sample is collected per subject, as implemented in the original trial and in many other examples in clinical research, there is a 60% probability of biased parameter estimates (>25%). In conjunction with adult prior information and optimisation techniques, the probability of bias could be limited to <20% by increasing the number of samples/subject from 1 to 3. The methodology described here can be easily applied to other studies in small populations.
Towards a consistent understanding of the metal hydride reaction kinetics: Measurement, modeling and data processing
2018, Journal of Alloys and Compounds
Hydriding/dehydriding reaction kinetics of metal hydrides (MH) is an important yet controversial issue. In the last few decades, numerous studies have been dedicated to elaborate the reaction phenomena for various hydride materials. Some crucial aspects sketching the physical picture of the reaction, e.g. the controlling mechanism, Avrami exponent as well as apparent activation energy, are frequently discussed using the solid-state kinetic models. It is noted, however, that the results show considerable disagreement even for the material prepared in a single batch. From the literature review, it appears that no consistent understanding on the reaction kinetics of hydride materials could ever be achieved, unless carefully selected models and appropriate statistical processing techniques are used to interpret data obtained from well-designed experiments.
Least squares polynomial chaos expansion: A review of sampling strategies
2018, Computer Methods in Applied Mechanics and Engineering
Citation Excerpt :
In addition to the greedy algorithms discussed here, other techniques such as simulated annealing (SA) [80,81], genetic algorithms (GA) [82,83], semi-definite programming (SDP) [84,85] and pattern search [59,86] have been employed in the literature to construct alphabetic optimal designs. Moreover, software packages such as JMP [87], POBE [88], AlgDesign [89], PopED [90,91] and Gosset [92] have been developed to construct ODE using the algorithms discussed here. We note that the majority of the software packages for the construction of ODE are developed for scenarios where low-order (mostly quadratic) polynomials are employed, hence, limiting the applicability of such packages for high-order PCE’s.
As non-intrusive polynomial chaos expansion (PCE) techniques have gained growing popularity among researchers, we here provide a comprehensive review of major sampling strategies for the least squares based PCE. Traditional sampling methods, such as Monte Carlo, Latin hypercube, quasi-Monte Carlo, optimal design of experiments (ODE), Gaussian quadratures, as well as more recent techniques, such as coherence-optimal and randomized quadratures are discussed. We also propose a hybrid sampling method, dubbed alphabetic-coherence-optimal, that employs the so-called alphabetic optimality criteria used in the context of ODE in conjunction with coherence-optimal samples. A comparison between the empirical performance of the selected sampling methods applied to three numerical examples, including high-order PCE’s, high-dimensional problems, and low oversampling ratios, is presented to provide a road map for practitioners seeking the most suitable sampling technique for a problem at hand. We observed that the alphabetic-coherence-optimal technique outperforms other sampling methods, specially when high-order ODE are employed and/or the oversampling ratio is low.
Computational aspects of experimental designs in multiple-group mixed models
2024, Statistical Papers

View all citing articles on Scopus

¹: Authors contributed equally to this work.

²: Present address: School of Pharmacy, University of Queensland, Brisbane, Australia.

View full text

PopED: An extended, parallelized, nonlinear mixed effects models optimal design tool

Abstract

Introduction

Section snippets

Optimal design

Fisher information matrix

Graphical user interface

Example 1: Group size optimization

Discussion

Conflict of interest

Acknowledgements

Computer Methods and Programs in Biomedicine

Computer Methods and Programs in Biomedicine

Computer Methods and Programs in Biomedicine

Mathematical Biosciences

Computer Methods and Programs in Biomedicine

Computer Methods and Programs in Biomedicine

R. A. Fisher and the design of experiments, 1922–1926

The American Statistician

Methods of estimation

Mathematical Methods of Statistics

Information and accuracy attainable in the estimation of statistical parameters

Bulletin of the Calcutta Mathematical Society

Optimum Experimental Designs

Optimal design in random-effects regression models

Biometrika

Optimal experimental design for assessment of enzyme kinetics in a drug discovery screening environment

Drug Metabolism and Disposition

Further developments of the Fisher information matrix in nonlinear mixed effects models with evaluation in population pharmacokinetics

Journal of Biopharmaceutical Statistics

Some considerations on the Fisher information in nonlinear mixed effects models

mODa 9 – Advances in Model-Oriented Design and Analysis

Bayesian optimal designs for pharmacokinetic models: sensitivity to uncertainty

Journal of Biopharmaceutical Statistics

Robust population pharmacokinetic experiment design

Journal of Pharmacokinetics and Pharmacodynamics

A comparison of three methods for selecting values of input variables in the analysis of output from a computer code

Technometrics

Simultaneous optimal experimental design on dose and sample times

Journal of Pharmacokinetics and Pharmacodynamics

Model-Oriented Design of Experiments

The convergence of a class of double-rank minimization algorithms 1. General considerations

IMA Journal of Applied Mathematics