Computing Stackelberg Equilibrium for Cancer Treatment

Abstract

Recent work by Kleshnina et al. has presented a Stackelberg evolutionary game model in which the Stackelberg equilibrium strategy for the leading player corresponds to the optimal cancer treatment [4]. We present an approach that is able to quickly and accurately solve the model presented in that work.

1 Introduction and problem formulation

In this section we review the Stackelberg evolutionary game dynamic model of cancer evolution previously studied [4]. There are two players: a follower and a leader. The leader is a physician who selects amounts of two different drugs to use for therapy, m₁ and m₂. The follower is a cancer population consisting of three cell types: 0-type denotes a cell that does not develop resistance to either drug, 1-type cells are resistant to just drug 1, and 2-type cells are resistant to just drug 2. The follower selects a population size for each type, denoted x₀, x₁, x₂, as well as a trait for each type, denoted u₀, u₁, u₂. It is assumed that each of these variables are nonnegative, with the u_i ∈ [0, 1]. It is also assumed that they are all implicitly functions of time t. Note that u₀ does not appear anywhere in the analysis so can be ignored.

For each cell type i there is a fitness function G_i(u_i, m, x) that the follower is trying to maximize. We assume that the dynamics of the population x are governed by In order to ensure that we are in equilibrium of the ecological dynamics we must have that for i = 0, 1, 2. Thus, the follower is selecting u₁ that maximizes G₁, u₂ that maximizes G₂, and x₀, x₁, x₂ that ensure equilibrium of the ecological dynamics. The leader, knowing that the follower will subsequently select their actions in this way, selects m₁, m₂ to maximize a quality of life function Q(m, u, x).

Thus, we can formulate the problem of determining the optimal strategies for both players as follows: This general model is instantiated by the following functional forms for the fitness functions G₀, G₁, G₂ and quality of life function Q. Note that the model presentation is slightly different between the paper [4] and the implementation in the code repository [3]. We will be using the model presented in the code. The model has several parameters, whose interpretations are summarized in Table 1. Note that in the code additional parameters a₀, a₁, a₂, a₃ are defined, with

Table 1: Interpretations of model parameters

2 Prior approach

In this section we present the approach described in Github repository created by the authors [3]. They first solve for expressions for x_i in terms of u_i, m_i such that the condition for equilibrium of the ecological dynamics is satisfied. This involves solving a system of three equations G_ix_i = 0 with unknowns x₁, x₂, x₃. They calculate the following analytical solution: where X₀₁, X₀₂, X₀₃ denote the following quantities: While it is not given in the model formulation, the code assumes that m₁ and m₂ fall in the interval , with . They discretize the space so that m₁ and m₂ are multiples of h = 0.1, and iterate over all combinations, of which there are . For each combination of values they calculate the optimal values of u₁, u₂ as follows. They assume that the optimal value of u₁ is where , and the optimal value of u₂ is where . They calculate the following expressions for the partial derivatives: To find values of u₁, u₂ for which these derivatives equal zero, they perform an optimization to minimize the quantity . They perform this optimization in Matlab using the fminsearchbnd function, using bounds 0 ≤ u₁ ≤ 1, 0 ≤ u₂ ≤ 1. Note that the above expressions for which are in terms of m_i and u_i are substituted into the expressions for , so that u_i is the only variable in the objective. This procedure results in optimal values for each combination of values for (m₁, m₂). The values for are determined by these values.

Finally, they iterate over all of these combinations of values to determine which one maximizes the value of Q. This determines the optimal values of for the leader, which in turn determine the optimal strategies for the follower. The overall procedure is summarized in Algorithm 1.

Algorithm 1

Prior approach

3 Limitations of prior approach

The prior approach has several significant limitations. The first is that it does not check that the calculated values for for given values of u_i and m_i are biologically sensible. Since the x_i correspond to populations, they must be nonnegative.

Another limitation is that the coarse discretization of values for m_i means that only a small number of possibilities are considered. This also means that the running time will potentially be large, since we must perform separate optimizations.

Another significant limitation is that it is assumed that G_i is maximized when , and the boundary cases when it is maximized at u_i = 0 or 1 are ignored.

A final limitation is that the procedure invokes the fminsearch algorithm in Matlab, which is not even guaranteed to find a local minimum, let alone a global minimum.

4 New approach

We now describe our new approach that addresses the limitations of the prior approach. We will formulate a single quadratic program that corresponds to the full optimization problem and solve it using Gurobi’s nonconvex MIQCP solver which has a guarantee of global optimality (subject to numerical precision).

First we have the main decision variables x_i, u_i, m_i with x_i ≥ 0, m_i ≥ 0, 0 ≤ u_i ≤ 1. The objective function Q is a quadratic function of these variables. Next we encode the conditions for equilibrium of the ecological dynamics. We must define several auxiliary variables to do this.

First define η_i = g_iu_i, and for i = 1, 2. For the latter, we use Gurobi’s addGenConstrExp function that uses a piecewise linear approximation for the exponential function. We set these variables to be nonnegative. To provide tighter upper bounds we can set , since u_i ≤ 1. We next define the auxiliary variable .We can do this by including the quadratic constraint k_iγ_i+bu_iγ_i − m_i = 0 for i = 1, 2. Using these variables we can now encode the conditions for equilibrium of ecological dynamics using constraints that are quadratic in the variables.

Next we must encode the conditions that u_i is a maximizer of G_i. To do this we define several additional auxiliary variables. We define by adding in the constraint σ_iτ_i = 1. Next we define . We can do this by including the quadratic constraint . Finally wedefine by including the quadratic constraint ω_iβ_i − m_i = 0. Using these variables, we can now encode the expressions for that are quadratic in the variables.

Recall that we are trying to select u_i ∈ [0, 1] to maximize G_i, for i = 1, 2. We can do this by introducing two Lagrange multipliers λ_i1 ≥ 0, λ_i2 ≥ 0. Then the KKT optimality condition is equivalent to the following three constraints: These constraints are all quadratic in the variables and ensure that we find u_i ∈ [0, 1] that maximizes G_i regardless of whether it is at the boundary or at an interior solution with the derivative equal to zero.

Our full formulation is given below. Here the X_ij correspond to the same quantities as before and are just defined to simplify presentation, not as new variables. This formulation addresses the limitations of the prior approach. It ensures that all quantities are biologically relevant by imposing nonnegativity constraints on corresponding variables. It allows m_i to take on arbitrary nonnegative values, not a small set of discretized values. It involves solving a single optimization problem instead of separate optimization problems. It uses KKT conditions to ensure that the values of u_i that maximize G_i are found regardless of whether they are interior or boundary solutions. And the approach guarantees finding a global optimum since that is guaranteed by Gurobi’s nonconvex MIQCP solver.

5 Experiments

We ran experiments with both approaches on a problem instance using the same parameter values as the prior approach [3], which are provided in Table 2. All experiments were done on a single core of a laptop using Windows 11. Experiments with the prior approach used Matlab version 24.1.0.2689473 (R2024a) Update 6 [2], and experiments with the new approach were done using Gurobi version 11.03 [1] with Java version 14.0.2. For the optimizations in the prior approach, Matlab’s function fminsearchbnd was called using parameters TolX = 1 × 10⁻¹², MaxFunEvals = 1000. The results are shown in Table 3. We can see that the prior approach found a solution with a negative value for , which is not biologically sensible. The prior approach took nearly five minutes while our new approach took less than two seconds.

Table 2: Parameter values used in experiments

Table 3: Experimental results for both approaches

6 Conclusion

We presented a new approach for computing Stackelberg equilibrium strategies in a Stackelberg evolutionary game dynamic model of cancer evolution previously studied. Our approach is based on solving a new quadratic program formulation. We noted several limitations of the approach used by prior work which are addressed by our approach. When we compared the approaches on a sample instance our approach ran significantly faster and the prior approach output a solution that is not biologically relevant. As more complex game-theoretic and optimization models are being formulated for problems in biology and cancer treatment in particular, it is important to develop efficient algorithms for accurately solving them. While we focused on one instantiation presented in prior work, our approach is applicable more generally to computing optimal strategies in Stackelberg evolutionary games.