Artificial Metabolic Networks: enabling neural computation with metabolic networks

Abstract

Metabolic networks have largely been exploited as mechanistic tools to predict the behavior of microorganisms with a defined genotype in different environments. However, flux predictions by constraint-based modeling approaches are limited in quality unless labor intensive experiments including the measurement of media intake fluxes, are performed. Using machine learning instead of an optimization of biomass flux – on which most existing constraint-based methods are based – provides ways to improve flux and growth rate predictions. In this paper, we show how Recurrent Neural Networks can surrogate constraint-based modeling and make metabolic networks suitable for backpropagation and consequently be used as an architecture for machine learning. We refer to our hybrid - mechanistic and neural network – models as Artificial Metabolic Networks (AMN). We showcase AMN and illustrate its performance with an experimental dataset of Escherichia coli growth rates in 73 different media compositions. We reach a regression coefficient of R²=0.78 on cross-validation sets. We expect AMNs to provide easier discovery of metabolic insights and prompt new biotechnological applications.

Introduction

The increasing amounts of data available for biological research brings the challenge of data integration with machine learning (ML) methods. Systems and Synthetic Biology along with Metabolic Engineering are no exception to this trend [1][2][3][4]. Metabolic Engineering relies on models that predict the phenotype of a strain from its genotype and environment. In the past three decades, Constraint-Based Modelling (CBM) has been the main approach to study the relationship between the uptake of nutrients and the metabolic phenotype (i.e., the steady-state flux distribution) of a given organism, e.g., E. coli, with a model iteratively refined for 30 years [5]. The main pitfall in CBM is the under-determination of the system. As a result, CBM needs to be constrained to obtain realistic flux distributions. Constraining consists in setting bounds for some fluxes and restraining the possible solutions of the model to the ones compliant with the constraints. To find these constraints, fluxomic datasets can be obtained from different experimental flux determination methods. With isotopic labeling (like 13-C), one can follow the path of a nutrient in the metabolic network [6]. With metabolomics methods, one can derive metabolic fluxes from metabolite concentrations and possibly in a time-resolved approach [7]. With transcriptomics methods, RNA sequencing data is used as input for models estimating fluxes in an indirect fashion, which, recently has even been achieved at the single-cell level [8]. In some cases, data integration of several -omics methods is possible, constituting a state-of-the-art multi-omics data integration for Metabolic Flux Analysis (MFA) [9][10][11]. Naturally, ML approaches were developed to efficiently integrate the data and enhance the predictive power of CBM. However, as described by Sahu et al. [12], in MFA, the interplay between CBM and ML is still showing a gap: some approaches use ML as input for CBM, others use CBM as input for ML, but none of them can do both, like we attempt to do in this paper with the Artificial Metabolic Networks (AMNs) hybrid model.

The lack of progress towards integrating mechanistic modelling (MM, to which CBM belongs) and machine learning in fundamental biology studies is at odds with the widespread adoption of ML approaches by the life science communities. MM and ML approaches are based on two different paradigms. While the former is aimed at understanding biological phenomena, some systems are too complex to be modeled with interpretability. Conversely, the latter does a good job predicting the outcomes of complex biological processes using large input/output datasets, but does not provide understanding of the underlying mechanisms. The pros of one are the cons of the other, suggesting that a hybrid approach should be developed towards enabling a symbiotic relationship between both. An emerging field, Scientific Machine Learning (SciML), aims to develop such hybrid models [13]. The main advantage of hybrid modeling is to bring models that comply well with experimental results via ML, but that also take mechanistic insights from MM. Recently, Nilsson et al. [14] showed good performances of such a model, developed for signaling networks, where they used the supposed topology of a signaling network as a platform for learning on experimental data, therefore displaying more insights than a black-box approach. Also, Anantharaman et al. [15] showed that an implicit ML architecture, here a reservoir specifically designed for surrogating stiff mechanistic Ordinary Differential Equations (ODEs) used in quantitative systems pharmacology, can function in a more accurate, fast and robust fashion than classical ODE solvers, once these implicit architectures have been trained on mechanistic simulations. Another example of hybrid modeling for biological systems is the work of Lagergren et al. [16], who estimated the parameters of a mechanistic model using “biologically-informed neural networks”. Finally, Culley et al. [17] showed how simulating data with a mechanistic metabolic model could enhance machine learning approaches to characterize S. cerevisiae growth and bring more biological insights.

The hybrid AMN model shown here fits in the emerging SciML field. AMNs bridge the gap between ML and CBM by solving linear programming (LP) problems for metabolic flux models with a recurrent neural network (RNN) that has the same topology as the metabolic network itself. By doing so, our model is a mechanistic model, determined by the stoichiometry and any other possible constraint of CBM, but also a ML model, as it can be used as a learning architecture, with any MFA suitable data.

The use of RNNs for solving optimization problems is a long-standing field of research [18] inspired by the pioneering work of Hopfield and Tank [19]. A few years later, RNNs were showcased to perform well for solving linear problems [20]. Simpler and more efficient networks were developed over the years [21][22] integrating both linear and Quadratic Programming (QP) problems to be in the reach of the network. In this study, we were able to use those RNNs for solving the LP optimization problems of CBM.

CBM has a critical limitation for experimentalists: while realistic and condition-dependent bounds on uptake rates would be critical for growth rate predictions, the conversion from extracellular concentrations, i.e., the controlled experimental setting, to such bounds on uptake rates, is unknown. The satFBA [23] method exploits kinetic models to predict this conversion, relying on a saturation value for each exchange reaction. Consequently, this MM method relies on assumptions, not necessarily valid, and knowledge, not necessarily accessible, and does not integrate any large-scale regulation phenomena. As a result, classical CBM can predict growth yields but can hardly be used to predict actual growth rates [24]. AMNs are showcased in this study for tackling the same issue: predicting metabolites uptake rates from their external concentrations. To do so, where satFBA uses another layer of mechanical kinetic modelling, we use a pre-processing dense neural layer, that is accessible for learning thanks to the AMN gradient backpropagation abilities.

AMNs provide a new paradigm for MFA: instead of predicting fluxes and growth rates from an optimality principle (suited for CBM), we do so by learning from known flux distributions. First, we develop models that can learn from flux distributions and show their good performance. The flux distributions used as learning references (the training sets) are produced by parsimonious Flux Balance Analysis (pFBA), maximizing the biomass reaction (i.e., the growth rate) and subsequently minimizing all the other fluxes (with the biomass reaction fixed at its optimal value). Once the model has been trained on adequate simulated pFBA data, we immobilized its parameters, resulting in a gradient backpropagation compatible reservoir. This reservoir is then used to tackle the abovementioned issue of unknown uptake rates. A dense neural network layer predicting uptake rates from external metabolite concentrations is then trained to feed uptake rate to the reservoir. That dense layer can be reused by any FBA user to improve the predictive power of a metabolic model, with an adequate experimental set-up.

Results

We first present the basic design and functioning of two alternative neural computation methods that surrogate CBM, i.e., the prediction of intracellular fluxes and growth rate based on predefined bounds on cellular uptake rates. We use simulation data as reference, generated with pFBA, with different models of different sizes and different media compositions. Note that for this part, our methods are simply mimicking predictions by classical CBM with a neural network approach. Importantly, they are not AMNs per se, but are core methods that ‘replace’ CBM in our AMNs. A crucial step before establishing neural network approaches on metabolic networks is to ensure that all reactions are unidirectional which has been done by splitting all bi-directional reactions in the model into forward and backward reactions. In the resulting model, all reaction fluxes are positive, and flux cycles within a single reaction are suppressed by the usage of pFBA (see Methods - Making metabolic networks suitable for neural computations).

The first method (Fig. 1) learns a weight matrix from example flux distributions, which represents consensual flux branching ratios found in the training set. Consequently, we assume that a consensual metabolic state exists for most of the flux distributions examples of our training set. As a result, rules for generating the training sets are critical for good performance of this method, those are detailed in Methods – Generation of training sets with COBRApy. A simple toy model network is shown to demonstrate the functioning of this first method in Fig. 1A. With CBM, the flux distribution maximizing the rate of the ‘BIOMASS’ reaction providing a nutrient uptake through reaction ‘TPI’ is obtained by solving a linear program (Fig. 1B). At steady state, metabolite production fluxes can be calculated from reaction fluxes via the transition matrix V2M from fluxes to metabolites (V2M_i,j = {S_i,j if S_i,j > 0, 0 elsewhere} more details are given in Fig. 1C), similarly reaction fluxes can be calculated from metabolite production fluxes using the matrices M2V and W^sst. For a given metabolite i and a reaction j, the weight w_ij (in matrix W^sst) represents the fraction of metabolite i production flux going to reaction j, and is therefore flux distribution dependent. Here we assume that the branching ratios remain similar between flux distributions, then attempt to learn “typical” ratios from flux data.

Fig. 1. Computing steady-state fluxes with stoichiometric and neural models.

(A) Simple stoichiometric model. The model corresponds to the upper Glycolysis pathways of E. coli (iML1515 model extracted from the BiGG database [25]). The model is unidirectional, all fluxes values are positive. (B) Steady-state solution fluxes maximizing biomass production. At steady-state, the reaction fluxes (v_i) must satisfy stationarity conditions that guarantee mass balance of all metabolites, this is depicted by the equation SV = 0, where S is the stoichiometric matrix representing the connectivity of the model and V the vector of fluxes to be calculated. V⁰ is the medium represented by a vector of nutrient uptake fluxes (here v₁= TPI = 0.1, symbol “–” indicates that no value is provided). The steady-state solution V^sst is calculated by solving a linear program maximizing the objective c^TV = v₃ = BIOMASS, here we used the COBRApy package [26] to compute V^sst. (C) Stoichiometric model matrices. V2M is the matrix to compute metabolite production fluxes from reaction fluxes, M2V is a matrix to compute reaction fluxes from the production fluxes of the reaction substrates. When a metabolite is the substrate of several reactions, each reaction gets a fraction of the metabolite production flux, this is depicted in matrix W^sst. M^sst is the vector of metabolite production fluxes at steady-state, the operator ⊙ stands for element-wise matrix product, and ReLU(x)=max (0, x). (D) Unrolled neural network built from the model. In this theoretical example an initial flux vector is set, representing only the fluxes in the exchange reactions, is mapped to a flux vector covering the entire network. In the initial layer (l⁰) only v₁ has a value . In layer 1, v₁ value is passed to m₁, the production flux for metabolite 1, . Subsequently a fraction (w₂₁) of m₁ goes to v₂ and the other fraction (w₃₁) to v₃, therefore with w₂₁+w₃₁ = 1. Additionally, v₁ remains as in . In layer 2, v₁ continues to feed m₁, v₂ is passed on to m₂ and m₃, therefore, and , m₂ then goes to v₃ and v₄, and as in the previous layer we have with w₃₂+w₄₂ = 1, other fluxes remain the same as in l¹, i.e., . In layer 3, m₃ receives input from v₄ which in turn activates v₅. In layer 4, m₁ receives input from both v₁ and . (E) Recurrent neural network (RNN) representation. At each iteration step (l) V^l and M^l are computed using matrices V2M and M2V of panel C, while the matrix W can be learned by training with experimental data or model simulations. For example, setting and searching weights for which one finds w₂₁ = 0.74, w₃₁ = 0.26, w₃₂ = 0.20, and w₄₂ = 0.80. Taking these weights, the Vⁿ values obtained for n=30 match those of panel B. (F) Heatmap for flux values up to 30 RNN iterations.

To learn the weights w_ij, one first needs to translate the model into a neural-network-like architecture (Fig. 1D). Precisely, in Fig. 1D we start from an initial set of given fluxes and then propagate knowledge about the fluxes through the entire network, each layer corresponding to one step in flux propagation. Mathematically, each layer is composed of two simple operations that update the M and V vectors, respectively representing metabolites production fluxes and reaction fluxes. Those operations are repeated until convergence, with v₁ being constant because it is known from the start, and the equilibrium is reached in cycles (for instance {v₂, v₄, v₅}). The network shown in Fig. 1D is the unrolled representation of the Recurrent Neural Network (named RNN-W) depicted in Fig. 1E. As it will be discussed later, the matrix W can be learned through training, assuming the learned weights are those of the matrix W^sst (Fig. 1C) the steady state fluxes of RNN-W iterated 30 times (or more) equal to those obtained solving MFA with CBM’s linear programs (Fig. 1F).

While the RNN-W method is simple to implement it has several drawbacks. First, the method is not performant enough for learning on large metabolic models and datasets, notably because of the large number of parameters (cf. Table 1, architecture RNN-W). More problematic is the fact that a fixed set of weights – corresponding to assumed fixed branching ratios in flux distributions - may not fit all instances of a training set, since these branching ratios are typically changing (for example, in a switch between overflow metabolism and respiration). Supplementary Fig. S1 shows an example in which two flux distributions, with different uptake fluxes, lead to different weights. Consequently, we cannot assume that our model will precisely represent very diverse flux distributions. To overcome this shortcoming, we next present an alternative RNN method for solving CBM optimization problems. These other methods are much closer to the LP optimization behind CBM: their input can be exact or upper bounds on fluxes, and an objective reaction (or set of reactions) to optimize can be set – or not.

Table 1. AMN performance.

All SBML models from different strains were downloaded from the BiGG database [25]. AMN architectures and parameters section. ⁽¹⁾ Training set size and ratio of variable compounds in medium turned on, method used when running COBRApy [26] (with pFBA) ⁽²⁾ All: all fluxes are calculated, Biomass: only the growth rate is calculated. ⁽³⁾ Number of trainable parameters and epochs, in all cases dropout = 0.25, batch size = 5, the optimizer is Adam and the loss function is the mean squared error between predicted and provided fluxes to which the value IISVII² is added (which should be near zero) ⁽⁴⁾ Measured on 2.4 GHz 8-Core Intel Core i9. ⁽⁵⁾ Regression coef. and IISVII² values for X-fold = 5-fold on training and validation sets ⁽⁶⁾ Regression coef. and IISVII² values on an independent test set of size 1000.

Our second method is inspired by the works of Ghasabi-Oskoei et al. [21] and Yang et al. [22]. The method is still a core method that aims to ‘replace’ CBM but is not the AMN itself. The method makes use of RNN to solve linear and quadratic problems, using exact constraint bounds (EB) in Ghasabi-Oskoei et al. and upper-bounds (UB) in Yang et al. In the same way as the RNN-W method, RNN-EB and RNN-UB iteratively compute fluxes to come closer to the steady-state solution. However, calculations are more sophisticated (Fig. 2A-B) and integrate the same objective function as in classical CBM (cf. Method - Solving LP/QP with RNN methods - for further details). These RNNs can achieve almost identical results as pFBA. In both cases we used the same initial conditions as with pFBA: known intake fluxes with EB and upper bound for unknown intake fluxes with UB. The results obtained by the two RNNs are compared with the flux value to optimize (growth rate in Fig. 2C) or with the global regression coefficient (R²) on all fluxes (Fig. 2D). While the match with pFBA is excellent for the prediction of growth rates, it becomes a bit worse when comparing the predictions for intracellular fluxes. This is due to the fact that the flux solutions of pFBA can be non-unique: many pFBA calculated steady state flux distributions may lead to the same growth rate, so for any given growth rate there is no guarantee that the RNN calculated fluxes will match the pFBA fluxes.

Fig. 2. RNN architectures and performances.

(A) RNN-EB. The constrained linear optimization problem (eqs. 1 and 2 in Method section) was solved with a gradient based method that iteratively update flux values and that is proven to converge to the same solution as (1). The solution is called “exact bounds” (EB) because the intake fluxes are supposed to be known [21]. The notations are those of Fig. 1 for V, V⁰, S and c, while b = V2M V⁰. M is a vector representing the variables of the dual problem (also named metabolite shadow prices [27]), R is a vector used to simplify the notations, and δt is a constant (representing the increment at the t^th iteration). (B) RNN-UB. The general principle of this method is the same as the previous one. However, the equation system (eq. 3 in Method section) that is used allows for inequality boundaries, so it is called “upper bounds” (UB). This method makes used of an architecture developed by Yang et al. [22] solving linear and quadratic problems given upper bounds. S_ext and S_int are the stoichiometric matrices of the network taking respectively external and internal metabolites and reaction fluxes. These matrices are computed from S by removing rows and columns (cf. Fig. S2). , and P = QS_int (cf. Method – Solving LP/QP with RNNs). (C) AMNs predicted growth rate vs. FBA calculated growth rate on the same inputs and the same model (E. coli core) for both RNN-EB and RNN-UB. (D) Regression coefficient R² improvement for all reaction fluxes of E. coli core model with the number of iterations (t).

While RNN-EB and -UB perform well, their main weakness is the number of iterations needed to reach satisfactory performances, at least 400,000 (Fig. 2D). Since our goal is to integrate such RNNs in a learning architecture, this drawback has to be tackled. As illustrated in Fig. 3A, our solution is to improve our initial guesses for fluxes, by training a prior network (a classical dense ANN architecture) to compute initial values for all fluxes (V⁰) from medium intake fluxes (Vⁱⁿ). This prior network is trained along with the RNN with the goal of finding V⁰ values that are as close as possible to the optimum flux values (pFBA steady state fluxes). This is achieved by setting the iteration number of the RNN to low values.

Fig. 3. AMN architectures enabling training.

(A) Basic architecture principle. Vⁱⁿ is the vector representing upper or exact bounds on exchange reactions, i.e., the medium uptake rates. Each white circle represents one reaction. V⁰ contains the initial fluxes predicted by a dense, fully connected layer. Vⁿ is the final flux vector, according to the mechanistic constraints applied on V⁰ and resulting from S: mass and charge conservation imposed by SV=0. The architectures RNN-W, RNN-EB and RNN-UB of Figs. 1 and 2 can be used for the mechanistic layer. (B) Architecture example for RNN-EB. Here the weight matrix W_θ is obtained via training to compute initial values for V and M, then the RNN-EB cell is run iteratively to return Vⁿ. (C) Architecture example for RNN-W. Here the fully connected dense layer is plugged inside the recurrent cell containing the mechanistic layer. At each iteration V⁰ is computed via W_θin obtained through training. V and M are updated via W_θr (also obtained via training) and the values of V and M at the previous iteration. M is calculated using the M2V matrix defined in Fig. 1.

In the remainder of the paper, we name Artificial Metabolic Network (AMN), the hybrid model shown in Fig. 3A composed of a prior dense network and an RNN. We note that the RNN acts as a mechanistic layer as it contains the stoichiometry and bounds of a metabolic network. In Fig 3B and 3C we present two examples of AMNs making use of RNN-EB and RNN-W respectively, further details can be found in the section Method - AMN architectures and parameters. Performances of our AMNs are compared with other more classical architectures in Table 1. Overall, we find that the architecture RNN-EB provides the best performances, especially for cross-validation independent test sets.

Once an AMN has been trained on a large dataset of pFBA simulations, we can exploit it in subsequent, further learning, this time on experimental data. Precisely, we used the iML1515 AMN of Table 1 for characterizing and predicting the growth of E. coli on many different media compositions. To that end, we grew E. coli DH5-alpha in 73 different media compositions, with M9-glycerol as a basis and a wide variety of possibly added nutrients (see Methods - Culture conditions). As already mentioned in the introduction section, the intake rates of E. coli nutrients (i.e., the nutrient exchange reactions values), as well as their relation to external concentrations, remain largely unknown: the quantitative rate for each compound may vary between growth media, unlike in the FBA calculations, where the same upper bound (or zero, if a compound is absent) is used in all cases. This is a common flaw of classical FBA (one that is only partially remedied in satFBA or FBA with molecular crowding [28]). To use the architectures of Fig. 3, we thus need to first convert nutrient concentrations into uptake rates. This is achieved by adding a dense layer that precedes the AMN (Fig 4A). In this architecture the AMN is no longer trainable, only the prior dense layer is, and the AMN acts as a reservoir as in reservoir computing [29].

Fig. 4. Benchmarking growth rate predictions with experimental measurements for E. coli (strain DH5-alpha, model iML1515).

(A) AMN used in reservoir computing. The non-trainable reservoir (rounded box) is the AMN shown in Figure 3.B (with performances given the last row of Table 1) and is connected to a prior trainable network which purpose is to compute medium intake fluxes (Vⁱⁿ) from the concentration (Cⁱⁿ) of metabolites added to medium. Taking as input Vⁱⁿ, the non-trainable reservoir prints out all fluxes including the growth rate. (B) AMN (RNN-EB architecture) predicted growth rate vs. measured growth rate. All points plotted correspond to predicted values not present in the training set. This was compiled using all predicted values obtained for each validation sets in 10-fold stratified cross validation repeated 3 times with different initial random seeds. (C) COBRApy calculated growth rate vs. measured growth rate. COBRApy was run taking as input upper bound intake fluxes for added metabolites in medium. Intake flux values were set to arbitrary values (0 when the metabolite was absent in the medium and 10 when it was present). (D) COBRApy calculated growth rate vs. measured growth rate with optimized medium intake fluxes (see section Method - Optimization of exchange reaction - for details on optimization procedure).

The predictive power of our AMN-reservoir was assessed by a regression coefficient R² = 0.96±0.02 on training sets; and R² =0.78 (Fig. 4B) on a test set build from aggregated validation sets during 10-fold cross-validation. We obtained similar performance with a state-of-the-art black-box model (XGBoost [30]) and a simple multivariate regression model, with the Ordinary Least Squares (OLS) method of the statsmodels [31] python package. However, those black-box models have no mechanistic insights to propose, and no possible modifications that have a biological sense, like a knockout of a gene that would delete a reaction in the metabolic network, which can be accounted for in our AMNs. To compare with an ‘out-of-the-box’ CBM approach, we first used an arbitrary value of 10 for the present compounds, and 0 for the absent compounds. Applying these values on the corresponding exchange reactions upper bounds, and optimizing the biomass reaction rate produced results uncorrelated with experimental measurements (Fig. 4C). To enhance these results, we optimized the upper bound values of exchange reactions corresponding to absent or present compounds in the medium (see Methods - Optimization of exchange reactions upper bounds in FBA). It resulted in a R² = 0.47 performance (Fig. 4D), showing a poor fit and weak predictive power (no cross-validation scheme here), indicating AMNs have a purpose in augmenting the capabilities of CBM without costly experimental work.

Discussion and conclusion

In this study, we showed that metabolic networks can be used as a learning architecture for neural network approaches. Previous work on RNN for constrained optimization was re-used and adapted to enable machine learning methods (such as backpropagation of errors) within metabolic networks. For improved scalability and adaptability, we first trained an AMN-reservoir on large, simulated pFBA datasets. We then exploited the reservoir to improve the predictive power of MFA on growth rate datasets of E. coli. Figure 4 shows good predictive power of AMN, far outperforming those obtained by standard CBM solvers (like COBRApy pFBA solver in Fig. 2C and 2D).

Determining the uptake rates is a core experimental work needed for making constraint-based MFA realistic. Here, we developed an approach that gets rid of such needs for reaching plausible fluxes distributions. We did so by backpropagating the error on the growth rate, to find complex relationships between the medium concentrations and the medium uptake rates. To that end, we demonstrated the high predictive power of AMNs, and their re-usability in classical CBM approaches. Indeed, the constraint-based methods developers and users can now make use of our AMN-reservoir method for solving the medium uptake rate issues. In this regard, supplementary Table S2 gives uptake rates for the metabolites used in our benchmarking work with E. coli (Fig. 4), which can directly be used with CBM software products like COBRApy.

As mentioned earlier, in classical CBM, plausible flux distributions can be found by constraining some fluxes to measured flux values. Here, we avoided costly experimental flux determinations and found plausible flux distributions just by learning the consensual metabolic behavior of an organism in response to its environment. We also learned the relationship between medium nutrient concentrations and the actual nutrient uptake by the bacteria, eventually revealing complex regulations between E. coli’s environment to its steady-state metabolic phenotype. Added to the possible unveiling of biological mechanisms, AMNs can also be exploited for industrial applications. Indeed, since one can design any objective function to optimize with AMNs, they can be used to search optimum media for the bioproduction of compounds of interest as well as microorganism-based decision-making devices for the multiplexed detection of metabolic biomarker or environmental pollutants.

Methods

Making metabolic networks suitable for neural computations

AMNs should have a property that is not guaranteed by usual metabolic models: all reactions must be unidirectional. In other words, AMN computations have been developed for models showing positive-only fluxes. We wrote a standardization script that loads an SBML model into COBRApy and screens for all two-sided reactions, then duplicating them into two separate reactions; and writes a new version of the model with bi-directional reactions split into separate forward and backward reactions. To avoid confusion, we add a suffix for these reactions, either “for” or “rev” respectively designating the original forward reaction and the reversed reaction. For more clarity, the exchange reactions were also duplicated, even if encoded as one-sided, and their suffix was set to “i” for inflow reactions, and “o” for outflow reactions.

Generation of training sets with COBRApy

Training sets of metabolic flux distributions were generated using models downloaded from the BiGG database [25]. The models were used to generate data using COBRApy [26] following a precise set of rules. First, we identified essential exchange reactions for the models we used (E. coli core and iML1515). Precisely, if one of these reactions has its upper bound set to zero, the biomass reaction optimization is impossible, even if all other exchange reactions are set to a high value, e.g., 1000. In other words, we identified the minimal uptake fluxes that enable growth, according to the models. During data generation, the upper bounds on these reactions were always set to 1000.

Two carbon sources always had their exchange reaction turned on, glycerol and arabinose, which were given an upper bound of 10. Then, we selected a set of 27 exchange reactions to be the variables of our training set: the 20 canonical proteinogenic amino acids (L-histidine, L-leucine, L-phenylalanine, L-lysine, L-tyrosine, L-valine, L-leucine, L-methionine, L-isoleucine, L-glutamine, L-cysteine, L-serine, L-alanine, L-tryptophan, L-asparagine, L-proline, L-threonine, Glycine, L-arginine, L-aspartate), 4 sugars (Melibiose, Sucrose, Trehalose, Lactose) and 3 acids (Acetate, Pyruvate, Citrate). Note that these exchange reactions corresponding compounds are used as the possibly added compounds in the experimental datasets.

These constitute the so-called set of variable exchange reactions (VER). On these reactions, we first applied a binomial drawing rule B(n, p) with n=27 and p ∈ [0, 1], a tunable parameter related to the ratio of ‘activated’ VER (to have an upper bound larger than zero). Consequently, the mean number of activated variable exchange reactions was close to n x p. For each of those activated VER, one out of several levels were randomly drawn from a uniform law, to give the upper bound actual value. In the workflow, the number of levels and the maximum value are tunable parameters. To retrieve the FBA- solved fluxes, we used COBRApy optimization on the biomass reaction without any constraints on the network reactions. In the end, each point (x, y) in the dataset consisted of a vectors of values for all the VER (X), and all the fluxes obtained running FBA or pFBA with COBRApy (Y).

Solving LP/QP with RNN methods

When intake fluxes are known (EB method), the linear optimization problem solved with FBA for a metabolic network with n fluxes and m metabolites can be written as: and its dual form:

In the case where intake fluxes are unknown (UB and general case for FBA) it can be written as: with V the vector of fluxes, M its dual referred to as shadow prices of metabolites, S the stoichiometric matrix, b a vector of dimension m with b_i corresponding to input flux of the ith metabolite and c the objective vector of dimension n. Keep in mind that b takes positive values only for metabolites transported by exchange reactions, consequently b_i = 0 for all internal metabolites. Using the notation of Fig. 1 we note that b = M2V V⁰, where V⁰ is the vector of intake fluxes. In this work c is a null vector with 1 for the ‘biomass reaction’ flux.

The usual method to solve this linear programming problem is the Simplex algorithm [32], which is fast and scalable. But, Simplex has no gradient, nor backpropagation compatibility. Thus, it couldn’t be integrated in AMNs, which are designed to be hybrid models.

Another way to solve this problem is through recurrent neural networks (RNN). As this method relies on gradient descent and is compatible with backpropagation. The two methods that were used are illustrated in Fig. 2A and 2B. The general solving happens by iteratively updating V the vector of all reaction fluxes and M the dual vector of shadow prices:

With dV and dM depend on the architecture (RNN-EB or RNN-UB).

For RNN-EB, the constraints are simplified to SV = b and the solution used is given in Figure 2A. For the UB method, the stoichiometric matrix S is split into S_int the part of S for internal fluxes and S_ext for exchange reaction fluxes. In addition, terms to ensure the positivity of the fluxes were added to S_ext (cf. Fig S2). The resulting equations are given in Fig. 2B. As the matrix S_int S_int^T should be invertible, we guarantee this property by giving its initial form the row echelon form. As a result, M in the UB method is of dimension p<m.

Both methods were proven to converge to the global optimum independently of the initialization. The time step was arbitrarily fixed to 3.0 10^-4 for EB and 5.0^-2 for UB and it was multiplied by 0.7 every 10⁴ iterations if the objective value was not decreasing.

For both Fig. 2C and Fig. 2D the results were obtained with 50 simulations and 10⁷ iterations. The reference values were obtained using COBRApy and E. coli core model. Each replicate was generated with a different medium computed with the training set generation method (cf. previous section).

In the first part of this work (i.e., Fig. 2), RNNs are used with the maximization of growth rate as an objective function. Whereas, in the second part (i.e., Figs. 3–4, Table 1) RNNs are embedded within larger architectures for training purposes, the objective function is no longer needed and c=0.

AMN architectures and parameters

We propose three AMNs architectures as described in Fig. 3A (i.e., composed of a dense layer and a mechanistic layer): RNN-EB, RNN-UB and RNN-W. RNN-EB and RNN-UB have a mechanistic layer as described in Fig. 2A and 2B. The architecture of RNN-W is shown in Fig. 3C. In the architecture ‘ANN dense’ and ‘ANN SV’ we omit the mechanistic layer and simply use the output of the first dense layer as the final output of the model. For all architecture, we use the Mean Squared Error (MSE) on all fluxes as the objective function to minimize while learning. In the architecture ‘ANN SV’ and in all RNN architectures we added to the MSE loss function, the term IISVII² to enforce the metabolic network constraint, i.e., the mass balance of internal metabolites.

To summarize the parameters used when training ANN and AMN can be decomposed into different categories:

1. Simulated data parameters. As described in the previous section (Generation of training sets with COBRApy), we can tune the size of the training set to be generated. We can also modify the mean number of activated VER per data point, the maximum value each VER can take, and the number of levels (i.e., the resolution) between 0 and a maximum. We can also modify the actual VER list, but this modifies the architecture of the model (initial layer size), so we kept the same list in the present work.

2. Learning parameters. During learning on simulated data, an AMN has a small set of parameters to tune: the loss function, the number of epochs, the batch size, and the X-fold validation, where X is the number of folds.

3. Architecture parameters. One can also select the architecture of the model, amongst the 5 benchmarked architectures: ‘RNN-W’, ‘RNN-EB’, ‘RNN-UB’, ‘ANN Dense’ and ‘ANN SV’.

Optimization of exchange reactions upper bounds in FBA

The goal of this optimization was to find the best VER fluxes to match experimentally determined growth rates, by using ‘out-of-the-box’ FBA, simply informing the presence or absence of the flux according to the experimental medium composition. As a result, the optimized fluxes values were assigned (as an upper bound) to the corresponding exchange reaction only when the experimental data point has the compound present. The Mean Squared Error (MSE) between all the FBA predictions and all the experimentally determined growth rates was the function to minimize. For that, the input vector to optimize was composed of the upper bounds for all VER, but also arabinose and glycerol (which are present in all media compositions). The Broyden, Fletcher, Goldfarb, and Shanno (BFGS) algorithm [33] was used under the SciPy library, as a local optimization method, starting from an initial vector filled of the value 5. Final optimized uptake values are shown in supplementary Table S1.

Culture conditions and growth rate determination

The basic medium for culturing E. coli DH5-α was a M9-glycerol medium prepared with those final concentrations: 100μM CaCl2, 2mM MgSO4, 1X M9 salts (3 g.L-1 KH2PO4, 8.5 g.L-1 Na2HPO4 2H2O, 0.5 g.L-1 NaCl, 1g.L-1 NH4Cl), 1X trace elements (15 mg.L-1 Na2EDTA 2H20, 4.5 mg.L-1 ZnSO4 7H2O, 0.3 mg.L-1 CoCl2 6H2O, 1 mg.L-1 MnCl2 4H2O, 1 mg.L-1 H3BO3, 0.4mg.L-1 Na2MoO4 2H20, 3 mg.L-1 FeSO4 7H2O, 0.3 mg.L-1 CuSO4 5H2O; solution adjusted at pH=4 and stored at 4°C), 1 mg.L-1 Thiamine-HCl and 0.5g.L-1 glycerol. The additional compounds that could be added (the VER, See Methods - Generation of training sets with COBRApy) were all set to a final concentration of 0.1 g.L-1. The pH was adjusted at 7.4 prior to a 0.22μm filter sterilization of the medium. Pre-cultures were recovered from glycerol −80°C stocks, grew in Luria-Bertani (LB) broth overnight, then used as inoculate in 200μL M9-glycerol (supplemented with variable compounds) in 96 U-bottom wells plates. The temperature was set to 37°C in a plate reader (BioTek HTX Synergy), with continuous orbital shaking, allowing aerobic growth for 24 hours. A monitoring every 10 minutes of the optical density at 600 nm was performed and the steepest part of the growth curve was retrieved for growth rate determination, by a simple logistic regression.

Author contributions and codes information

LF and JLF wrote the core of the text of the manuscript. JLF designed the study and wrote all the AMN codes used to produce Figs. 1, 3, 4, S1 and Tables 1 and S2. BM wrote the RNN codes producing Figs. 2 and S2 and wrote the corresponding part in the method section. LF wrote the code transforming SBML models into unidirectional networks and the code to produce Table S1. LF also performed all experimental work reported in Fig. 4 and wrote the corresponding experimental method section. WL contributed to designing the project and was involved in the discussions. All authors read, edited, and approved the manuscript. All AMN codes make use of COBRApy, numpy, Pandas, Tensorflow, Sklearn and Keras libraries. All codes are available within Google Colab notebooks and will be released upon journal publications.

Supplementary Materials

Fig. S1. Different weights for different intake fluxes.

In the two cases all flux values (v_i) satisfy the steady state constraints (SV = 0, cf. Fig. 1.B). Following the equations provided in Fig. 1.C, the production fluxes for m₁ and m₂ are respectively 1 and 0.5 in (A) and 0.75 and 0.25 in (B). The reaction for flux v₄ is taking two substrates m₁ and m₂ and the value for v₄ is the minimum metabolite production flux (i.e., the rate limiting metabolite among m₁ and m₂). Consequently, the value for v₄ is 0.5 (A) and 0.25 (B). Therefore, the fraction (w₄₁) of m₁ contributing to v₄ is 1/2 (A) and 1/3 (B). The weights are different in panel (A) and (B) as they depend on the intake flux values.

Fig. S2. Matrices used with the UB method.

We show here an example for the same model as in Fig. 1. The goal here is to produce two matrices from S, one where we remove the stoichiometric coefficients of the exchange reactions (here the only exchange reaction is TPI, the red column of the top-left matrix); the other where we only keep exchange reactions. Thus, the stoichiometric matrix S was split between reactions involving internal metabolites (S_int) and exchange reactions, importing or exporting external (medium) metabolites in or out of the cell (S_ext). In S_int external reactions (TPI) and involved metabolites (g3p) are zeroed out. An identity matrix is added to S_ext to ensure that V≥0 and S_ext is filled with negative value to ensure that intake fluxes are positive.

Table S1. Optimized intake fluxes for ‘out-of-the-box’ FBA.

‘VER ID’ indicates the column of variable exchange reactions IDs in the unidirectional iML1515 model. ‘Name’ indicates the actual human-readable name of the compound. The third line is the optimized upper bound value for this compound, in mmol/gDW/h.

Table S2. Intake fluxes predicted by the architecture of Fig. 4.

The Table provides in the two first columns the measured (TRUE) and COBRApy calculated growth rate (PRED) for 38 different media for which measured growth rate ≥ 0.05. The other columns provide intake fluxes for medium metabolites. The intake flux values have been calculated by the grey dense layer of Figure 4A. The intake flux values can differ quite a lot for one medium to another, as intake fluxes of metabolites are not independent from one another. Intake flux values are in mmol/gDW/h. Values are provided for minimum (m_i) and variable (v_i) medium metabolites. The compound name / ID are given below the Table. The Table below is also provided as a supplementary .csv file.

Acknowledgements

JLF would like to acknowledge funding provided by the ANR funding agency grant numbers ANR-18-CE44-0015 (SynBioDiag project) and ANR-21-CE45-0021-01 (AMN project). LF is supported by INRAE’s MICA department and the by INRAE’s metaprogram BIOLPREDICT. BM is supported by an Ecole Normale Supérieure Scholarship. JLF thanks Aymeric Gaudin (CentraleSupélec Engineering School) for early development in reservoir computing with AMN, and Ivan Radkevich (University of Paris Saclay Systems & Synthetic Biology master program student) for his work on custom RNN cells loading and saving.

Abbreviations

AMN

Artificial Metabolic Networks

ANN

Artificial Neural Network

CBM

Constraint-Based Modelling

(p)FBA

(parsimonious) Flux Balance Analysis

LP(QP)

Linear (Quadradic) Programming

MFA

Metabolic Flux Analysis

ML

Machine Learning

MM

Mechanistic Modelling

RNN

Recurrent Neural Network