Theoretical background.

We consider a DFBA model with a set of M input parameters that are denoted by x = (x₁, …, x_M). These parameters can appear in the initial conditions s₀, rate of change function f, cellular objective h, and/or the flux limits v^LB, v^UB. We look to develop a computationally cheap-to-evaluate representation of some output of the DFBA model referred to as the model response. The model response can be any chosen function of the states or fluxes that appear in (1) and (2) including, for example, metabolite concentrations, growth rate, or time-to-consumption of any metabolite. The model response function need not be known analytically, and can be approximated using a finite number of model evaluations. We focus on the scalar response case y for notational simplicity. However, the developed procedure can be easily applied separately to each component of a vector of responses y.

Unless the parameters x are perfectly known, they must be treated as uncertain. Parameter uncertainty can generally be represented by a random vector X with some known probability density function (PDF). In this case, the model response also becomes a random variable with some unknown PDF that is implicitly defined by (3) where ∼ denotes “distributed as” and f_X denotes the PDF of uncertain parameters. Determining the distribution f_Y (or its statistical moments) of the model response represents the forward UQ problem that can be tackled in various ways, the majority of which require extensive sampling that is not feasible whenever is a computationally expensive model. The polynomial chaos expansion (PCE) method addresses this problem by constructing a surrogate model that accurately approximates , but is significantly cheaper to evaluate. The PCE surrogate model can also be straightforwardly applied to other UQ tasks, as discussed later in the Results section. Provided that Y has finite variance, it can be represented with a PCE as follows [17] (4) where are coefficients of the expansion, are multivariate polynomials, α = (α₁, …, α_M) is a multi-index that identifies the degree of the multivariate polynomials in each of the input parameters X_i, and is the set of positive integers. The polynomial basis functions are required to be orthonormal with respect to the parameter distribution, such that they satisfy (5) where S is the support of the distribution of X and δ_αβ is the Kronecker delta that is 1 whenever α = β and 0 otherwise. For computational purposes, the series (4) must be truncated after a finite number of P terms, which yields the following approximation (6) where is a finite set of multi-indices with cardinality equal to P, is a vector of the coefficients, and is a vector containing all polynomial basis functions. The expansion coefficients are defined to be those that minimize the mean-square error (MSE) between the exact representation (4) and the truncated PCE (6) (7) The right-hand side of this expression represents the analytic solution to the MSE optimization problem and directly follows from the Hilbert projection theorem [32].

The expressions in (5) and (7) involve multivariate integration over complicated nonlinear functions. As such, the construction of the polynomial basis and computation of the expansion coefficients are usually carried out numerically in practice, which leads to additional sources of error. The choice of also plays an important role in PCE performance because directly controls the number of coefficients that must be estimated. Larger P values require more computational effort and are more susceptible to numerical sources of error. An overview of state-of-the-art methods for addressing these challenges is provided next.

Orthonormal basis construction.

The complexity of determining the polynomials depends fully on the structure of the PDF f_X. Whenever the uncertain parameters are statistically independent, then (5) reduces to the tensor product of M univariate polynomials that are orthonormal with respect to each marginal density . These polynomials have been analytically derived for many common PDFs [17], and can be found numerically for generic PDFs using algorithms in terms of the three-term recurrence relationship for orthogonal polynomials [33]. There are two main approaches for handling the more general case that X has statistically dependent (or correlated) elements. The first approach involves transforming the generic random vector X into a standard random vector Z for which it is simpler to build the polynomial basis functions [34]. Any isoprobabilistic transformation that preserves the PDFs of these random vectors can be utilized, though the most commonly used is the Rosenblatt transformation [35]. The second approach involves applying a more sophisticated numerical procedure that is able to impose the conditions in (5) simultaneously in M dimensions. This includes the Gram-Schmidt process [36] as well as the modified Cholesky decomposition of the Gram moment matrix [37, 38].

Sparse truncation and regression.

We denote the approximate PCE with numerically estimated coefficients as follows (8) A variety of methods have been proposed for estimating the coefficients that can be broadly categorized as intrusive (e.g., Galerkin projection [12]) or non-intrusive (e.g., pseudo-spectral projection [39] or regression [15]). Here, we focus exclusively on non-intrusive methods. The phrase “non-intrusive” implies that coefficient estimates are obtained over a finite set of parameter realizations , referred to as the experimental design (ED). These samples can be chosen in various ways including Monte Carlo sampling, quasi-random samples derived from Sobol or Halton sequences, or sparse grids to name a few [40]. The computational model is then evaluated at every point in the ED, i.e., with for all i = 1, …, N. As such, non-intrusive approaches are “black-box” in the sense that they can be applied to any function, even when this function is not explicitly known, and do not require any modification to the deterministic solver.

We will focus on regression methods due to their flexibility when it comes to enforcing sparsity. In the regression approach, coefficients are defined as those that minimize the least-square residual of the polynomial approximation over the ED (9) where is the model matrix that contains the values of all polynomial basis functions evaluated at all ED points. The solution of (9) requires a minimum number of sample points N ≥ P to ensure a unique solution exists. Since every sample requires an expensive DFBA simulation here, the truncation scheme plays a central role in reducing the complexity of surrogate model construction. The total degree method is the most commonly used approach for specifying , which looks to keep all polynomials up to a specified order p in the series. For total degree truncation, the set of multi-indices is defined as , where ‖α‖₁ = α₁ + ⋯ + α_M and . Due to the sharp increase in P as the polynomial order increases, the total degree truncation scheme can quickly lead to a prohibitive number of model evaluations, especially in high dimensions. This issue is often termed the curse-of-dimensionality, which is known to considerably limit standard PCE methods.

We look to take advantage of two approaches for overcoming the curse-of-dimensionality limitation. The first approach involves replacing the total order truncation with the so-called hyperbolic truncation scheme, which is defined as (10) where 0 < q ≤ 1. Lower values for q limit the number of high-order interaction terms considered, which directly lead to sparser solutions. The second approach looks to further sparsify the solution, without sacrificing potentially important interaction terms, by including a regularization term of the form with λ ≥ 0 in the least-squares problem (9). This regularization term is known to force the minimization to favor low-rank solutions and ensures the existence of a unique solution even when N < P.

The key challenge with regularization is a proper choice of λ, which indirectly specifies the number of non-zero coefficients included in the expansion. In this work, we use the hybrid least angle regression (LAR) method to solve the regularized version of (9). LAR is an efficient procedure for variable selection, which aims to select the predictors (i.e., polynomials Ψ_α) that have the greatest impact on the model response among a potentially large set of candidates [41]. Hybrid LAR is a variant of the original LAR that uses a modified cross-validation scheme to estimate the approximation error [19]. This modification relies on only a single call to the LAR procedure, which provides significant savings in computational cost when compared to the original method. The relative MSE (RMSE), which is defined as ε = MSE/Var{Y}, is the natural choice of the approximation error in PCE and can be robustly estimated by the leave-one-out (LOO) cross-validation error ε_LOO. Not only can ε_LOO be calculated analytically for PCE models [42], but it is known to be much less sensitive to overfitting than the empirical estimator [43].

Provided a sensible sampling strategy has been chosen, the remaining parameters that must be selected are related to truncation p and q and the ED size N. We use a systematic procedure for selecting these parameters to achieve a target error level ε_target. As discussed in [19], a basis-adaptive strategy can help overcome potential limitations of an a priori fixed truncation set by letting the maximum degree be driven directly from the data. The basic idea is to start with small values for p and q, estimate the coefficients using hybrid LAR, and calculate ε_LOO. These steps are repeated for incremented values of p and q, and the algorithm returns the PCE model with the lowest error. Early stop criteria can easily be introduced to avoid an excessive number of iterations. However, when dealing with computationally expensive models, the number of model evaluations N dominates the cost of construction of the surrogate model. We therefore propose an iterative “greedy” approach for constructing the ED to ensure that N can be kept as small as possible. This sequential ED strategy can be summarized as

1. Initialize the current ED with a relatively small number of samples N_init.
2. Train a sparse basis-adaptive PCE using the current ED and calculate ε_LOO.
3. If ε_LOO < ε_target, stop the algorithm and return current PCE. Otherwise, enrich the current ED with N_add more samples and return to Step 2.

Note that any method can be used in the training step of this algorithm. Thus, in the proposed nsPCE method, the desired accuracy level is the key parameter that must chosen by the user.

Decomposition of parameter space.

Before selecting the element decomposition, we must first simulate the DFBA model to locate any significant singularities. The extracellular glucose, xylose, and biomass concentration profiles are plotted in Fig 2 for one hundred randomly sampled parameter values. For a given realization of the parameter, the full simulation requires approximately 1.5 seconds of CPU time.

At the start of the batch, glucose is consumed preferentially over xylose. Once glucose has been depleted, the LP solution switches and xylose becomes the main carbon source. The final batch time is then specified as the time that both glucose and xylose have been fully depleted, at which point the LP becomes infeasible and the solution ceases to exist. The E. coli cells stop growing at this point due to the lack of a carbon source. Although physically the cells would begin to die in this situation, DFBA models cannot directly predict the cell death phase and thus we assume the biomass remains constant for simplicity. The time-to-consumption of glucose t_g and xylose t_z represent the two singularities in this problem, and clearly depend on the value of the model parameters. Since the singularity time functions cannot be derived analytically, we look to construct PCE approximations for both t_g and t_z. We investigate two different fitting methods: classical full PCE with coefficients estimated using ordinary least squares (OLS) and sparse basis-adaptive PCE with coefficients estimated using hybrid LAR. The degree of the polynomials is varied from 1 to 6 in the full PCE method, where N = 2P model evaluations are used for regression with P denoting the size of the basis. In the sparse PCE method, the maximum degree is allowed to vary from 1 to 20, and a hyperbolic truncation scheme (10) is used with q = 0.75. The experimental designs (EDs) are generated using Monte Carlo (MC) sampling with a fixed random seed to ensure repeatable results. Fig 3a and 3b show the RMSE as a function of the number of model evaluations used to fit surrogate models for t_g and t_z, respectively. The sparse PCE method consistently outperforms full PCE, achieving approximately an order-of-magnitude lower RMSE for all ED sizes.

The sparse PCE surrogate models for t_g and t_z are used in the nsPCE method to build surrogates for the extracellular concentrations. Additionally, these surrogate models contain useful information on which parameters influence the consumption of different substrates. The Sobol’ indices of t_g(X) and t_z(X) are shown in Fig 4, which are a commonly used tool in global sensitivity analysis for ranking the parameters according to their contribution to the variance of the model response. The Sobol’ indices can be computed analytically from the PCE coefficients [43], which requires less than one second of CPU time here. It is interesting to note that u_g,max and u_o mainly contribute to the variance of t_g(X), while u_g,max, u_z,max, and u_o are the significant contributors to the variance of t_z(X).

The surrogate models can also be used to estimate the PDF of t_g(X) and t_z(X), as shown in Fig 4. From the estimated PDFs, we find that t_g(X) ranges from approximately 6.31 to 7.87 hr, whereas t_z(X) ranges from approximately 7.33 to 9.12 hr. This suggests that the model response is a non-smooth function of X ∈ S for any t ∈ [6.31, 9.12] hr, so that we must split S into two disjoint regions according to 15. Since the supports of t_g(X) and t_z(X) partially overlap for any t ∈ [7.33, 7.87] hr, additional elements should be introduced to ensure the model response is smooth. However, for times outside of this window, we can exclusively define the elements of the parameter space in terms of t_g for times before 7.33 hr and t_z for times after 7.87 hr. Plots of these two regions at times 6.5, 7.0, and 7.25 hr projected onto the two most sensitive parameters are shown in Fig 5. The blue region represents S₁ while the red region represents S₂. For comparison purposes, we also show the decision boundary in green (along with 95% confidence limits with dashed green lines) learned from a support vector machine (SVM) binary classifier [46] that was trained using the same 500 data points. The SVM model is unable to capture the significant nonlinear behavior of the boundary as it evolves over time. Thus, SVM results in relatively large misclassification errors due to the limited training data. The sparse PCE model, however, is able to accurately represent the t_g function over the full support (see the parity plot in Fig 5d), which leads to a much more accurate representation of these two elements using limited data.

The “true” RMSE values reported in Fig 3 were estimated using a large validation set that consisted of 10,000 evaluations of the full DFBA model, which required over 3 hours of CPU time. Ideally, these additional model evaluations could be avoided by directly estimating the RMSE from the ED either empirically or using cross-validation techniques. The empirical estimate of the RMSE is based on sample-based approximations to the integral expressions for mean and variance. Cross-validation obtains a more robust RMSE estimate by splitting the ED into various training and validation sets, fitting different models with each training set, and averaging the prediction error of each model. We focus exclusively on ε_LOO in this work. Table 1 gives the estimated RMSE values for the sparse PCE surrogate models fit using different ED sizes. We observe that the empirical estimator greatly underpredicts the “true” RMSE found from the large validation set. In fact, for the smallest size N = 10, the empirical estimate is a factor of 10⁴ smaller than the true RMSE. The cross-validated RMSE, on the other hand, predicts the correct order in all considered cases except N = 10 where it is off merely by a factor of 10 instead of 10⁴. Note that ε_LOO is used within the hybrid LAR algorithm to select the best surrogate out of all potential candidates.

Validation of nsPCE surrogate models.

We have verified that the PCE surrogate models are able to accurately represent the singularity manifold that leads to non-smooth behavior in the states of the DFBA model. Thus, they can be used to build nsPCE surrogates for the extracellular concentrations based on the algorithm summarized in Fig 1. We choose three quantities of interest for illustrative purposes: glucose at time 7.0 hr, xylose at time 8.0 hr, and biomass at time 8.0 hr. We look to compare so-called global PCE to the proposed nsPCE method for these three quantities of interest. In global PCE, a single surrogate model is constructed over the full parameter support, while nsPCE systematically breaks down the support into two disjoint elements using the singularity time function as a dividing boundary. To ensure a fair comparison, the expansion coefficients of both global PCE and nsPCE are estimated using the basis-adaptive hybrid LAR strategy with maximum degree varying from 1 to 20 and q = 0.75 in the hyperbolic truncation scheme (10). In addition, the ED in both approaches are sequentially enriched using MC sampling with a fixed random seed. To simplify the construction of the polynomial basis functions when training the nsPCE surrogate models, the elements S₁ and S₂ were outerbounded with hyper-rectangles. However, only parameter values that explicitly fall within these sets are incorporated into the local ED. This simple approach for dealing with elements of any shape is currently used in the provided scripts [29], but other ways of dealing with generic elements can also be explored.

The convergence properties of the nsPCE surrogate models for the three quantities of interest are compared to that of global PCE in Fig 6. The nsPCE surrogates achieve significantly lower RMSE than the global PCE surrogates in virtually all cases considered, while requiring many fewer samples to converge to the target error level. In addition, global PCE saturates at the maximum number of ED samples for all three quantities of interest. This implies that global PCE is unable to achieve the desired accuracy levels, whereas nsPCE only saturates for the lowest target error of xylose. This behavior is expected since the convergence rate of global PCE is known to be substantially lowered whenever singularities are present in the model response function. Thus, nsPCE is able to significantly improve the rate of convergence based on a properly chosen elemental decomposition of the parameter support. To show that lower target error levels translate to improved predictions, parity plots for the three quantities of interest are shown in Fig 7. Note that global PCE has large prediction errors for particular values of the parameters (see the blue dots that largely deviate from the y = x line), which is likely due to the fact that an inherently non-smooth function is being represented by smooth polynomials. This is highly undesirable when using the PCE to predict specific response values, as opposed to predicting statistical quantities that average over the response values where individual points are not as important. The nsPCE surrogate models clearly mitigate this limitation of global PCE in a significant way since there are no outlier predictions in the set of 10,000 validation points.

Bayesian parameter inference.

Here, we focus on the inverse UQ problem of estimating parameters from data, which can be greatly accelerated using nsPCE. The same data set used in [8] is utilized, which includes measurements of the extracellular biomass, glucose, and xylose concentrations at t ∈ {5.5, 6.0, 6.5, 7.0, 7.25, 8.0, 8.25, 8.5} hr. The measurements are corrupted with noise (21) where and are, respectively, vectors of the measured data and noise at the ith time point. The concatenated data (respectively noise) vector is denoted by D = (D₁, …, D₈) (respectively E = (E₁, …, E₈)). The measurement noise variables are modeled as independent zero-mean Gaussian random variables with state-dependent variance that equals 5% of the measured signal, i.e., (22) Given a set of measurements, the change in the state of information about the parameters is given by Bayes’ rule [11] (23) where f_X|D is the posterior density; f_D|X is the likelihood function; f_X is the prior density; and f_D is the evidence. As Bayesian inference looks to characterize the full posterior density, it directly provides an explicit representation of the uncertainty in the parameter estimates.

The prior and likelihood function must be specified before solving (23). We assume the same uniform priors as those used to construct the nsPCE surrogate models, though these can differ in general. The likelihood function describes the discrepancy between the observed data and the model predictions in a probabilistic way. The likelihood function is specified by the data and noise models in (21) and (22), and is given by (24) Although we use a Gaussian likelihood here, the same Bayesian estimation approach can be applied to any choice of likelihood function and thus can be easily modified to incorporate other potentially important factors including sensor bias or asymmetric noise.

Since (23) cannot be solved analytically, we must resort to sample-based approximations that rely on generating samples from the target posterior distribution. A variety of methods have been developed for sampling from the unknown posterior f_X|D, including Markov Chain Monte Carlo (MCMC) [47–49] and sequential Monte Carlo (SMC) [50–52] algorithms. The proposed surrogate models can be used to accelerate any sampling-based method; however, we focus on SMC since this is a class of algorithms that can be made fully parallelized. SMC is based on the concept of importance sampling, which can be implemented in an iterative fashion such that the posterior is updated every time a new measurement becomes available. For a given number of particles N_p, the SMC approximation to (23) can be summarized as follows:

1. Initialization: set k = 1 and generate samples and weights from prior.
2. Reweighting: update the weights w_i ← w_i × w_k(x_i) where w_k(x_i) ∝ f_Dk|X(d_k|x_i).
3. Resampling: resample for particles with equal weights .
4. Loop: set k ← k + 1 and . Return to Step 2 if k < k_f.

When the algorithm stops at time k_f, the set of N_p particles targets the posterior distribution of interest. We use systematic resampling in Step 3 due to its computational simplicity and good empirical performance, though a variety of other methods are available [50]. Step 2 is usually the computational bottleneck because the model must be repeatedly solved in order to evaluate the likelihood weight factors using (24). Therefore, we propose to replace the evaluation of v(t_i; x) with a nsPCE surrogate model v^nsPCE(t_i; x) for every v ∈ {b, g, z} and i = 1, …, 8. We must then construct a total of 24 surrogates before running the SMC algorithm.

The same basic strategy described in the previous section is used for constructing all 24 of the nsPCE surrogate models. Similarly to how the samples for the singularity time are used to initialize the ED in each element, we can store the list of state and time points generated when integrating the DFBA model and interpolate these points to calculate the extracellular concentrations at every time point of interest. By keeping a working ED that is used to initialize each element at every time point, we can greatly limit the number of expensive DFBA simulations that represent the computational bottleneck in SMC. The proposed algorithm in Fig 1 is run with a target error of ε_target = 10⁻³, 250 initial ED samples, 10 ED samples added at each iteration, 2500 maximum ED samples, maximum degree varying from 1 to 20, and hyperbolic truncation with q = 0.75. The algorithm converged with cross-validated errors ε_LOO below the desired tolerance using only a total of 1200 DFBA simulations to train all 24 nsPCE surrogate models. The basis-adaptive hybrid LAR method consistently estimated coefficients in less than 30 seconds, verifying that the DFBA simulations are the dominant cost in this case study. The validation RMSE values are summarized in S2 Table, which are all below the target error threshold.

Fig 8 shows the posterior density estimated using SMC with N_p = 1 × 10⁶ particles for a synthetic data set, where the likelihood weights are evaluated using the inexpensive nsPCE surrogate models. The synthetic data (‘x’ marks in S1 Fig) was obtained by simulating the genome-scale E. coli DFBA model with fixed parameters (red lines in Fig 8) and adding random noise realizations (22) to the resulting model outputs. The 1200 DFBA simulations used to construct the surrogates require ∼ 30 minutes of CPU time while the surrogate-based SMC algorithm, which takes advantage of vectorization, finishes in ∼ 2 minutes of CPU time. Hence, over 800-fold savings in computational cost is achieved when compared to SMC without surrogates that would require approximately 17 days of CPU time under the same settings (1 × 10⁶ DFBA simulations at a cost of 1.5 seconds per evaluation). The DFBA model predictions under the MAP estimates (i.e., parameters that maximize the posterior) are shown in S1 Fig, which closely match the observed data. To verify that the SMC algorithm approximately converged with this many particles, we performed 10 separate bootstrap runs that produced a set of very similar posterior densities. Note that a discussion on challenges and open issues in Bayesian estimation is provided in the Discussion section. The SMC code is provided in the main_smc.m script in [29].

The estimated posterior density in Fig 8 provides interesting physical insights. Three of the parameters (K_g, K_z, K_ig) are unobservable with the current data set since their posterior (blue) and prior (green) densities are equivalent. This observation could not be easily made before running the estimation procedure due to the nonlinear and indirect relationship between D and X. A change in the experimental conditions such as the initial conditions, controlled oxygen concentration, or substrate feed profiles can enhance the sensitivity of the data to parameters (K_g, K_z, K_ig). For example, running the batch at low glucose concentrations g(t) ≪ K_g results in a glucose uptake rate that is a strong function of K_g, whereas running the batch at high glucose concentrations (as done in this case study) produces a nearly constant uptake rate u_g(t) ≈ u_g,max that is independent of K_g. Although the data is sensitive to (u_g,max, u_z,max, u_o), these parameters are highly correlated as seen in the off-diagonal plots of their joint densities in Fig 8. Thus, the currently available data from one single batch is insufficient for accurately estimating all the parameters of interest. The evolution of the marginal posterior densities of the observable parameters over time is shown in Fig 9. Since glucose is mostly consumed by 7.25 hr, the densities of u_g,max and u_o remain constant for the remaining batch time. The density of u_z,max, however, is constant before 7.25 hr because xylose remains mostly at its initial condition.

Forward uncertainty propagation.

Let denote the vector of all model responses. The forward UQ problem looks to characterize the uncertainty in the model predictions by propagating uncertainty in the parameters through . This can involve estimating either the prior predictive distribution f_Y (before any data has been collected), or the posterior predictive distribution f_Y|D (after data has been obtained). The only difference between these two problems is that is evaluated at i.i.d. samples drawn from the prior in the former and the posterior in the latter. The densities of the model predictions estimated using 1 × 10⁶ samples are shown in Fig 10. By replacing the full DFBA model with the nsPCE surrogate model, these histograms were obtained in less than 1 minute of CPU time. As expected, the prior predictive distributions are much wider than the posterior predictive distributions, indicating there is significant uncertainty in the predictions before incorporating data. In addition, we see that many of these distributions have sharp changes and long tails due to the non-smooth behavior of the model responses, which can be accurately captured with the proposed nsPCE framework. It is also interesting to note that the posterior predictive distributions have low variance, even though the parameters are not perfectly estimated. This highlights the impact that nonlinearity can have on both estimation and uncertainty propagation.

Global sensitivity analysis.

To locate any possible singularities, we first simulate the DFBA model with randomly sampled parameter values. The results are shown in Fig 11 wherein we see that the penalty state becomes positive α(t) > 0 once all substrates are depleted, which introduces a strong discontinuity into the state profiles. Even though this is a considerably smaller metabolic network than the one considered in the E. coli case study, it still takes approximately 0.5 seconds of CPU time per realization of the parameter. Thus, it is still advantageous to construct a surrogate model to speedup both forward and inverse UQ problems.

We look to run the proposed nsPCE method (see Fig 1) using the time that the penalty state switches from zero to positive as the singularity time. As suggested in [31, Chapter 8], we consider the seven substrate and product concentrations (X, C, N, O, L, E, COX) at four time points t ∈ {10, 20, 30, 40} hr as our main quantities of interest. The nsPCE method was applied in the same manner as described in the previous case study. Here, we specified a target error of ε_target = 10⁻³, 100 initial ED samples, 100 ED samples added at each iteration, 1500 maximum ED samples, the maximum polynomial degree could vary from 1 to 30, and a hyperbolic truncation scheme with q = 0.6. The algorithm converged using a total of 2800 DFBA simulations. The resulting parity plots are shown in S2 Fig, which all have empirical RMSE values significantly below the target error. To further assess the accuracy of these models, we calculated the RMSE using 1000 additional samples that were not used during the training process. The validation RMSE averaged over the 28 models was found to be 8.1 × 10⁻⁴. Only 6 of the 28 surrogates had RMSE values slightly above the target, with the largest overall RMSE being 3.5 × 10⁻³, indicating that the surrogates are reasonably accurate representations of the original model. Note that these errors can be refined by specifying a lower ε_target at the cost of more DFBA simulations.

Once the nsPCE surrogate models are constructed, they can be used to efficiently perform global sensitivity analysis in order to quantify the respective effects of each individual parameter on the variance of the model response. Although many sensitivity measures exist, we use Sobol’ indices since they make no assumption on the underlying linearity or monotonicity of the model. The global sensitivity results for the various quantities of interest over time are shown in Fig 12. A variety of interesting conclusions can be drawn from these results. For example, the model appears to be insensitive to K_C and K_iE, which is likely due to the fact that the batch was run at high carbon and low ethanol concentrations. In addition, the measurements of carbon, nitrogen, and oxygen are highly sensitive to their respective initial conditions C₀, N₀, and O₀ at the first measurement time of 10 hr; however, this sensitivity drops considerably as time evolves. We emphasize that obtaining such insights using random sampling on the full model can be prohibitively expensive, but requires negligible cost using the nsPCE surrogate models.

Optimization-based parameter estimation.

We now utilize maximum a posteriori (MAP) estimation to estimate the unknown model parameters from synthetically-generated experimental data (see S5 Table). The MAP estimate is defined as the mode of the posterior distribution, and can be stated directly as an optimization problem of the form [53] (28) where the prior acts as a regularization term that can stabilize the solution whenever the parameters cannot be uniquely inferred from the available data [54]. We consider a Gaussian likelihood, with noise standard deviations reported in S5 Table, and a Gaussian prior whose mean is equal to the midpoint of the bounds in S4 Table and standard deviations equal to 10% of the mean values. Under the Gaussian restrictions, we can convert the MAP problem to the minimization of a regularized weighted least squares objective by applying a negative log transformation. We solved the optimization (28) using both the full DFBA and nsPCE surrogate models in order to assess the computational gains afforded by the nsPCE method. To ensure a fair comparison, we solved both of these MAP problems in Matlab using the non-smooth optimizer SolvOpt [55] with default parameter settings and the mean of the prior as the initial guess. The algorithm took approximately 2.5 hr to converge when using the full DFBA model, which was substantially reduced to less than 2 minutes (i.e., a factor of 60) when the full model was replaced with the nsPCE surrogates.

Not only did the use of the nsPCE surrogate models accelerate the optimization, it also produced a solution with a lower overall objective function value. The objective improved from 471.73 to 1.56 when using the surrogate models as compared to 63.57 when using the full DFBA model. Convergence to a suboptimal solution is likely a consequence of numerical issues related to the stability of derivative approximation using finite difference in DFBA models, which were also observed in [31, Chapter 8]. On the other hand, since the nsPCE surrogate models are defined in terms of simple polynomial functions, the finite difference derivative approximation seems to produce more stable iterations towards the solution of the MAP problem, at least in this particular case study. The predictions of the DFBA model under the MAP estimates found using the nsPCE surrogates are shown in Fig 13. We see that the predictions using the posterior parameter estimates very closely match the observed data, which is a large improvement when compared to the predictions based on the prior parameter estimates.

Posterior distribution analysis.

The MAP estimation (28) determines the parameters that maximize the posterior density. However, we need a characterization of the entire posterior to assess uncertainty in these estimates. Here, we use a Laplace approximation of the posterior density, which is based on a second-order Taylor series of −log(f_X|D(x|d)) around the MAP estimate [56]. As shown in [57], this leads to a Gaussian approximation of the posterior whose mean is equal to the MAP estimate and whose covariance is defined in terms of the model response sensitives. The approximated posterior marginal densities and 95% confidence regions for the twenty MAP parameter estimates are shown in Fig 14. As can be seen, the true (unknown) parameter values are contained within the reported confidence regions. We also see that the parameters with the highest global sensitivity indices (see Fig 12) are accurately estimated, whereas the parameters that have little-to-no sensitivity to the data have much wider variances that are similar to that of the prior. Lastly, we observe that physically-related parameters exhibit a significant degree of correlation including, for example, nitrogen uptake parameters v_max,N and K_N. It is worth noting that the surrogate models can also enable the use of more advanced methods for posterior characterization such as randomized MAP [58], which would require the repeated solution of (28) with randomly perturbed data.