A global Monte-Carlo method for fitting parameters of differential equation models
0
Citation
11
Reference
20
Related Paper
Abstract:
Finding the parameter values of differential equation models from data is an important part of the modelling process. Large models and sparse data often make the parameters very difficult to find. Over the years there have been a number of methods proposed to solve this problem, some with good results but there is still plenty of room for improvement. In this work, we combine ideas from sequential Monte Carlo methods and genetic algorithms to create a new method to fit model parameters. One strength of our method is that it can perform well even when the order of magnitude of the parameters is unknown. We test our method in different models with real and simulated data and it is able to retrieve good parameter values.Keywords:
Experimental data
Cite
Computer models usually have a variety of parameters that can (and need to) be tuned so that the model better reflects reality. This problem is called calibration and is an inverse problem. We assume that we have a set of observed responses to given inputs in a physical system and a computer model that depends on parameters that models the physical system being studied. It is often the case that many more simulations can be run than experiments conducted, so we typically have many more simulation results (at various parameter values) than experimental results (at the “true” parameter value). In this paper, we use Maximum Likelihood Estimation (MLE) to calibrate model parameters. We assume that the response data is vector-valued, e.g. a response is given as a function of time. We approximate the underlying models with Gaussian Processes (GPs) and fit the parameters of the GPs with MLE. Specifically, we propose a decomposition approach to identify the basis vectors that allows for efficient calculation of the parameters. Experimental data is then used to calibrate the model parameters. This approach is demonstrated on one test problem.
Cite
Citations (2)
Estimation of parameter sensitivities for stochastic chemical reaction networks is an important and challenging problem. Sensitivity values are important in the analysis, modeling and design of chemical networks. They help in understanding the robustness properties of the system and also in identifying the key reactions for a given outcome. In a discrete setting, most of the methods that exist in the literature for the estimation of parameter sensitivities rely on Monte Carlo simulations along with finite difference computations. However these methods introduce a bias in the sensitivity estimate and in most cases the size or direction of the bias remains unknown, potentially damaging the accuracy of the analysis. In this paper, we use the random time change representation of Kurtz to derive an exact formula for parameter sensitivity. This formula allows us to construct an unbiased estimator for parameter sensitivity, which can be efficiently evaluated using a suitably devised Monte Carlo scheme. The existing literature contains only one method to produce such an unbiased estimator. This method was proposed by Plyasunov and Arkin and it is based on the Girsanov measure transformation. By taking a couple of examples we compare our method to this existing method. Our results indicate that our method can be much faster than the existing method while computing sensitivity with respect to a reaction rate constant which is small in magnitude. This rate constant could correspond to a reaction which is slow in the reference time-scale of the system. Since many biological systems have such slow reactions, our method can be a useful tool for sensitivity analysis.
Robustness
Cite
Citations (0)
Optimisation of distribution parameters is a very common problem.There are many sorts of distributions which can be used to model environment processes, biological functions or graphical data.However, it is common that parameters of those distribution may be, partially or completely unknown.Mixture models composed of a few distributions are easier to solve.In such a case simple estimation methods may be used to obtain results.Usually models are composed of several distributions.Those distributions may be of the same or different type.Such models are called mixture models.Finding their parameters may be complicated.Usually in such cases iterative methods need to be used.The paper gives a brief survey of algorithms designed for solving mixtures of distributions and problems connected with their usage.One of the most common method used to obtain mixture model parameters is Expectation-Maximization (EM) algorithm.EM is the iterative algorithm performing maximum likelihood estimation.The authors present the results of adjusting the Gaussian mixture models to the data.It is done with the usage of EM algorithm.The article gives advantages and disadvantages of EM algorithm.Improvements of EM applied in the case of large data are also presented.They help increase efficiency and decrease operation time of the algorithm.Another considered issue is the problem of optimal input parameters selection and its influence on the adjustment results.The authors also present algorithm performance observations.
Cite
Citations (1)
The use of reduced order models to describe a dynamical system is pervasive in science and engineering. Often these models are used without an estimate of their error or range of validity. In this paper we consider dynamical systems and reduced models built using proper orthogonal decomposition. We show how to compute estimates and bounds for these errors, by a combination of the small sample statistical condition estimation method and of error estimation using the adjoint method. More importantly, the proposed approach allows the assessment of so-called regions of validity for reduced models, i.e., ranges of perturbations in the original system over which the reduced model is still appropriate. This question is particularly important for applications in which reduced models are used not just to approximate the solution to the system that provided the data used in constructing the reduced model, but rather to approximate the solution of systems perturbed from the original one. Numerical examples validate our approach: the error norm estimates approximate well the forward error while the derived bounds are within an order of magnitude.
Cite
Citations (9)
The optimal selection of experimental conditions is
essential in maximizing the value of data for inference and
prediction, particularly in situations where experiments are
time-consuming and expensive to conduct. A general Bayesian
framework for optimal experimental design with nonlinear
simulation-based models is proposed. The formulation accounts for
uncertainty in model parameters, observables, and experimental
conditions. Straightforward Monte Carlo evaluation of the objective
function - which reflects expected information gain
(Kullback-Leibler divergence) from prior to posterior - is
intractable when the likelihood is computationally intensive.
Instead, polynomial chaos expansions are introduced to capture the
dependence of observables on model parameters and on design
conditions. Under suitable regularity conditions, these expansions
converge exponentially fast. Since both the parameter space and the
design space can be high-dimensional, dimension-adaptive sparse
quadrature is used to construct the polynomial expansions.
Stochastic optimization methods will be used in the future to
maximize the expected utility. While this approach is broadly
applicable, it is demonstrated on a chemical kinetic system with
strong nonlinearities. In particular, the Arrhenius rate parameters
in a combustion reaction mechanism are estimated from observations
of autoignition. Results show multiple order-of-magnitude speedups
in both experimental design and parameter
inference.
Uncertainty Quantification
Divergence (linguistics)
Polynomial Chaos
Autoignition temperature
Cite
Citations (14)
Author(s): Shirman, Aleksandra | Advisor(s): Abarbanel, Henry D. I. | Abstract: Data Assimilation (DA) is a method through which information is extracted from measured quantities and with the help of a mathematical model is transferred through a probability distribution to unknown or unmeasured states and parameters characterizing the system of study. With an estimate of the model paramters, quantitative predictions may be made and compared to subsequent data. Many recent DA efforts rely on an probability distribution optimization that locates the most probable state and parameter values given a set of data. The procedure developed and demonstrated here extends the optimization by appending a biased random walk around the states and parameters of high probability to generate an estimate of the structure in state space of the probability density function (PDF). The estimate of the structure of the PDF will facilitate more accurate estimates of expectation values of means, standard deviations and higher moments of states and parameters that characterize the behavior of the system of study. The ability to calculate these expectation values will allow for an error bar or tolerance interval to be attached to each estimated state or parameter, in turn giving significance to any results generated. The estimation method’s merits will be demonstrated on a simulated well known chaotic system, the Lorenz 96 system, and on a toy model of a neuron. In both situations the model system provides unique challenges for estimation: In chaotic systems any small error in estimation generates extremely large prediction errors while in neurons only one of the (at minimum) four dynamical variables can be measured leading to a small amount of data with which to work. This thesis will conclude with an exploration of the equivalence of machine learning and the formulation of statistical DA. The application of previous DA methods are demonstrated on the classic machine learning problem: the characterization of handwritten images from the MNIST data set. The results of this work are used to validate common assumptions in machine learning work such as the dependence of the quality of results on the amount of data presented and the size of the network used. Finally DA is proposed as a method through which to discern an `ideal' network size for a set of given data which optimizes predictive capabilities while minimizing computational costs.
Cite
Citations (5)
Abstract Iterative ensemble smoothers have been widely used for calibrating simulators of various physical systems due to the relatively low computational cost and the parallel nature of the algorithm. However, iterative ensemble smoothers have been designed for perfect models under the main assumption that the specified physical models and subsequent discretized mathematical models have the capability to model the reality accurately. While significant efforts are usually made to ensure the accuracy of the mathematical model, it is widely known that the physical models are only an approximation of reality. These approximations commonly introduce some type of model error which is generally unknown and when the models are calibrated, the effects of the model errors could be smeared by adjusting the model parameters to match historical observations. This results in a bias estimated parameters and as a consequence might result in predictions with questionable quality. In this paper, we formulate a flexible iterative ensemble smoother, which can be used to calibrate imperfect models where model errors cannot be neglected. We base our method on the ensemble smoother with multiple data assimilation (ES-MDA) as it is one of the most widely used iterative ensemble smoothing techniques. In the proposed algorithm, the residual (data mismatch) is split into two parts. One part is used to derive the parameter update and the second part is used to represent the model error. The proposed method is quite general and relaxes many of the assumptions commonly introduced in the literature. We observe that the proposed algorithm has the capability to reduce the effect of model bias by capturing the unknown model errors, thus improving the quality of the estimated parameters and prediction capacity of imperfect physical models.
Smoothing
Ensemble forecasting
Iterative refinement
Physical system
Cite
Citations (9)
Nonlinear mixed effects models are mixed effects models in which some of the fixed and random effects parameters enter nonlinearly to the model function. Nonlinear models are parsimonious so that we can capture the nonlinear variation with a minimum number of parameters. Due to their great importance, fitting of these models are also of crucial matters. A number of methods for fitting nonlinear mixed effects model are available in literature, most of the methods require approximating wither the model function or the likelihood function. A new method is proposed which numerically evaluate the integrations involved in the likelihood function with Monte Carlo integration using Sobol's sequence. The methods are compared using simulation studies and the method based on Laplace approximation is found to fit the nonlinear mixed effects model the best. The proposed Sobol's sequence based method performs better than some of the existing methods, especially in some cases; it produces good result in estimating random effects parameter. Thus, the Sobol's sequence based proposed method is very much compatible with the existing ones as well as the approximation based methods are quite handy.
Sobol sequence
Sequence (biology)
Mixed model
Laplace's method
Cite
Citations (0)
This paper addresses parameter estimation of spatial regression models incorporating spatial lag. These models are very important in spatial econometrics, where spatial interaction and structure are introduced into regression analysis. Because of spatial interactions, observations are not truly independent, and traditional regression techniques fail. Estimation techniques include maximum likelihood estimation, ordinary least squares, and the method of moments. However, parameters of spatial lag models are difficult to estimate due to the simultaneity bias (Ord, 1975). These estimation problems are generally intractable by standard numerical methods, and, consequently, robust and efficient optimization techniques are needed. In the case of simple general spatial regressive models (GSRMs), standard local optimization methods, such as Newton-Raphson iteration (as suggested by Ord) converge to high-quality solutions. Unfortunately, a good initial guess of the parameters is required for these local methods to succeed. In more complex autoregressive spatial models, an analytic expression for good initial guesses is not available, and, consequently, local methods generally fail. In this paper, global optimization (specifically, particle swarm optimization, or PSO) is used to estimate parameters of spatial autoregressive models. PSO is an iterative, stochastic population-based technique that is increasingly used in a variety of fields to solve complex continuous- and discrete-valued problems. In contrast to genetic algorithms and evolutionary strategies, PSO exploits cooperative and social behavior among members of a population of agents, or particles, which represent a point in the search space. This paper first motivates the need for global methods by demonstrating that GSRM parameters can be estimated with PSO even without a good initial guess, while the local Newton-Raphson and Nelder-Mead approaches have a greater failure rate. Next, PSO was tested with an autoregressive spatial model, for which no analytic initial guess can be computed, and for which no analytic parameter estimation method is known. Simulated data were generated to provide ground truth values to assess the viability of PSO. The global PSO method was found to successfully estimate the parameters using two different MLE approximation techniques for trials with 10, 20, and 40 samples (R2 > 0.867 for all trials). These results indicate that global optimization is a viable approach to estimating the parameters of spatial autoregressive models, and suggest that future directions should focus on more advanced global techniques, such as branch-and-bound, dividing rectangles, and differential evolution, which may further improve parameter estimation in spatial econometrics applications.
Ordinary least squares
Cite
Citations (2)
Operator (biology)
Continuous variable
Component (thermodynamics)
Surrogate model
Cite
Citations (44)