Progressive Response Surfaces

Response surface functions are often used as simple and inexpensive replacements for computationally expensive computer models that simulate the behavior of a complex system over some parameter space. “Progressive” response surfaces are ones that are built up progressively as global information is added from new sample points in the parameter space. As the response surfaces are globally upgraded based on new information, heuristic indications of the convergence of the response surface approximation to the exact (fitted) function can be inferred. Sampling points can be incrementally added in a structured fashion, or in an unstructured fashion. Whatever the approach, at least in early stages of sampling it is usually desirable to sample the entire parameter space uniformly. At later stages of sampling, depending on the nature of the quantity being resolved, it may be desirable to continue sampling uniformly over the entire parameter space (Progressive response surfaces), or to switch to a focusing/economizing strategy of preferentially sampling certain regions of the parameter space based on information gained in early stages of sampling (Adaptive response surfaces). Here we consider Progressive response surfaces where a balanced indication of global response over the parameter space is desired. We use a variant of Moving Least Squares to fit and interpolate structured and unstructured point sets over the parameter space. On a 2-D test problem we compare response surface accuracy for three incremental sampling methods: Progressive Lattice Sampling; Simple-Random Monte Carlo; and Halton Quasi-Monte-Carlo sequences. We are ultimately after a system for constructing efficiently upgradable response surface approximations with reliable error estimates. Introduction and Background Large-scale optimization and uncertainty analyses are often made feasible through the use of response surfaces as surrogates for computational models that may not be directly employable because of prohibitive expense and/or noise properties and/or coupling difficulties in multidisciplinary analysis. Examples of response surface usage to facilitate large-scale optimization and uncertainty analyses are cited in Roux et al. (1996), Unal et al. (1996), and Venter et al. (1996). Two issues that arise when using response surface approximations (RSA) are accuracy and the cost of procuring the data samples needed to create the RSA. With a sufficiently flexible global fitting/interpolating function over the parameter space, response surface accuracy generally increases as the number of data points increases (if the points are appropriately placed throughout the parameter space), until the essential character of the function is effectively mapped out. Thereafter, it is not cost effective to continue adding samples. Since a single high-fidelity physics simulation (i.e., one data sample) can take many hours to compute, it is desirable to minimize the number of simulations that are needed to construct an accurate response surface. For our purposes here it is assumed that: 1) the computer model is relatively expensive to evaluate; 2) the parameter space is a unit hypercube or can be accurately and inexpensively mapped into one; 3) the sampled or “target” function is a continuous, deterministic function over the parameter space; 4) reasonably general, arbitrary target functions are to *Sandia is a multiprogram laboratory operated by Sandia Corporation, a Lockheed Martin Company, for the United States Department of Energy under Contract DE-AC04-94AL85000. 1 Romero, Krishnamurthy, and Swiler be fitted; and 5) approximate response values are desired over the entire parameter space or subspace being considered –i.e., for global and local optimization or mapping inputs to outputs in uncertainty propagation. Given these specifications, Romero et al. (2000) examined several formulations for constructing progressive response surfaces built on Progressive Lattice Sampling (PLS) incremental sampling designs. PLS is a paradigm for structured uniform sampling of a hypercube parameter space by placing and incrementally adding sets of samples such that all samples are efficiently leveraged as the design progresses from one level to the next. Figures 1 4 show successive PLS levels in 2-dimensions. (Also shown for comparison are point sets from classical simple-random Monte Carlo sampling -using three different seeds for the random-number generator, and from Halton “quasiMonte Carlo” low-discrepancy sequences (see, e.g., Owen, 2003) where we have used two different sets of prime-number bases to create the two sets of Halton samples.) PLS endeavors to preserve uniformity of sampling coverage over the parameter space in the various stages or levels of the incremental experimental design. Uniform coverage over the parameter space is desirable for general response surface construction because this reduces the redundancy or marginalization of new information from added samples. This is a basic concept of upgradable quadrature methods (Patterson-1968, and Genz & Malik 1983). PLS builds knowledge by reducing global knowledge deficit over the parameter space. It does not attempt to build specific or targeted knowledge by building on previous information in the manner of “adaptive” sampling, which efficiently maximizes knowledge over particular regions of the parameter space. Thus, PLS designs select sample locations strictly on geometric principles such that new samples are intended to be “maximally far” from each other and from all other existing samples at each level of sampling. Thus, global uniformity of coverage is maintained at each level as the scheme progresses. The arrangement of samples in each PLS level allows the parameter space to be subdivided into a regular pattern of adjacent polygons, which for two parameter space dimensions results in triangular and quadrilateral 2-D finite elements (FEs) yielding linear to quadratic polynomial interpolation over each element (see Romero & Bankston, 1998). The collection of all the elements together creates a C0-continuous global function over the parameter space. As such, the global RSA has considerable freedom to locally conform to the data values of the sample points (see Figures 6 8). A mathematical analysis (Strang & Fix, 1973) of finite element interpolation of this nature shows that for a continuous and infinitely differentiable function over the parameter space, the domain integral of the pointwise absolute error goes to zero as the spacing between samples goes to zero. This analysis can be applied to the FE/PLS method. Hence, in the limit of infinite sampling, FE/PLS response surfaces converge everywhere in the domain to target functions in this class (though the convergence rate is generally not uniform over the domain). Therefore the FE/PLS method provides a convergent reference against which the accuracy of other progressive response surface methods can be compared. Upon upgrading from one level to the next of the PLS design, the resulting change in the response surface at any point in the domain is a heuristic indicator of the magnitude of the local error in the response surface approximation. When the incremental change goes to zero, this tentatively indicates that the local approximation error has become negligible. Romero, Krishnamurthy, and Swiler 2