New Basic Hessian Approximations for Large-Scale Nonlinear Least-Squares Optimization

A simple modification technique is introduced to the limited memory BFGS (L-BFGS) method for solving large-scale nonlinear least-squares problems. The L-BFGS method computes a Hessian approximation of the objective function implicitly as the outcome of updating a basic matrix, \(H_k^0\) say, in terms of a number of pair vectors which are available from most recent iterations. Using the features of the nonlinear least-squares problem, we consider certain modifications of the pair vectors and propose some alternative choices for \(H_k^0\), instead of the usual multiple of the identity matrix. We also consider the possibility of using part of the Gauss-Newton Hessian which is available on each iteration but cannot be stored explicitly. Numerical results are described to show that the proposed modified L-BFGS methods perform substantially better than the standard L-BFGS method.