A fast algorithm for optimizing ridge parameters in a generalized ridge regression by minimizing a model selection criterion

Abstract In this paper, we deal with the optimization method of ridge parameters in a generalized ridge regression by minimizing a model selection criterion (MSC). The optimization methods based on minimizations of generalized C p criterion and GCV criterion are fast because the minimizers of the two criteria can be derived as closed forms. Although the methods are fast, they do not work well when the number of explanatory variables is larger than the sample size. Methods based on minimizations of other MSCs may work well even if the number of explanatory variables is larger than the sample size and are not fast because the minimizers of the MSCs have not been derived as closed forms. Even though a minimization problem of MSC cannot be solved analytically, we propose a fast optimization algorithm to minimize MSC by specifying the small number of minimizer candidates. By conducting numerical examinations, we compare performances of the optimization methods based on the minimizations of MSCs.