Variable selection and forecasting via automated methods for linear models: LASSO/adaLASSO and Autometrics

In this paper we compare two approaches of model selection methods for linear regression models: classical approach - Autometrics (automatic general-to-specific selection) — and statistical learning - LASSO (l1-norm regularization) and adaLASSO (adaptive LASSO). In a simulation experiment, considering a simple setup with orthogonal candidate variables and independent data, we compare the performance of the methods concerning predictive power (out-of-sample forecast), selection of the correct model (variable selection) and parameter estimation. The case where the number of candidate variables exceeds the number of observation is considered as well. Finally, in an application using genomic data from a highthroughput experiment we compare the predictive power of the methods to predict epidermal thickness in psoriatic patients.