Efficient Hyperparameters optimization Through Model-based Reinforcement Learning and Meta-Learning

Hyperparameter optimization (HPO) plays a vital role in the performance of machine learning algorithms. When the algorithm is complex or the dataset is large, the computational cost of algorithm evaluation is very high, which is a major challenge for HPO. In this paper, we propose a reinforcement learning optimization method for efficient hyperparameter tuning. In particular, an RL agent selects hyperparameters sequentially, thus the search space is greatly reduced. The k-fold cross-validation result is used as a reward signal to update the agent. To speed up the training of the agent, we employ a predictive model to evaluate an algorithm with the selected hyperparameters. However, model inaccuracy is further exacerbated by long-horizon rollout, resulting in collapse performance. We employ a straightforward method to discipline the model rollout in short-horizon. After the short-horizon model rollout, the model is trained again with new samples. The model training and the model rollout are alternately repeated. To fast adapt to new tasks, meta-learning is used to train the predictive model on a variety of learning tasks, such that it can solve new learning tasks using only a small number of training samples. We evaluate our approach on 25 datasets and the experimental results demonstrate that with the limited resources, our method can significantly improve the efficiency and outperforms state-of-the-art Bayesian methods and evolution method.