Data selection to avoid overfitting for foreign exchange intraday trading with machine learning

Abstract Algorithmic trading requires tuning hyperparameters to fit the time series data; however, it often suffers from overfitting of data that can lead to loss of money in action. Further, only a few studies discuss how to select trading exchange pairs and frequencies in response to the fitness of machine learning models. To cope with these problems, we developed a log-distance path loss model (to measure and reduce the overfitting in data modeling and determine exchange pairs and frequencies effectively. We conducted several experiments for different metrics using several influential factors such as machine learning models, learning objectives, trading strategies, and hyperparameter turning cases to validate the proposed approach. The obtained results indicate that the proposed metric is significantly superior to other methods in terms of accuracy, in-sample return (i.e., return of training data), and F1-score. Thus, using our path loss metric to guide data modeling, we provide a method to deal with the overfitting problem and yield positive trading returns.