Investigating Optimal Regimes for Prediction in the Stock Market

Forecasting stock prices in the market its known to be an extremely difficult task, where even the predictability of the series itself is a controversial matter. The present study investigates the existence of periods within the series more suitable for prediction, and whether the identification and exploitation of those periods could be learned from data. In order to do that, the Predictability Crawler (P-Craw) framework is proposed. The technique uses optimizations routines such as the Particle Swarm optimization (PSO) or Genetic Algorithms (GA) to select subsets of historical data where statistical learning algorithms can be more efficiently trained. When tested against simulated data, The P-Craw is able to reliably identify the optimal subsets in scenarios ranging from 40% to 100% of predictable samples in the data. To access if the framework brings any improvement when used in a real world scenario, it is tested in a dataset containing intraday data from the Brazilian stocks exchange (BOVESPA). When benchmarked against training with all the samples for the series in the BOVESPA dataset the use of the framework is able to significantly raise the Correct Directional Changes (CDC) of the trained models while reducing the Mean Absolute Error (MAE) in up to 19%.