Building Efficient Probability Transition Matrix Using Machine Learning from Big Data for Personalized Route Prediction

Abstract Personalized route prediction is an important technology in many applications related to intelligent vehicles and transportation systems. Current route prediction technologies used in many general navigation systems are, by and large, based on either the shortest or the fastest route selection. Personal traveling route prediction is a very challenging big data problem, as trips getting longer and variations in routes growing. It is particularly challenging for real-time in-vehicle applications, since many embedded processors have limited memory and computational power. In this paper we present a machine learning algorithm for modeling route prediction based on a Markov chain model, and a route prediction algorithm based on a probability transition matrix. We also present two data reduction algorithms, one is developed to map large GPS based trips to a compact link-based standard route representation, and another a machine learning algorithm to significantly reduce the size of a probability transition matrix. The proposed algorithms are evaluated on real-world driving trip data collected in four months, where the data collected in the first three months are used as training and the data in the fourth month are used as testing. Our experiment results show that the proposed personal route prediction system generated more than 91% prediction accuracy in average among the test trips. The data reduction algorithm gave about 8:1 reduction in link-based standard route representation and 23:1 in reducing the size of probability transition matrix.