Transformers \`a Grande Vitesse.

Robust travel time predictions are of prime importance in managing any transportation infrastructure, and particularly in rail networks where they have major impacts both on traffic regulation and passenger satisfaction. We aim at predicting the travel time of trains on rail sections at the scale of an entire rail network in real-time, by estimating trains' delays relative to a theoretical circulation plan. Existing implementations within railway companies generally work using the approximation that a train's delay will stay constant for the rest of its trip. Predicting the evolution of a given train's delay is a uniquely hard problem, distinct from mainstream road traffic forecasting problems, since it involves several hard-to-model phenomena: train spacing, station congestion and heterogeneous rolling stock among others. We first offer empirical evidence of the previously unexplored phenomenon of delay propagation in the French National Railway Network, leading to delays being amplified by interactions between trains. We then contribute a novel technique using the transformer architecture and pre-trained embeddings to make real-time massively parallel predictions for train delays at the scale of the whole rail network (over 3k trains at peak hours, making predictions at an average horizon of 70 minutes). Our approach yields very positive results on real-world data when compared to currently-used and experimental prediction techniques. Our work is in the early stages of implementation for industrial use at the French railway company SNCF for passenger information systems, and a contender as a tool to aid traffic regulation decisions.