Processing High-Volume Geospatial Data: A case of monitoring Heavy Haul Railway operations

Sensor technology such as GPS can be used in the mapping of transportation networks (e.g., road, rail). The advancement of sensor technology enables fast and cost-effective acquisition of geospatial data. However, GPS suffers errors in positional accuracy due to factors such as signal arrival time, ionospheric effects, multipath distortions and so on. In railway systems, positional accuracy is of utmost importance because it is possible that the current position of a particular wagon may be read to be in a wrong track causing incorrect analysis for safety and maintenance of track and wagons. Also, the numerous lightweight sensors installed in each wagon along with GPS produce a large amount of continuous data streams as multiple trains operate for long hours during its trip. This will cause problems as huge amounts of location data needs to be processed continuously and the traditional data processing and storage applications can not handle it. In this paper, the authors propose efficient algorithms and a suitable data structure to achieve rapid and accurate location mappings to increase both the run time performance and mapping accuracy based on 2 months of historical data consisting of approximately 250 million records. The authors' large scale evaluation demonstrates that their system is capable of real-time performance - processing tens of thousands of records per second and, accurate - mapping rail track information with 98.5% accuracy.