Chapter 4 – The Centrality of Data: Data Lifecycle and Data Pipelines

As Intelligent Transportation Systems (ITS) technologies mature, we can envision many scenarios where intelligent agents provide adaptable, dynamic information needed to make decisions in real time. Such information depends on both real-time and historical data. Knowing what data to use and how to compare past and present measurements is essential in assessing the impact of phenomena such as climate change or population migration. Equally important is understanding of the long-term value of data independent of its immediate use or rushed evaluations of its relevance. The data lifecycle is an abstract view that enables researchers and practitioners to respect data, steward it, and care for it for future generations. This chapter provides the reader with an understanding of various models of the lifecycle of data and makes the connection from the data lifecycle to the “data pipeline” as a physical manifestation of portions or all of a data lifecycle through running software.