TDAG: A Tunable Distributed Data Processing Model for Data Stream

As the Internet of Vehicles (IoV) becomes flourishing and the data generated by sensors be ubiquitous, there exist various kinds of IoV applications with different performance requirements. Hence, different distributed data processing systems (DDPS) clusters will coexist, e.g., a stream processing system cluster for real-time tasks and a batch one for statistics based data mining tasks, to meet the requirements of such IoV applications. However, it is not an economical or convenient way to maintain varied systems clusters, as the developers and/or administrators have to be familiar with all of these DDPSs, and of course, the deployment of multiple DDPS means a waste of resources compared to the deployment of one DDPS. Based on these observations, this paper proposes the TDAG as a solution. TDAG allows users to adjust the data processing from the streaming style to the batch style by encapsulating the input data with specific packing strategies. We have implemented TDAG in a prototype called TStream. The experimental tests show that our TStream is both effective and efficient.