AuraEN: Autonomous Resource Allocation for Cloud-Hosted Data Processing Pipelines

Ensuring cost-effective end-to-end QoS in an IoT data processing pipeline (DPP) is a non-trivial task. A key factor that affects the overall performance is the amount of computing resources allocated to each service in the pipeline. In this demo paper, we present AuraEN, an Autonomous resource allocation ENgine that can proactively scale the resources of each individual service in the pipeline in response to predicted workload variations so as to ensure end-to-end QoS while optimizing the associated costs. We briefly describe the AuraEN system architecture and its implementation and demonstrate how it can be used to manage the resources of a DPP hosted on the Amazon EC2 cloud.