Identifying Operational Points for Deterministic Execution in Cloud Computing

Cloud based applications are increasingly structured as loosely coupled microservices for various reasons including agility and scalability. Especially in such architectures that feature interacting microservices, performance degradation from a subset of resource-starved microservices could rapidly spread to other microservices creating additional performance hotspots and increasing tail latencies in inter-microservice communication - directly impacting deterministic execution of the overall cloud application. Optimizing resource allocation is one way to address this problem, and it has received some research attention. Specifically, two categories of approaches have been explored: a) correction after detection of performance bottlenecks, and b) predictive identification and correction of upcoming bottlenecks. This paper focuses on CPU support for deterministic cloud computing for either of these approaches, and also on identification of better operational points upfront. It highlights the benefits of using CPU hardware-based approaches through real-world benchmarks and explores an active learning approach to identify operational points enabling static and semi-static pre-configuration of resources. We present measurements from industry benchmark applications to demonstrate benefits of this CPU-based approach and illustrate effectiveness of our proposed preconfiguration method through a simulator featuring configurable number of microservices and inter-connectivity.