Hermes: Efficient Cache Management for Container-based Serverless Computing

Serverless computing systems are shifting towards shorter function durations and larger degrees of parallelism to eliminate intolerable latency. For container-based serverless computing, the state-of-the-art efforts fail to ensure low latency because on-demand container images reloading from remote storage can increase the data transmission rate and downgrades system performance. In this paper we propose Hermes with a two-level caching mechanism to reduce the latency and minimize data transmission rate when massive serverless workloads arrive. Hermes optimizes memory caching by persisting metadata cache and prolonging the lifetime of file cache to improve the cache efficiency of image files. Instead of reclaiming memory, Hermes uses disk caching to reduce memory usage, and gets a low data transmission rate by reloading from local disk cache. Experiment results show that Hermes can reduce 90% of the data transmission rate and improve the runtime performance of serverless workloads up to 5 × in a machine with 300 concurrent containers compared to state-of-the-art efforts.