BeFaaS: An Application-Centric Benchmarking Framework for FaaS Platforms.

Following the increasing interest and adoption of FaaS systems, benchmarking frameworks for determining non-functional properties have also emerged. While existing (microbenchmark) frameworks only evaluate single aspects of FaaS platforms, a more holistic, application-driven approach is still missing. In this paper, we design and present BeFaaS, an application-centric benchmarking framework for FaaS environments that focuses on the evaluation with realistic and typical use cases for FaaS applications. BeFaaS comes with two built-in benchmarks (an e-commerce and an IoT application), is extensible for new workload profiles and new platforms, supports federated benchmark runs in which the benchmark application is distributed over multiple providers, and supports a fine-grained result analysis. Our evaluation compares three major FaaS providers in single cloud provider setups and analyzes the traces of a federated fog setup. It shows that BeFaaS is capable of running each benchmark automatically with minimal configuration effort and providing detailed insights for each interaction.