A Self-Scalable and Auto-Regulated Request Injection Benchmarking Tool for Automatic Saturation Detection

Software applications providers have always been required to perform load testing prior to launching new applications. This crucial test phase is expensive in human and hardware terms, and the solutions generally used would benefit from further development. In particular, designing an appropriate load profile to stress an application is difficult and must be done carefully to avoid skewed testing. In addition, static testing platforms are exceedingly complex to set up. New opportunities to ease load testing solutions are becoming available thanks to cloud computing. This paper describes a Benchmark-as-a-Service platform based on: (i) intelligent generation of traffic to the benched application without inducing thrashing (avoiding predefined load profiles), (ii) a virtualized and self-scalable load injection system. The platform developed was experimented using two use cases based on the reference JEE benchmark RUBiS. This involved detecting bottleneck tiers, and tuning servers to improve performance. This platform was found to reduce the cost of testing by 50 percent compared to more commonly used solutions.