A container-based cloud-native architecture for the reproducible execution of multi-population optimization algorithms

Abstract Splitting a population into multiple instances is a technique used extensively in recent years to help improve the performance of nature-inspired optimization algorithms. Work on those populations can be done in parallel, and they can interact asynchronously, a fact that can be leveraged to create scalable implementations based on, among other methods, distributed, multi-threaded, parallel, and cloud-native computing. However, the design of these cloud-native, distributed, multi-population algorithms is not a trivial task. Using as a foundation monolithic (single-instance) solutions, adaptations at several levels, from the algorithmic to the functional, must be made to leverage the scalability, elasticity, (limited) fault-tolerance, reproducibility, and cost-effectiveness of cloud systems while, at the same time, conserving the intended functionality. Instead of an evolutive approach, in this paper, we propose a cloud-native optimization framework created from scratch, that can include multiple (population-based) algorithms without increasing the number of parameters that need tuning. This solution goes beyond the current state of the art, since it can support different algorithms at the same time, work asynchronously, and also be readily deployable to any cloud platform. We evaluate this solution’s performance and scalability, together with the effect other design parameters had on it, particularly the number and the size of populations with respect to problem size. The implemented platform is an excellent alternative for running locally or in the cloud, thus proving that cloud-native bioinspired algorithms perform better in their “natural” environment than other algorithms, and set a new baseline for scaling and performance of this kind of algorithms in the cloud.