PAVER 2.0: an open source environment for automated performance analysis of benchmarking data

In this paper we describe PAVER 2.0, an environment (i.e. a process and a suite of tools supporting that process) for the automated performance analysis of benchmarking data. This new environment improves on its predecessor by addressing some of the shortcomings of the original PAVER (Bussieck et al. in Global optimization and constraint satisfaction, lecture notes in computer science, vol 2861, pp 223−238. Springer, Berlin, 2003. doi: 10.1007/978-3-540-39901-8_17 ) and extending its capabilities. The changes serve to further the original goals of PAVER (automation of the visualization and summarization of benchmarking data) while making the environment more accessible for the use of and modification by the entire community of potential users. In particular, we have targeted the end-users of optimization software, as they are best able to make the many subjective choices necessary to produce impactful results when benchmarking optimization software. We illustrate with some sample analyses conducted via PAVER 2.0.