Performance characteristics of Source calibration service

Using optimisers to calibrate hydrological models is a computationally intensive process. Most optimisation algorithms run on desktop machines, with some running on Linux clusters and a couple that run on cloud infrastructure (e.g. cloudPEST). Complex hydrological models require a relatively powerful machine and calibration runtimes vary from an hour or less, to days and sometimes weeks. Increasingly, organisations are looking to outsource provision and management of computationally intensive infrastructures. While virtualisation technology can provide similar performance to high end desktops, there are opportunities to harness parallelisation and reduce calibration times, by hosting the modelling software on the cloud infrastructure and exposing its functionality through web services. This paper investigates the practicality and performance of implementing a calibration wrapper to the eWater Source river modelling package. The Source calibration service allows user to calibrate models, where the modelling software, eWater Source, is running on the cloud and not on end user's premises. The aim of this analysis was to compare the performance characteristics of a simple GR4J model for the Legerwood catchment using eWater Source running as desktop software versus running as a Source calibration service on the cloud. Shuffle Complex Evolution was used as the parameter optimisation algorithm for the GR4J model parameters. The eWater Source product running as desktop software took around 4 minutes to calibrate the model whereas the Source calibration service took around 73 minutes to do the same calibration with similar results. The difference in run times can be attributed either to: 1) the chatty nature of communication between the machines running the eWater Source and the optimization algorithm; and/or 2) time inefficient implementation of SCEoptim routine from the hydromad package; and/or 3) performance bottle necks in Source's external interface which exposes eWater Source modelling capability through command prompt. Given the long simulation runtimes, the current Source calibration service fails to meet expectations of hydrological model builders for improved performance. For software implementers, we would recommend careful attention to the software architecture and performance characteristics of proposed cloud-based software implementations early in development. In this case, we anticipate future improvements to the infrastructure, or renewed effort improving the implementation would lead to a faster implementation.