Empirical Evaluation of a Method for Monitoring Cloud Services based on Models at Runtime

Cloud computing is being adopted by commercial and governmental organizations driven by the need to reduce the operational cost of their information technology resources and search for a scalable and flexible way to provide and release their software services. In this computing model, the Quality of Services (QoS) is agreed between service providers and their customers through Service Level Agreements (SLA). There is thus a need for systematic approaches with which to assess the quality of cloud services and their compliance with the SLA. In previous work, we introduced a generic method for Monitoring cloud Services using models at RunTime (MoS@RT), which allows the monitoring requirements or the metric operationalizations of these requirements to be changed at runtime without the modification of the underlying infrastructure. In this paper, we present the design of a monitoring infrastructure that supports the proposed method with its instantiation to a specific platform and reports the results of an experiment carried out to evaluate the perceived efficacy of 58 undergraduate students when using the infrastructure to configure the monitoring of cloud services deployed on the Microsoft Azure platform. The results show that the participants perceived MoS@RT to be easy to use, useful, and they also expressed their intention to use the method in the future. Although further experiments must be carried out to strengthen these results, MoS@RT has proved to be a promising monitoring method for cloud services.