Continuous Integration for Web-Based Software Infrastructures: Lessons Learned on the webinos Project

Testing web-based software infrastructures is challenging. The need to interact with different services running on different devices, with different expectations for security and privacy contributes not only to the complexity of the infrastructure, but also to the approaches necessary to test it. Moreover, as large-scale systems, such infrastructures may be developed by distributed teams simultaneously making changes to APIs and critical components that implement them. In this paper, we describe our experiences testing one such infrastructure – the webinos software platform – and the lessons learned tackling the challenges faced. While ultimately these challenges were impossible to overcome, this paper explores the techniques that worked most effectively and makes recommendations for developers and teams in similar situations. In particular, our experiences with continuous integration and automated testing processes are described and analysed.