An Empirical Study of User Story Quality and Its Impact on Open Source Project Performance

When software development teams apply Agile Software Development practices, they commonly express their requirements as User Stories. We aim to study the quality of User Stories and its evolution over time. Firstly, we develop a method to automatically monitor the quality of User Stories. Secondly, we investigate the relationship between User Story quality and project performance measures such as the number of reported bugs and the occurrence of rework and delays. We measure User Story quality with the help of a recently published quality framework and tool, Automatic Quality User Story Artisan (AQUSA). For our empirical work, we use six agile open source software projects. We apply time series analysis and use the Windowed Time Lagged Cross Correlation (WTLCC) method. Our results indicate that automatic User Story quality monitoring is feasible and may result in various distinct dynamic evolution patterns. In addition, we found the following relationship patterns between User Story quality and the software development aspects. A decrease/increase in User Story quality scores is associated with (i) a decrease/increase of the number of bugs after 1–13 weeks in short-medium projects, and 12 weeks in longer ones, (ii) an increase in rework frequency after 18–28, 8–15, and 1–3 weeks for long, medium, and short projects, respectively, and (iii) an increase in delayed issues after 7–20, 8–11, and 1–3 weeks for long, medium, and short duration projects.