Measuring Agile teamwork: a comparative analysis between two models

Background: The literature presents distinct models to assess the Teamwork Quality (TWQ) for agile teams. These models have different constructs and, consequently, measures. Unfortunately, there are no results of empirical studies contrasting the existing models. Goal: To fill this gap, this study aims to provide a deep insight into how two state-of-the-art TWQ models (i.e., instruments) compare to each other with respect to the calculated results for practical use. Method: We performed an empirical study to compare both models in terms of their constructs and measures. For comparing their constructs, we ranked the variables from both models, given their relative impact on teamwork quality, and compared the ranks. For comparing their measures, we collected data using both instruments by interviewing 158 team members from two software development companies. First, we theoretically mapped the variables from both models, given their definition. Afterward, from the collected data, we calculated each of the models' variables using the procedures proposed by their creators. Then, we analyzed the level of agreement between the models for each variable, using the Mean Relative Error (MRE). Results: In terms of the constructs, for practical purposes, both models are equivalent, except for including a variable for Team Autonomy. For the measures, the instruments produced similar results for five variables (i.e., Communication, Coordination, Balance of Member Contribution, Effort, and Cohesion). For the sixth variable (Mutual Support), we presented evidence that the results are similar in the models. Still, we believe that more research is needed to analyze it. Conclusions: We believe that we have enough evidence to claim that, for practical purposes, both models yield similar results. Thus, the study indicates that, it is up to the teams to choose which model best suits their context. Still, we concluded that there is a need for more research regarding how to assess TWQ for agile teams.