A Revised Open Source Usability Defect Classification Taxonomy

Abstract Context : Reporting usability defects is a critical part of improving software. Accurately classifying these reported usability defects is critical for reporting, understanding, triaging, prioritizing and ultimately fixing such defects. However, existing usability defect classification taxonomies have several limitations when used for open source software (OSS) development. This includes incomplete coverage of usability defect problems, unclear criticality of defects, lack of formal usability training of most OSS defect reporters and developers, and inconsistent terminology and descriptions. Objective : To address this gap, as part of our wider usability defect reporting research, we have developed a new usability defect taxonomy specifically designed for use on OSS projects. Method : We used Usability Problem Taxonomy (UPT) to classify 377 usability defect reports from Mozilla Thunderbird, Firefox for Android, and the Eclipse Platform. At the same time, we also used the card-sorting technique to group defects that could not be classified using UPT. We looked for commonalities and similarities to further group the defects within each category as well as across categories. Results : We constructed a new taxonomy for classifying OSS usability defects, called Open Source Usability Defect Classification (OSUDC). OSUDC was developed by incorporating software engineering and usability engineering needs to make it feasible to be used in open source software development. The use of the taxonomy has been validated on five real cases of usability defects. However, evaluation results using the OSUDC were only moderately successful. Conclusion : The OSUDC serves as a common vocabulary to describe and classify usability defects with respect to graphical user interface issues. It may help software developers to better understand usability defects and prioritize them accordingly. For researchers, the OSUDC will be helpful when investigating both trends of usability defect types and understanding the root cause of usability defect problems.