A Multi-Level Usability Evaluation of Mobile Health Applications: A Case Study

Abstract Objective To report a methodological approach for the development of a usable mHealth application (app). Materials and methods This work was guided by a 3-level stratified view of health information technology (IT) usability evaluation framework. We first describe a number of methodologies for operationalizing each level of the framework. Following the description of each methodology, we present a case study which illustrates the use of our preferred methodologies for the development of a mHealth app. At level 1 (user-task), we applied a card sorting technique to guide the information architecture of a mobile HIV symptom self-management app, entitled mVIP. At level 2 (user-task-system), we conducted a usability evaluation of mVIP in a laboratory setting through end-user usability testing and heuristic evaluation with informatics experts. At level 3 (user-task-system-environment), usability of mVIP was evaluated in a real-world setting following the use of the app during a 3-month trial. Results The 3-level usability evaluation guided our work exploring in-depth interactions between the user, task, system, and environment. Integral to the findings from the 3-level usability evaluation, we iteratively refined the app’s content, functionality, and interface to meet the needs of our intended end-users. Discussion and conclusion The stratified view of the health IT usability evaluation framework is a useful methodological approach for the design, development, and evaluation of mHealth apps. The methodological recommendations for using the theoretical framework can inform future usability studies of mHealth apps.