Using the SUS to Help Demonstrate Usability’s Value to Business Goals
18
Citation
10
Reference
10
Related Paper
Citation Trend
Abstract:
As part of an effort to connect results of human factors testing with the needs of internal business clients, a usability testing program was created for smartphones to establish a link between usability metrics and business indicators of success in the marketplace. In addition to standard usability metrics of effectiveness, efficiency, and satisfaction (ISO, 1998), a measure of hedonics (Hassenzahl, Beu, and Burmester, 2001) was used. Of particular note was the use of the System Usability Scale (SUS) (Brooke, 1996) to provide a single measure of user satisfaction for all smartphones tested. Data from 54 studies involving 872 participants were analyzed to determine if usability metrics predicted marketplace outcomes. Results show that usability metrics are significantly correlated with indicators of business success and that the SUS accounted for most of usability’s contribution. Discussion of these results, the value of SUS, and implications for usability professionals are discussed.Keywords:
System usability scale
Usability goals
Usability defects test escapee can have a negative impact on the success of software. It is quite common for projects to have a tight timeline. For these projects, it is crucial to ensure there are effective processes in place. One way to ensure project success is to improve the manual processes of the usability inspection via automation. An automated usability tool will enable the evaluator to reduce manual processes and focus on capturing more defects in a shorter period of time. Thus improving the effectiveness of the usability inspection and minimizing defects escapee. There exist many usability testing and inspection methods. The scope of this paper is on the Heuristic Evaluation (HE) procedures automation. The Usability Management System (UMS) was developed to automate as many manual steps as possible throughout the software development life cycle (SDLC). It is important for the various teams within the organization to understand the benefits of automation. The results show that with the help of automation more usability defects can be detected. Hence, enhancing the effectiveness of usability evaluation by an automated Heuristic Evaluation System is feasible.
Cognitive walkthrough
Usability goals
System usability scale
Time line
Cite
Citations (17)
Introduction Usability evaluation is essential to make sure that software products newly released are easy to use, efficient, and effective to reach goals, and satisfactory to users. For example, when a software company wants to develop and sell a new product, the company needs to evaluate usability of the new product before launching it at a market to avoid the possibility that the new product may contain usability problems, which span from cosmetic problems to severe functional problems. Three widely used methods for usability evaluation are Think Aloud (TA), Heuristic Evaluation (HE) and Cognitive Walkthrough (CW). TA method is commonly employed with a lab-based user testing, while there are variants of TA methods, including thinking out aloud at user's workplace instead of at labs. What we discuss here is the TA method that is combined with a lab-based user testing, in which test users use products while simultaneously and continuously thinking out aloud, and experimenters record users' behaviors and verbal protocols in the laboratory. HE is a usability inspection method, in which a small number of evaluators find usability problems in a user interface design by examining an interface and judging its compliance with well-known usability principles, called heuristics. CW is a theory-based method, in which evaluators evaluate every step necessary to perform a scenario-based task, and look for usability problems that would interfere with learning by exploration. These three methods have their own advantages and disadvantages. For instance, TA method provides good qualitative data from a small number of test users, but laboratory environment may influence test user's behaviors. HE is a cheap, fast and easy-to-use method, while it often finds too specific and low-priority usability problems, including even not real problems. CW helps find mismatches between users' and designers' conceptualization of a task, but it needs extensive knowledge of cognitive psychology and technical details to apply. However, even though these advantages and disadvantages show overall characteristics of three major usability evaluation methods, we cannot compare them quantitatively and see their efficiency clearly. Because one of reasons why so-called discounted methods, such as HE and CW, were developed is to save costs of usability evaluation, cost-related criteria for comparing usability evaluation are meaningful to usability practitioners as well as usability researchers. One of the most disputable issues related to cost of usability evaluation is sample size. That is, how many users or evaluators are needed to achieve a targeted usability evaluation performance, for example, 80% of overall discovery rate? The sample size of usability evaluation is known to depend on an estimate of problem discovery rate across participants. The overall discovery rate is a common quantitative measure that is used to show the effectiveness of a specific usability evaluation method in most of usability evaluation studies. It is also called overall detection rate or thoroughness measure, which is the ratio of 'the sum of unique usability problems detected by all experiment participants' against 'the number of usability problems that exist in the evaluated systems', ranging between 0 and 1. The overall discovery rates were reported more than any other criterion measure in the usability evaluation experiments and also a key component for projecting required sample size for usability evaluation study. Thus, how many test users or evaluators participate in the usability evaluation is a critical issue, considering its cost-effectiveness.
Cognitive walkthrough
Usability goals
Think aloud protocol
Software walkthrough
Heuristics
Cite
Citations (349)
The evaluation of software usability is a very important topic for society. Usability depends on factors intrinsic to the task to be performed. In hydroelectric plants, the main task to be performed by operators is the analysis and monitoring of the sensors of the generating unit. The hypothesis is that the use of a 3D interface brings more usability and efficiency to find the sensors. The work applies Nielsen's heuristics to ensure good usability. This work intends to evaluate the usability of SOMA – System Oriented to Monitoring of Engineering Assets, a software that monitor Hydroelectric Power Plant, operation of electricity generation plants, in terms of increasing availability and reducing maintenance costs. The results of this indirect usability evaluation were positive, suggesting new features to the software to help engineers to perform even better their tasks on monitoring assets.
System usability scale
Cognitive walkthrough
Cite
Citations (0)
As the field of Usability Engineering gains more importance day by day, the phenomenon of testing the Usability of software and hardware products is becoming popular too. But, in presence of so many usability testing techniques, selecting an assessment parameter for performing usability testing is considered to be a great challenge for testers. The aim of this research is to suggest usability heuristics as an assessment parameter to Usability testing. To achieve the aim and objectives of this research, renowned Usability expert Jakob Nielsen's 10 Usability Heuristics have been chosen for assessing usability of some public sectors universities websites. In the second step, by using these Heuristics as an assessment parameter, Usability Survey and Heuristic Evaluation techniques are applied on these websites. It has been found that Heuristics are key attributes of software products and considered good measuring parameters to assess usability. Meanwhile this research is also a contribution towards rating the Usability of some academic websites. And hence, in this research, Usability Heuristics have been chosen as an assessment parameter for performing Usability Testing.
Usability goals
Cognitive walkthrough
System usability scale
Heuristics
Cite
Citations (47)
Usability testing is a method of evaluating the usability of a given product in a given environment. Formal usability testing is an empirical method that requires the design of a formal usability experiment that is undertaken under controlled conditions in a usability laboratory. Formal usability testing produces actual measures of usability in terms of task effectiveness and efficiency. Existing software tools for visualising the results of formal usability testing are, however, limited and inadequate. This paper will discuss the typical components of such usability data and the design of a multiple view visualisation system to effectively view task efficiency that was based on recent guidelines for multiple view systems.
Usability goals
Cognitive walkthrough
System usability scale
Component-based usability testing
Cite
Citations (8)
The purpose of this chapter is to provide an overview of usability and usability techniques. It discusses interaction design with a bias on web-based technologies, focusing on usability and usability definitions. Usability principles are discussed with a particular focus on web usability, and some of the techniques that can be used to incorporate usability design and testing. The chapter also considers aspects of electronic commerce usability. These techniques are divided into usability evaluation, (heuristic evaluation, checklists, usability tests, think aloud approaches) followed by designing with usability in mind (including user and task analysis, walkthroughs, user experience and scenario-based usability). Several E-commerce case studies, from a developing economy perspective, are then analysed, before finally discussing E-commerce usability.
Cognitive walkthrough
Usability goals
System usability scale
Cite
Citations (0)
This chapter introduces the concept of usability and provides examples of how usability has been used in digital library evaluations. Usability is a user-centered evaluation and has a theoretical base in human-computer interaction. The most concise definition of usability is “fit for use.” The dimensions of usability may also include usefulness, usableness, ease of use, effectiveness, efficiency, satisfaction, learnability, memorability, and error tolerant. The common methods of usability evaluation are described in this chapter, including formal usability testing, usability inspection, card sort, category membership expectation, focus groups, questionnaires, think aloud, analysis of site usage logs, cognitive walkthrough, heuristic evaluation, claims analysis, concept-based analysis of surface and structural misfits (CASSM), paper prototyping, and field study. Some evaluations employed one method; some used a combination of methods. There is a need for more empirical studies in order to understand users’ needs. Culturability is an interesting area to explore.
Cognitive walkthrough
Usability goals
Card sorting
System usability scale
Think aloud protocol
Cite
Citations (4)
Usability goals
System usability scale
Cognitive walkthrough
Cite
Citations (4)
Usability plays a major role towards user acceptance of website and it is increasingly becoming an important topic for organizations that develop and implement websites to market their products and services. Thus evaluating usability is critical for organizations in order to develop user friendly websites. One major challenge in usability evaluation is that most of the evaluation methods are subjective which depends on the evaluator's personal interpretations and judgment of the website. This is due to the lack of availability of automated tools in the respective field. Further, most of the usability evaluation methods are involved with manual approaches that consume time and money which is not affordable for organizations. Site-ability is an automation tool for website usability evaluation which is capable of mimicking human usability experts and it is based on usability guidelines provided by U.S. Department of Health and Human Services' (HHS) Research-Based Web Design and Usability Guidelines.
Usability goals
System usability scale
Cognitive walkthrough
Cite
Citations (3)
The use of the System Usability Scale (SUS) questionnaire in the measurement of e-commerce usability has been widely carried out. However, the SUS score is not an adequate measure to express the level of user acceptance and satisfaction. Other evaluations are needed to complement the usability test, including assessments based on expert judgment. The proposed method consists of two stages, the heuristic evaluation stage, which involves expert judgment, and the SUS questionnaire stage based on user perceptions of the e-commerce website. Input from experts is expected to be able to show better the usability issues faced in using the website. Expert and user perspectives are combined to get user input in design improvements. We collect data from experts and users about their perceptions of the usability of Shoppee e-commerce websites. Most users agree that the Shopee site is excellent (grade B-). The results of the examination by the expert stated that the Shopee site was also excellent. Nine out of ten evaluation criteria scored above 72%. The most usability issue is the flexibility and efficiency of the system, especially problems in search engines.
System usability scale
Usability goals
Cognitive walkthrough
Cite
Citations (42)