A comparison of usability evaluation methods: A case study on a camera application for visually impaired people

Abstract

The rapid growing demand of smart-phone has provided the benefit of convenient access to copious information and services for users. Furthermore, it behaves as the primary cause of continuous increase in smart-phone usages. However, the potential negative effects exist as well. Since the target user bases for a smart-phone device are non-handicapped users, the smart-phone and applications provide inconvenient user interfaces for the disabled, especially for the blind and visually impaired users. As an illustration, a mobile phone application, especially with short product development cycle has a higher probability of negligence in terms of the user interfaces for the disabled. The bottleneck in the smart-phone usages by the blind is difficult to detect by experts. Indeed, an actual experiment conducted by the blind would convey more suitable solutions to efficient and effective usages of requirements that would satisfy the blind users compare to experts. Thus, careful contrast of the usability evaluation methods for a short application product development cycle and requirement of the blind users is essential. This paper aims to investgate and analyze the variances in the smart-phone application for the blind after experts and the blind conducts usability testing. The experiment was proceeded with two groups of experimenters: three experts for a heuristic evaluation and twenty blind people for a benchmarking test. A target application for the blind is a prototype model of camera application with seven functionalities (Changing font size on the screen, Help function, Description of the figures on the screen, Automatic shooting guide function that automatically instruct optimum configurations, Modifying and checking photo name, Recording a description of the photo, and Sharing photos on SNS).After comparing usability problems from conducted experiments (heuristic evaluation and benchmarking test), detected problems were tended to have a sufficient gap between two experimenter groups in different usability principle and components of UI properties. Specifically, providing information in terms of usability principle was the most frequently detected from both evaluation methods, usability problems relative to the size and the absence of feedback in UI properties were the most often identified from heuristic evaluation, and usability problems related to the feedback contents and the function were notably spotted from benchmarking test. Among all the detected bottlenecks, twenty-two problems were from only experts group, twenty-five problems were from both experts and the blind groups, and two hundred-eight problems were from only the blind group. In detail, some usability problem drawn from the benchmarking test were associated with problems such as an inconvenient horizontal usage of a smart-phone.These results reflect a message that those problems were detected due to a lack of considering characteristics of the blind. Indeed, the understanding of their needs would bring more suitable smart-phone application for the blind.Consequently, this paper will be used creditably for the further improvement in usability evaluation method that supports the future smart-phone application design and development for the blind.