A Multi-touch Gesture Vocabulary Design Methodology for Mobile Devices

Abstract

This study aims to propose a methodology for designing multi-touch gesture vocabularies applicable on mobile devices. The specific research objectives are 1) Proposing an integrated procedure for designing gesture vocabularies

2) Proposing a systematic framework to analyze and create gesture vocabularies

3) Validating the proposed methodology through a case study for designing mobile web browsing gestures

4) Suggesting best multi-touch gesture vocabularies for mobile web browsers (MWBs).The integrated procedure involved both empirical approach and analytical approach. The empirical approach collected intuitive gestures from end-users for mobile web browsing commands. Users defined appropriate gestures intuitively themselves. On the other hand, the analytical approach created gestures based on a systematic framework. The framework consisted of gesture features and gesture elements, and the two attributes were combined to create various gestures. Designers generated gesture vocabularies analytically based on the framework. The analytical approach followed the empirical approach in the procedures.Basic gesture elements that constitute multi-touch gestures were identified. Extrinsic postures or motions were decomposed into the smallest units, and the categorized units became gesture elements. The gesture elements allowed observers to describe gestures concretely, and moreover the elements were used to analyze the usage patterns of gestures. Taxonomy of multi-touch gestures was also developed based on the basic elements. This study identified 6 basic elements of multi-touch gestures in mobile devices. The proposed elements were ‘posture’, ‘location’, ‘pose’, ‘touch’, ‘path’, and ‘device’.The usefulness and practicality of the methodology were validated through a case study. The case study identified mobile web browsing gestures to improve short-cut interactions. The initial gestures for the commands were collected from 36 end-users who did not have any experience with gesture interaction on web browsers. A total of 642 gestures were collected in the empirical approach. Six volunteers created gestures based on the systematic design framework, and they generated 314 gestures in the analytical approach. The proposed methodology helped designers to find appropriate gesture vocabularies for various commands on mobile devices.Gesture candidates were selected from the gestures which were collected by the empirical approach and created by the analytical approach. The selected candidates were evaluated by the 22 subjects. Their intuitiveness and subjective satisfaction scores are compared with each other to identify the most suitable gesture vocabularies. Finally, 11 best gestures and 38 top group gestures were identified for mobile web browsing commands.This study tried to find cognitively and physically appropriate web browsing gestures to end-users. However, technological issues such as confliction between different commands and restriction of recognition algorithm were not considered. Based on gestures identified in this study, further study considering these issues should be conducted in the future.