Comparing objective and subjective measures of usability in a human-robot dialogue system

Foster, M. E., Giuliani, M. and Knoll, A. (2009) Comparing objective and subjective measures of usability in a human-robot dialogue system. In: Joint Conference of the 47th Annual Meeting of the ACL and the 4th International Joint Conference on Natural Language Processing of the AFNLP, Suntec, Singapore, 02-07 August 2009. ACL '09 Proceedings of the Joint Conference of the 47th Annual Meeting of the ACL and the 4th International Joint Conference on Natural Language Processing of the AFNLP: UNSPECIFIED Available from: http://eprints.uwe.ac.uk/31040

Full text not available from this repository

Publisher's URL: http://dl.acm.org/citation.cfm?id=1690270

Abstract/Description

We present a human-robot dialogue system that enables a robot to work together with a human user to build wooden construction toys. We then describe a study in which na¨ıve subjects interacted with this system under a range of conditions and then completed a user-satisfaction questionnaire.The results of this study provide a wide range of subjective and objective measures of the quality of the interactions. To assess which aspects of the interaction had the greatest impact on the users’ opinions of the system, we used a method based on the PARADISE evaluation framework (Walker et al., 1997) to derive a performance function from our data. The major contributors to user satisfaction were the number of repetition requests (which had a negative effect on satisfaction), the dialogue length, and the users’ recall of the system instructions (both of which contributed positively).

Item Type:Conference or Workshop Item (Paper)
Uncontrolled Keywords:comparing, objective, subjective, measures, usability, human-robot, dialogue, system
Faculty/Department:Faculty of Environment and Technology > Department of Engineering Design and Mathematics
ID Code:31040
Deposited By: Dr M. Giuliani
Deposited On:21 Feb 2017 15:21
Last Modified:21 Feb 2017 15:21

Request a change to this item