How can I help you? Comparing engagement classification strategies for a robot bartender

Foster, M. E., Gaschler, A. and Giuliani, M. (2013) How can I help you? Comparing engagement classification strategies for a robot bartender. In: 15th ACM on International Conference on Multimodal Interaction, Sydney, Australia, 09-13 December 2013. ICMI '13 Proceedings of the 15th ACM on International conference on multimodal interaction: UNSPECIFIED Available from:

Full text not available from this repository

Publisher's URL:


A robot agent existing in the physical world must be able to understand the social states of the human users it interacts with in order to respond appropriately. We compared two implemented methods for estimating the engagement state of customers for a robot bartender based on low-level sensor data: a rule-based version derived from the analysis of human behaviour in real bars, and a trained version using supervised learning on a labelled multimodal corpus. We first compared the two implementations using cross-validation on real sensor data and found that nearly all classifier types significantly outperformed the rule-based classifier. We also carried out feature selection to see which sensor features were the most informative for the classification task, and found that the position of the head and hands were relevant, but that the torso orientation was not. Finally, we performed a user study comparing the ability of the two classifiers to detect the intended user engagement of actual customers of the robot bartender; this study found that the trained classifier was faster at detecting initial intended user engagement, but that the rule-based classifier was more stable.

Item Type:Conference or Workshop Item (Paper)
Uncontrolled Keywords:social signal processing, supervised learning
Faculty/Department:Faculty of Environment and Technology > Department of Engineering Design and Mathematics
ID Code:31043
Deposited By: Dr M. Giuliani
Deposited On:21 Feb 2017 15:41
Last Modified:21 Feb 2017 15:41

Request a change to this item