Multi-modality gesture detection and recognition with un-supervision, randomization and discrimination

Chen, G., Clarke, D., Weikersdorfer, D., Giuliani, M., Gaschler, A. and Knoll, A. (2014) Multi-modality gesture detection and recognition with un-supervision, randomization and discrimination. In: Agapito, L., Bronstein, M. and Rother, C., eds. (2014) Computer Vision - ECCV 2014 Workshops. Zurich, Switzerland: Springer International Publishing, pp. 608-622. ISBN 9783319161778 Available from:

Full text not available from this repository

Publisher's URL:


We describe in this paper our gesture detection and recognition system for the 2014 ChaLearn Looking at People (Track 3: Gesture Recognition) organized by ChaLearn in conjunction with the ECCV 2014 conference. The competition’s task was to learn a vacabulary of 20 types of Italian gestures and detect them in sequences. Our system adopts a multi-modality approach for detecting as well as recognizing the gestures. The goal of our approach is to identify semantically meaningful contents from dense sampling spatio-temporal feature space for gesture recognition. To achieve this, we develop three concepts under the random forest framework: un-supervision; discrimination; and randomization. Un-supervision learns spatio-temporal features from two channels (grayscale and depth) of RGB-D video in an unsupervised way. Discrimination extracts the information in dense sampling spatio-temporal space effectively. Randomization explores the dense sampling spatio-temporal feature space efficiently. An evaluation of our approach shows that we achieve a mean Jaccard Index of 0.64890.6489, and a mean average accuracy of 90.3%90.3% over the test dataset.

Item Type:Book Section
Uncontrolled Keywords:multi-modality gesture, unsupervised learning, random forest, discriminative training
Faculty/Department:Faculty of Environment and Technology > Department of Engineering Design and Mathematics
ID Code:31037
Deposited By: Dr M. Giuliani
Deposited On:21 Feb 2017 16:43
Last Modified:21 Feb 2017 16:43

Request a change to this item