Gender and gaze gesture recognition for human-computer interaction

Zhang, W., Smith, M., Smith, L. and Farooq, A. (2016) Gender and gaze gesture recognition for human-computer interaction. Computer Vision and Image Understanding, 149. pp. 32-50. ISSN 1077-3142 Available from:

PDF - Accepted Version
Available under License Creative Commons Attribution Non-commercial No Derivatives 4.0.

[img] Microsoft Word 2007 - Accepted Version
Available under License Creative Commons Attribution Non-commercial No Derivatives 4.0.


Publisher's URL:


The identification of visual cues in facial images has been widely explored in the broad area of computer vision. However theoretical analyses are often not transformed into widespread assistive Human-Computer Interaction (HCI) systems, due to factors such as inconsistent robustness, low efficiency, large computational expense or strong dependence on complex hardware. We present a novel gender recognition algorithm, a modular eye centre localisation approach and a gaze gesture recognition method, aiming to escalate the intelligence, adaptability and interactivity of HCI systems by combining demographic data (gender) and behavioural data (gaze) to enable development of a range of real-world assistive-technology applications. The gender recognition algorithm utilises Fisher Vectors as facial features which are encoded from low-level local features in facial images. We experimented with 4 types of low-level features: greyscale values, Local Binary Patterns (LBP), LBP histograms and Scale Invariant Feature Transform (SIFT). The corresponding Fisher Vectors were classified using a linear Support Vector Machine. The algorithm has been tested on the FERET database, the LFW database and the FRGCv2 database, yielding 97.7%, 92.5% and 96.7% accuracy respectively. The eye centre localisation algorithm has a modular approach, following a coarse-to-fine, global-to-regional scheme and utilising isophote and gradient features. A Selective Oriented Gradient filter has been specifically designed to detect and remove strong gradients from eyebrows, eye corners and self-shadows (which sabotage most eye centre localisation methods). The trajectories of the eye centres are then defined as gaze gestures for active HCI. The eye centre localisation algorithm has been compared with 10 other state-of-the-art algorithms with similar functionality and has outperformed them in terms of accuracy while maintaining excellent real-time performance. The above methods have been employed for development of a data recovery system that can be employed for implementation of advanced assistive technology tools. The high accuracy, reliability and real-time performance achieved for attention monitoring, gaze gesture control and recovery of demographic data, can enable the advanced human-robot interaction that is needed for developing systems that can provide assistance with everyday actions, thereby improving the quality of life for the elderly and/or disabled.

Item Type:Article
Uncontrolled Keywords:gender, human computer interaction
Faculty/Department:Faculty of Environment and Technology > Department of Engineering Design and Mathematics
ID Code:28497
Deposited By: W. Zhang
Deposited On:21 Mar 2016 14:21
Last Modified:08 Apr 2017 00:40

Request a change to this item

Total Document Downloads in Past 12 Months

Document Downloads

Total Document Downloads

More statistics for this item...