Title: Automated facial expression recognition – an integrated approach with optical flow analysis and Support Vector Machines

Authors: Nanda Surendran, Shane Xie

Addresses: 1029 W 24th Street, Los Angeles, CA 90007, USA. ' Mechanical Engineering Department, The University of Auckland, Private bag 92019, Auckland, New Zealand

Abstract: In this article, an automated facial expression recognition system capable of classifying expressions by analysing the shape of the strongest facial feature responsible for the expression, namely the eyebrows or the mouth region is presented. This novel approach integrates Lucas–Kanade method for optical flow analysis to determine the motion flow vectors between two neighbouring images from a video sequence and support vector machines (SVM) for expression classification. The expressions classified are anger, disgust, fear, sad, smile with open mouth and closed mouth, surprise and neutral expression. By using just the strongest facial feature to train 16 SVM classifiers optimises the expression recognition irrespective of the degree of expressiveness and makes the system less computationally expensive. The SVM has been trained with 300 faces (2,400 images) and evaluated using 80 faces (640 images) to produce an average classification accuracy of 95.5 and 92.8% using mouth and eyebrows, respectively. The simulation results of testing the data set with LibSVM tool based on sequential minimal optimisation (SMO) algorithm and iterative single data algorithm (ISDA) proves that ISDA is faster than other SMO-based algorithms while delivering the same generalisation results.

Keywords: cross validation; feature extraction; Lagrange multiplier method; LoG; Laplacian of Gaussian; Lucas–Kanade algorithm; SVM; support vector machines; facial expressions; face recognition; optical flow; motion flow; neighbouring images; video sequences; classification; facial features; feature recognition.

DOI: 10.1504/IJISTA.2009.027112

International Journal of Intelligent Systems Technologies and Applications, 2009 Vol.7 No.3, pp.316 - 346

Published online: 15 Jul 2009 *

Full-text access for editors Access for subscribers Purchase this article Comment on this article