Title: Facial expression recognition using local binary pattern and modified hidden Markov model

Authors: Mayur Rahul; Narendra Kohli; Rashi Agarwal

Addresses: Department of Computer Applications, University Institute of Engineering and Technology, Chhatrapati Shahu Ji Maharaj University, Kanpur, India ' Department of Computer Science and Engineering, Harcourt Butler Technical University, Kanpur, India ' Department of Information Technology, University Institute of Engineering and Technology, Chhatrapati Shahu Ji Maharaj University, Kanpur, India

Abstract: Facial expression is the non-verbal communication used to send and receive their inner emotions state. It plays a key role in person's behaviour and communication. Facial expression recognition can be applied in various areas like human-computer interaction, surveillance systems, medicines, home security and credit card verification. An efficient and robust face descriptor is very important in FER systems. This paper is able to analyse the local binary pattern (LBP) to represent the facial features in face images. LBP can be calculated in all eight directions of each pixel to obtain the binary coded number. Each expression is represented using binary code. We incorporated this LBP with our new modified HMM, which acts as a classification method. New modified HMM classifier is made up of two layers: upper layer and bottom layer. Bottom layer consists of atomic expressions and upper layer consists of combinations of atomic expressions. There are seven classes of known expression, i.e., anger, disgust, fear, joy, sadness, surprise, neutral are recognised with this approach. We have also tested our new framework using confusion matrix, ROC curve, recognition performance, processing time, error rates and found the overall accuracy of 85%.

Keywords: local binary pattern; LBP; machine learning; ROC curve; processing time; error rates; binary code; face descriptor.

DOI: 10.1504/IJAIP.2020.109523

International Journal of Advanced Intelligence Paradigms, 2020 Vol.17 No.3/4, pp.367 - 378

Received: 10 Jul 2018
Accepted: 15 Jul 2018
Published online: 11 Sep 2020 *

Full-text access for editors Full-text access for subscribers Purchase this article Comment on this article