Fine-grained emotion recognition: fusion of physiological signals and facial expressions on spontaneous emotion corpus
by Feri Setiawan; Aria Ghora Prabono; Sunder Ali Khowaja; Wangsoo Kim; Kyoungsoo Park; Bernardo Nugroho Yahya; Seok-Lyong Lee; Jin Pyo Hong
International Journal of Ad Hoc and Ubiquitous Computing (IJAHUC), Vol. 35, No. 3, 2020

Abstract: The recognition of fine-grained emotions (i.e., happiness, sad, etc.) has shown its importance in a real-world implementation. The emotion recognition using physiological signals is a challenging task due to the precision of the labelled data while using facial expressions is less appropriate for the real environment. This work proposes a framework for fusing physiological signals and facial expressions modalities to improve classification performance. The feature-level fusion (FLF) and decision-level fusion (DLF) techniques are explored in this work to recognise seven fine-grained emotions. The performance of the proposed framework is evaluated using 34 subjects' data. Our result shows that the fusion of the multiple modalities could improve the overall accuracy compared to the unimodal system by 11.66% and 13.63% for facial expression and physiological signals, respectively. Our work achieved a 73.23% accuracy for seven emotions which is considerable accuracy for the spontaneous emotion corpus.

Online publication date: Thu, 29-Oct-2020

The full text of this article is only available to individual subscribers or to users at subscribing institutions.

 
Existing subscribers:
Go to Inderscience Online Journals to access the Full Text of this article.

Pay per view:
If you are not a subscriber and you just want to read the full contents of this article, buy online access here.

Complimentary Subscribers, Editors or Members of the Editorial Board of the International Journal of Ad Hoc and Ubiquitous Computing (IJAHUC):
Login with your Inderscience username and password:

Username:        Password:          Forgotten your password?

Want to subscribe?
A subscription gives you complete access to all articles in the current issue, as well as to all articles in the previous three years (where applicable). See our Orders page to subscribe.

If you still need assistance, please email subs@inderscience.com