Title: Multi-modal user experience evaluation on in-vehicle HMI systems using eye-tracking, facial expression, and finger-tracking for the smart cockpit

Authors: Wenbo Li; Yingzhang Wu; Guanzhong Zeng; Fan Ren; Mingqing Tang; Huafei Xiao; Yujing Liu; Gang Guo

Addresses: School of Automotive Engineering, Chongqing University, Chongqing, 400044, China ' School of Automotive Engineering, Chongqing University, Chongqing, 400044, China ' School of Automotive Engineering, Chongqing University, Chongqing, 400044, China ' Changan Automobile Group, Changan Automobile, Chongqing, 400023, China ' Shanghai Volkswagen Automotive Co., Ltd., Shanghai, 201805, China ' School of Automotive Engineering, Chongqing University, Chongqing, 400044, China ' School of Automotive Engineering, Chongqing University, Chongqing, 400044, China ' School of Automotive Engineering, Chongqing University, Chongqing, 400044, China

Abstract: The trend toward intelligent connected vehicles (ICVs) led to numerous more novel and more natural human-vehicle relationships, which will bring about tremendous changes in smart cockpit functions and interaction methods. However, most in-vehicle human-machine interaction (HMI) systems focus on adding more functions, while few of them focus on the user experience (UX) of the system. This study presents an evaluation method of UX based on eye-tracking, finger movement tracking, and facial expression, the study also proposed a pleasantness prediction based on multi-layer perception (MLP) algorithm using multi-modal data. Through the UX experiment on two in-vehicle HMI systems, the study verified that the proposed evaluation method can be objective and efficient to evaluate the in-vehicle HMI system. Based on the MLP algorithm, the study trained the pleasantness prediction model using multi-modal data. Besides, we collected new data of the third in-vehicle HMI system to test the trained model and presented excellent test results.

Keywords: HMI; human-machine interaction; user experience; driver emotion; behaviour analysis; smart cockpit.

DOI: 10.1504/IJVP.2022.125931

International Journal of Vehicle Performance, 2022 Vol.8 No.4, pp.429 - 449

Received: 29 Jan 2021
Accepted: 12 Aug 2021
Published online: 04 Oct 2022 *

Full-text access for editors Full-text access for subscribers Purchase this article Comment on this article