Article: Computer vision simulation with multimodal data for real-time user interaction in industrial design Journal: International Journal of Information and Communication Technology (IJICT) 2026 Vol.27 No.55 pp.43 - 77 Abstract: To solve the problems of modal heterogeneity, temporal asynchrony and cognitive adaptation imbalance in multimodal real-time interaction, a CLT-driven multi-modal real-time fusion architecture was proposed. Experimental verification on HoloAssist dataset shows that the interactive intention prediction accuracy of the proposed architecture reaches 95.2% ± 1.3%, which is 3.5 percentage points higher than that of AlignMamba model. The end-to-end delay is 0.18 s ± 0.02 s, and the alignment delay is as low as 0.028 s. The subjective score of cognitive load was 3.2 ± 0.8, which was significantly better than the baseline model. Ablation experiments confirm that each core module is crucial to performance improvement, and the model has excellent robustness in scenarios with modal loss and noise interference. This research provides support for the implementation of real-time multimodal interaction technology. Inderscience Publishers - linking academia, business and industry through research

Title: Computer vision simulation with multimodal data for real-time user interaction in industrial design

Authors: Jing Li; Hui Yuan

Addresses: School of Information Engineering, Shaanxi Polytechnic University, Xianyang, 712000, China ' School of Information Engineering, Shaanxi Polytechnic University, Xianyang, 712000, China

Abstract: To solve the problems of modal heterogeneity, temporal asynchrony and cognitive adaptation imbalance in multimodal real-time interaction, a CLT-driven multi-modal real-time fusion architecture was proposed. Experimental verification on HoloAssist dataset shows that the interactive intention prediction accuracy of the proposed architecture reaches 95.2% ± 1.3%, which is 3.5 percentage points higher than that of AlignMamba model. The end-to-end delay is 0.18 s ± 0.02 s, and the alignment delay is as low as 0.028 s. The subjective score of cognitive load was 3.2 ± 0.8, which was significantly better than the baseline model. Ablation experiments confirm that each core module is crucial to performance improvement, and the model has excellent robustness in scenarios with modal loss and noise interference. This research provides support for the implementation of real-time multimodal interaction technology.

Keywords: multi-modal fusion; cognitive load theory; optimal transmission; real-time human-computer interaction; adaptive weight.

DOI: 10.1504/IJICT.2026.153796

International Journal of Information and Communication Technology, 2026 Vol.27 No.55, pp.43 - 77

Received: 21 Jan 2026
Accepted: 27 Feb 2026
Published online: 26 May 2026 *

Title: Computer vision simulation with multimodal data for real-time user interaction in industrial design

Keep up-to-date