Article: Violin fingering teaching algorithm based on augmented reality and motion capture technology Journal: International Journal of Information and Communication Technology (IJICT) 2025 Vol.26 No.26 pp.17 - 32 Abstract: Violin fingering techniques change rapidly, making it difficult to correct incorrect fingering in real time during instruction. To address this issue, this paper first employs a multi-head attention mechanism (MAM) and multi-scale dilated convolutional neural networks (DCNN) for hand fingering motion capture. Since hand movement occurs during performance, an augmented reality (AR)-based hand pose estimation module is designed. The pose from orthography and scaling with iterations (POSIT) algorithm, optimised using the Gauss-Newton method, is used to estimate relatively precise camera-based fingering poses. Finally, cosine similarity is used to compare virtual and real finger techniques, and corrections are made based on the features of the target finger technique to improve teaching effectiveness. Experimental results show that the proposed algorithm achieves a correction accuracy rate 3.66%-13.13% higher than the baseline algorithm, laying a foundation for improving violin finger technique instruction. Inderscience Publishers - linking academia, business and industry through research

Title: Violin fingering teaching algorithm based on augmented reality and motion capture technology

Authors: Mengyingyi Lei

Addresses: Xi'an Peihua University, Xi'an 710125, China

Abstract: Violin fingering techniques change rapidly, making it difficult to correct incorrect fingering in real time during instruction. To address this issue, this paper first employs a multi-head attention mechanism (MAM) and multi-scale dilated convolutional neural networks (DCNN) for hand fingering motion capture. Since hand movement occurs during performance, an augmented reality (AR)-based hand pose estimation module is designed. The pose from orthography and scaling with iterations (POSIT) algorithm, optimised using the Gauss-Newton method, is used to estimate relatively precise camera-based fingering poses. Finally, cosine similarity is used to compare virtual and real finger techniques, and corrections are made based on the features of the target finger technique to improve teaching effectiveness. Experimental results show that the proposed algorithm achieves a correction accuracy rate 3.66%-13.13% higher than the baseline algorithm, laying a foundation for improving violin finger technique instruction.

Keywords: violin fingering instruction; motion capture; augmented reality; multi-scale dilated convolution; POSIT algorithm.

DOI: 10.1504/IJICT.2025.147465

International Journal of Information and Communication Technology, 2025 Vol.26 No.26, pp.17 - 32

Received: 06 May 2025
Accepted: 23 May 2025
Published online: 16 Jul 2025 *

Title: Violin fingering teaching algorithm based on augmented reality and motion capture technology

Keep up-to-date