Title: Convolutional recurrent neural network with attention for Vietnamese speech to text problem in the operating room

Authors: Trinh Tan Dat; Le Tran Anh Dang; Vu Ngoc Thanh Sang; Le Nhi Lam Thuy; Pham The Bao

Addresses: Information Science Faculty, Sai Gon University, HCM City, Vietnam ' Faculty of Electrical and Electronics Engineering, University of Technology, HCM City, Vietnam ' Information Science Faculty, Sai Gon University, HCM City, Vietnam ' Information Science Faculty, Sai Gon University, HCM City, Vietnam ' Information Science Faculty, Sai Gon University, HCM City, Vietnam

Abstract: We introduce automatic Vietnamese speech recognition (ASR) system for converting Vietnamese speech to text on a real operating room ambient noise recorded during liver surgery. First, we propose applying a combination between convolutional neural network (CNN) and bidirectional long short-term memory (BLSTM) for investigating local speech feature learning, sequence modelling, and transcription for speech recognition. We also extend the CNN-LSTM framework with an attention mechanism to decode the frames into a sequence of words. The CNN, LSTM and attention models are combining into a unified architecture. In addition, we combine connectionist temporal classification (CTC) and attention's loss functions in training phase. The length of the output label sequence from CTC is applied to the attention-based decoder predictions to make the final label sequence. This process helps to decrease irregular alignments and make speedup of the label sequence estimation during training and inference, instead of only relying on the data-driven attention-based encoder-decoder for estimating the label sequence in long sentences. The proposed system is evaluated using a real operating room database. The results show that our method significantly enhances the performance of the ASR system. We find that our approach provides a 13.05% in WER and outperforms standard methods.

Keywords: Vietnamese speech recognition; convolutional neural network; CNN; bidirectional long short-term memory; BLSTM; attention; operating room.

DOI: 10.1504/IJIIDS.2021.116476

International Journal of Intelligent Information and Database Systems, 2021 Vol.14 No.3, pp.294 - 314

Received: 27 Jul 2020
Accepted: 13 Oct 2020

Published online: 26 Jul 2021 *

Full-text access for editors Full-text access for subscribers Purchase this article Comment on this article