Article: A comprehensive review of deep learning for natural language processing Journal: International Journal of Data Mining, Modelling and Management (IJDMMM) 2022 Vol.14 No.2 pp.149 - 182 Abstract: Deep learning has attracted considerable attention across many natural language processing (NLP) domains. Deep learning models aim to learn embeddings of data with multiple levels of abstraction through multiple layers for either labelled structured input data or unlabelled unstructured input data. Currently, two research trends have emerged in building higher level embeddings. On one hand, a strong trend in deep learning leads towards increasingly powerful and complex models. On the other hand, multi-purpose sentence representation based on simple sums or averages of word vectors was recently shown to be effective. Furthermore, improving the performance of deep learning methods by attention mechanism has become a research hotspot in the last four years. In this paper, we seek to provide a comprehensive review of recent studies in building neural network (NN) embeddings that have been applied to NLP tasks. We provide a walk-through of deep learning evolution and a description of a variety of its architectures. We present and compare the performance of several deep learning models on standard datasets about different NLP tasks. We also present some deep learning challenges for natural language processing. Inderscience Publishers - linking academia, business and industry through research

Title: A comprehensive review of deep learning for natural language processing

Authors: Amal Bouraoui; Salma Jamoussi; Abdelmajid Ben Hamadou

Addresses: MIRACL Laboratory, Higher Institute of Computer Science and Multimedia, Sfax University, Technopole of Sfax: Av. Tunis Km 10 B.P. 242, Sfax – 3021, Tunisia ' MIRACL Laboratory, Higher Institute of Computer Science and Multimedia, Sfax University, Technopole of Sfax: Av. Tunis Km 10 B.P. 242, Sfax – 3021, Tunisia ' MIRACL Laboratory, Higher Institute of Computer Science and Multimedia, Sfax University, Technopole of Sfax: Av. Tunis Km 10 B.P. 242, Sfax – 3021, Tunisia

Abstract: Deep learning has attracted considerable attention across many natural language processing (NLP) domains. Deep learning models aim to learn embeddings of data with multiple levels of abstraction through multiple layers for either labelled structured input data or unlabelled unstructured input data. Currently, two research trends have emerged in building higher level embeddings. On one hand, a strong trend in deep learning leads towards increasingly powerful and complex models. On the other hand, multi-purpose sentence representation based on simple sums or averages of word vectors was recently shown to be effective. Furthermore, improving the performance of deep learning methods by attention mechanism has become a research hotspot in the last four years. In this paper, we seek to provide a comprehensive review of recent studies in building neural network (NN) embeddings that have been applied to NLP tasks. We provide a walk-through of deep learning evolution and a description of a variety of its architectures. We present and compare the performance of several deep learning models on standard datasets about different NLP tasks. We also present some deep learning challenges for natural language processing.

Keywords: deep learning; word embedding; sentence embedding; attention mechanism; compositional models; convolutional neural networks; CNNs; recurrent/recursive NNs; multi-purpose sentence embedding; natural language processing; NLP.

DOI: 10.1504/IJDMMM.2022.123356

International Journal of Data Mining, Modelling and Management, 2022 Vol.14 No.2, pp.149 - 182

Accepted: 22 Jun 2020
Published online: 11 Jun 2022 *

Full-text access for editors Full-text access for subscribers Purchase this article Comment on this article

Title: A comprehensive review of deep learning for natural language processing

Keep up-to-date