Article: Learning context-dependent word embeddings based on dependency parsing Journal: International Journal of Information Technology and Management (IJITM) 2020 Vol.19 No.4 pp.334 - 346 Abstract: Word embeddings constitute the basic methods of text representation. Whether they are the inputs to a machine learning algorithm or the features used in a natural language processing application, embeddings have proven helpful in solving various text processing tasks. In natural language texts, contextual information exerts a crucial influence on the semantics of word representations. In current research, most training models are based on shallow textual information and do not fully exploit deep relationships in sentences. To overcome this problem, this paper proposes the dependency-based continuous bag-of-words model which integrates the dependency relationships between words and sentences into the context with weights, thereby increasing the influence of specific contextual information on the prediction of target words. This method increases the abundancy of word context information and enhances the semantics of word embeddings. The experimental results show that the proposed method highlights semantic relations and improves the performance of word representations. Inderscience Publishers - linking academia, business and industry through research

Title: Learning context-dependent word embeddings based on dependency parsing

Authors: Ke Yan; Jie Chen; Wenhao Zhu; Xin Jin; Guannan Hu

Addresses: School of Computer Engineering and Science, Shanghai University, 99 Shangda Road, BaoShan District, Shanghai 20444, China ' School of Computer Engineering and Science, Shanghai University, 99 Shangda Road, BaoShan District, Shanghai 20444, China ' School of Computer Engineering and Science, Shanghai University, 99 Shangda Road, BaoShan District, Shanghai 20444, China ' School of Computer Engineering and Science, Shanghai University, 99 Shangda Road, BaoShan District, Shanghai 20444, China ' School of Computer Engineering and Science, Shanghai University, 99 Shangda Road, BaoShan District, Shanghai 20444, China

Abstract: Word embeddings constitute the basic methods of text representation. Whether they are the inputs to a machine learning algorithm or the features used in a natural language processing application, embeddings have proven helpful in solving various text processing tasks. In natural language texts, contextual information exerts a crucial influence on the semantics of word representations. In current research, most training models are based on shallow textual information and do not fully exploit deep relationships in sentences. To overcome this problem, this paper proposes the dependency-based continuous bag-of-words model which integrates the dependency relationships between words and sentences into the context with weights, thereby increasing the influence of specific contextual information on the prediction of target words. This method increases the abundancy of word context information and enhances the semantics of word embeddings. The experimental results show that the proposed method highlights semantic relations and improves the performance of word representations.

Keywords: word embedding; context-dependent; dependency parsing; semantics.

DOI: 10.1504/IJITM.2020.110241

International Journal of Information Technology and Management, 2020 Vol.19 No.4, pp.334 - 346

Received: 18 Jul 2018
Accepted: 26 Mar 2019
Published online: 12 Oct 2020 *

Full-text access for editors Full-text access for subscribers Purchase this article Comment on this article

Title: Learning context-dependent word embeddings based on dependency parsing

Keep up-to-date