Title: Learning context-dependent word embeddings based on dependency parsing

Authors: Ke Yan; Jie Chen; Wenhao Zhu; Xin Jin; Guannan Hu

Addresses: School of Computer Engineering and Science, Shanghai University, 99 Shangda Road, BaoShan District, Shanghai 20444, China ' School of Computer Engineering and Science, Shanghai University, 99 Shangda Road, BaoShan District, Shanghai 20444, China ' School of Computer Engineering and Science, Shanghai University, 99 Shangda Road, BaoShan District, Shanghai 20444, China ' School of Computer Engineering and Science, Shanghai University, 99 Shangda Road, BaoShan District, Shanghai 20444, China ' School of Computer Engineering and Science, Shanghai University, 99 Shangda Road, BaoShan District, Shanghai 20444, China

Abstract: Word embeddings constitute the basic methods of text representation. Whether they are the inputs to a machine learning algorithm or the features used in a natural language processing application, embeddings have proven helpful in solving various text processing tasks. In natural language texts, contextual information exerts a crucial influence on the semantics of word representations. In current research, most training models are based on shallow textual information and do not fully exploit deep relationships in sentences. To overcome this problem, this paper proposes the dependency-based continuous bag-of-words model which integrates the dependency relationships between words and sentences into the context with weights, thereby increasing the influence of specific contextual information on the prediction of target words. This method increases the abundancy of word context information and enhances the semantics of word embeddings. The experimental results show that the proposed method highlights semantic relations and improves the performance of word representations.

Keywords: word embedding; context-dependent; dependency parsing; semantics.

DOI: 10.1504/IJITM.2020.10028805

International Journal of Information Technology and Management, 2020 Vol.19 No.4, pp.334 - 346

Received: 18 Jul 2018
Accepted: 26 Mar 2019

Published online: 22 Apr 2020 *

Full-text access for editors Access for subscribers Purchase this article Comment on this article