Title: Learning context-dependent word embeddings based on dependency parsing
Authors: Ke Yan; Jie Chen; Wenhao Zhu; Xin Jin; Guannan Hu
Addresses: School of Computer Engineering and Science, Shanghai University, 99 Shangda Road, BaoShan District, Shanghai 20444, China ' School of Computer Engineering and Science, Shanghai University, 99 Shangda Road, BaoShan District, Shanghai 20444, China ' School of Computer Engineering and Science, Shanghai University, 99 Shangda Road, BaoShan District, Shanghai 20444, China ' School of Computer Engineering and Science, Shanghai University, 99 Shangda Road, BaoShan District, Shanghai 20444, China ' School of Computer Engineering and Science, Shanghai University, 99 Shangda Road, BaoShan District, Shanghai 20444, China
Abstract: Word embeddings constitute the basic methods of text representation. Whether they are the inputs to a machine learning algorithm or the features used in a natural language processing application, embeddings have proven helpful in solving various text processing tasks. In natural language texts, contextual information exerts a crucial influence on the semantics of word representations. In current research, most training models are based on shallow textual information and do not fully exploit deep relationships in sentences. To overcome this problem, this paper proposes the dependency-based continuous bag-of-words model which integrates the dependency relationships between words and sentences into the context with weights, thereby increasing the influence of specific contextual information on the prediction of target words. This method increases the abundancy of word context information and enhances the semantics of word embeddings. The experimental results show that the proposed method highlights semantic relations and improves the performance of word representations.
Keywords: word embedding; context-dependent; dependency parsing; semantics.
DOI: 10.1504/IJITM.2020.110241
International Journal of Information Technology and Management, 2020 Vol.19 No.4, pp.334 - 346
Received: 18 Jul 2018
Accepted: 26 Mar 2019
Published online: 12 Oct 2020 *