A feature-level attention-based deep neural network model for sentence embedding
by Amal Bouraoui; Salma Jamoussi; Abdelmajid Ben Hamadou
International Journal of Intelligent Systems Technologies and Applications (IJISTA), Vol. 20, No. 5, 2022

Abstract: Building a model to represent the semantic of a sentence is crucial for going beyond a sequence of words to a more abstract yet relevant representation. The relevance of such models is ubiquitous in various natural language processing tasks. Following this context, we capitalise on deep learning for proposing a new model-based attention mechanism. Our contribution aims at embedding the sentence meaning in an unsupervised iterative way. The word and the sentence embeddings therefore influence each other. Our model is inspired from the recursive auto-encoders. We coupled our model with a novel attention mechanism computed at the feature-level. This mechanism aims to increase representation power by focusing on important features of words within a sentence to refine the constructed meaning representation of this sentence. To highlight our newly proposed contribution, we carried out an exhaustive experimental study for evaluating the quality of the learned representations on semantic similarity task. The obtained results demonstrate the faithfulness of our learned semantic representations.

Online publication date: Fri, 16-Sep-2022

The full text of this article is only available to individual subscribers or to users at subscribing institutions.

 
Existing subscribers:
Go to Inderscience Online Journals to access the Full Text of this article.

Pay per view:
If you are not a subscriber and you just want to read the full contents of this article, buy online access here.

Complimentary Subscribers, Editors or Members of the Editorial Board of the International Journal of Intelligent Systems Technologies and Applications (IJISTA):
Login with your Inderscience username and password:

Username:        Password:          Forgotten your password?

Want to subscribe?
A subscription gives you complete access to all articles in the current issue, as well as to all articles in the previous three years (where applicable). See our Orders page to subscribe.

If you still need assistance, please email subs@inderscience.com