Two-stream attention network with local and non-local dependence for referring relationships Online publication date: Fri, 08-Apr-2022
by Jincheng Hu; Tao Wang
International Journal of Embedded Systems (IJES), Vol. 15, No. 1, 2022
Abstract: Referring relationships is a task aiming to retrieve entities in the image conditioned on the given structured relationship text. Previous works utilise convolution-network-base model to learn image features conditioned on the text and classify the pixels in the image for retrieval. However, convolution network concentrates on the local dependence in the image without considering correlations between pixels far apart. In this paper, a two-stream attention network (TSAN) is proposed to address the above issue. Firstly, cross-modal feature maps are initialised by adding text information to each pixel in the visual feature maps. Then, TSAN designs attention mechanism for the local and non-local dependence in the cross-modal features to obtain two probability maps. Finally, the network integrates the two probability maps to predict the localisation probability. Experiments are conducted on three datasets and the results demonstrate that the proposed TSAN achieves better performance compared with several powerful models.
Existing subscribers:
Go to Inderscience Online Journals to access the Full Text of this article.
If you are not a subscriber and you just want to read the full contents of this article, buy online access here.Complimentary Subscribers, Editors or Members of the Editorial Board of the International Journal of Embedded Systems (IJES):
Login with your Inderscience username and password:
Want to subscribe?
A subscription gives you complete access to all articles in the current issue, as well as to all articles in the previous three years (where applicable). See our Orders page to subscribe.
If you still need assistance, please email subs@inderscience.com
Our Blog 
Follow us on Twitter 
Visit us on Facebook