Visual relationship extraction in images and a semantic interpretation with ontologies
by Adel Zga; Brahim Nini
International Journal of Intelligent Information and Database Systems (IJIIDS), Vol. 15, No. 2, 2022

Abstract: Nowadays, three challenges should be considered in order to build a strong model that is used to extract and semantically interpret the relationship between objects in images namely: long-tail problem, large intra-class divergence, and the semantic dependency or semantic gap. In order to overcome those challenges, we propose three main contributions: 1) an ontological semantic model to filter false negatives/positives using a statistical ranking module; 2) a combination of semantic ontological module and visual relationship module that both takes as input the results of the statistical ranking module and produces as output classification of < human - predicate - object >; 3) a semantic model for the visual relationship module that ranks each prediction of relation classes by transferring the spatial relationship onto a high dimension spatial feature. We use HCVRD that highlights two important practical problems, the long-tail distribution issue, and the zero-shot problem. The experimental results on the HCVRD dataset demonstrate the superior performance of the proposed approach.

Online publication date: Thu, 07-Apr-2022

The full text of this article is only available to individual subscribers or to users at subscribing institutions.

 
Existing subscribers:
Go to Inderscience Online Journals to access the Full Text of this article.

Pay per view:
If you are not a subscriber and you just want to read the full contents of this article, buy online access here.

Complimentary Subscribers, Editors or Members of the Editorial Board of the International Journal of Intelligent Information and Database Systems (IJIIDS):
Login with your Inderscience username and password:

Username:        Password:          Forgotten your password?

Want to subscribe?
A subscription gives you complete access to all articles in the current issue, as well as to all articles in the previous three years (where applicable). See our Orders page to subscribe.

If you still need assistance, please email subs@inderscience.com