Title: Semantic indexing of multimedia content using textual and visual information

Authors: Abdesalam Amrane; Hakima Mellah; Rachid Aliradi; Youssef Amghar

Addresses: Research Center on Scientific and Technical Information (CERIST), Ben Aknoun, Algiers, Algeria ' Research Center on Scientific and Technical Information (CERIST), Ben Aknoun, Algiers, Algeria ' Research Center on Scientific and Technical Information (CERIST), Ben Aknoun, Algiers, Algeria ' University of Lyon, CNRS, INSA-Lyon, LIRIS, UMR5205, F-69621, France

Abstract: The challenge in multimedia information retrieval remains in the indexing process, an active search area. There are three fundamental techniques for indexing multimedia content: using textual information, using low-level information and combining different information extracted from multimedia. Each approach has its advantages and disadvantages as well to improve multimedia retrieval systems. The recent works are oriented towards multimodal approaches. In this paper, we propose an approach that combines the surrounding text with the information extracted from the visual content of multimedia and represented in the same repository in order to allow querying multimedia content based on keywords or concepts. Each word contained in queries or in description of multimedia is disambiguated using the WordNet ontology in order to define its semantic concept. Support vector machines (SVMs) are used for image classification in one of the defined semantic concept based on SIFT (scale invariant feature transform) descriptors.

Keywords: multimedia retrieval; information retrieval; automatic annotation; semantic representation; multimodal image representation; SVM; support vector machine; SIFT; scale invariant feature transform; textual querying; semantic indexing; textual information; visual information; image classification.

DOI: 10.1504/IJAMC.2014.060496

International Journal of Advanced Media and Communication, 2014 Vol.5 No.2/3, pp.182 - 194

Received: 13 Mar 2013
Accepted: 03 Sep 2013
Published online: 17 Apr 2014 *

Full-text access for editors Full-text access for subscribers Purchase this article Comment on this article