Title: Parallel weighted semantic fusion for cross-media retrieval

Authors: Nidhi Goel; Priti Sehgal

Addresses: Department of Computer Science, University of Delhi, Delhi, India ' Keshav Mahavidyalaya, University of Delhi, Delhi, India

Abstract: Efficiency and effectiveness of image retrieval (IR) system are based on the good interpretation and integration of multimodal information. Research results in the recent years show that combining the two modalities (text based and content based) even with simple fusion strategies alleviates the image retrieval results and also reduces the semantic gap. In this paper, we deploy parallel computing in weighted semantic similarity technique for IR using both text and content. This technique gives the weightage to the annotations associated with the query image based upon their semantic similarity with user's query and then establishes the semantics with database images. The semantic similarity has been measured using WordNet. For content matching, colour feature is extracted and is represented using fuzzy colour histogram (FCH). Furthermore, to fuse the two modalities, image reordering with late fusion strategy is used. Parallel processing is done at data level using the single program multiple data (SPMD) programming model that focuses on parallel execution of semantic similarity matching computations. The proposed approach shows that the parallel computing largely reduces the response time of the system. Whereas, semantics learned at an early stage helps in reducing the semantic gap. Experiments performed on two standard datasets reveal the good efficiency and effectiveness of the proposed approach.

Keywords: content-based image retrieval; CBIR; text-based image retrieval; TBIR; weighted semantic similarity; fuzzy colour histogram; FCH; WordNet; semantic gap; parallelisation; multimodal fusion; cross-media retrieval; late fusion; semantic fusion; parallel computing; content matching; colour features; feature extract.

DOI: 10.1504/IJCISTUDIES.2015.069832

International Journal of Computational Intelligence Studies, 2015 Vol.4 No.1, pp.50 - 71

Received: 24 Sep 2013
Accepted: 03 May 2014
Published online: 13 Jun 2015 *

Full-text access for editors Full-text access for subscribers Purchase this article Comment on this article