Title: Topical document clustering: two-stage post processing technique
Authors: Poonam Goyal; N. Mehala; Divyansh Bhatia; Navneet Goyal
Addresses: Department of Computer Science, Birla Institute of Technology and Science, 333 031, Pilani, India ' Department of Computer Science, Presidency University, 560089, Bengaluru, India ' eBay Inc., San Jose, USA ' Department of Computer Science, Birla Institute of Technology and Science, 333 031, Pilani, India
Abstract: Clustering documents is an essential step in improving efficiency and effectiveness of information retrieval systems. We propose a two-phase split-merge (SM) algorithm, which can be applied to topical clusters obtained from existing query-context-aware document clustering algorithms, to produce soft topical document clusters. The SM is a post-processing technique which combines the advantages of document and feature-pivot topical document clustering approaches. The split phase splits the topical clusters by relating them to the topics obtained by disambiguating web search results, and converts them into homogeneous soft clusters. In the merge phase, similar clusters are merged by feature-pivot approach. The SM is tested on the outcome of two hierarchical query-context aware document clustering algorithms on different datasets including TREC session-track 2011 dataset. The obtained topical-clusters are also updated by an incremental approach with the progress in the data stream. The proposed algorithm improves the quality of clustering appreciably in all the experiments conducted.
Keywords: topical clustering; query clustering; query context; document clustering; incremental clustering; soft clustering.
DOI: 10.1504/IJDMMM.2018.092536
International Journal of Data Mining, Modelling and Management, 2018 Vol.10 No.2, pp.127 - 170
Received: 01 Sep 2016
Accepted: 08 Sep 2017
Published online: 24 Jun 2018 *