Title: FPST: a new term weighting algorithm for long running and short lived events

Authors: Y. Jahnavi; Y. Radhika

Addresses: Department of Computer Science and Engineering, GITAM Institute of Technology, GITAM University, Visakhapatnam – 530 045, Andhra Pradesh, India ' Department of Computer Science and Engineering, GITAM Institute of Technology, GITAM University, Visakhapatnam – 530 045, Andhra Pradesh, India

Abstract: Term weighting is a useful technique that extracts important features from textual documents, thereby providing a basis for different text mining approaches. While several term weighting algorithms based on their frequency and some other statistical measures have been proposed in the past, they are inaccurate in extracting hot terms from internet-based digitised news documents. To overcome that problem, this paper presents an innovative and effective term weighting algorithm by considering position, scattering and topicality along with frequency. Frequency considers the number of occurrences of a term; position focuses on where the term appears; scattering focuses on the distribution of a term in the entire document. Here topicality is calculated for both short lived events and long running events. Experimental evaluation shows that the proposed term weighting algorithm outperforms the existing term weighting algorithms.

Keywords: topic detection; topic tracking; TDT; vector space models; VSM; term weighting; short lived events; long running events; clustering; feature extraction; text documents; text mining; position; scattering; topicality; frequency.

DOI: 10.1504/IJDATS.2015.073863

International Journal of Data Analysis Techniques and Strategies, 2015 Vol.7 No.4, pp.366 - 383

Published online: 27 Dec 2015 *

Full-text access for editors Full-text access for subscribers Purchase this article Comment on this article