Title: A graph-based unsupervised N-gram filtration technique for automatic keyphrase extraction

Authors: Niraj Kumar; Kannan Srinathan; Vasudeva Varma

Addresses: IIIT-Hyderabad, Hyderabad-500032, India ' IIIT-Hyderabad, Hyderabad-500032, India ' IIIT-Hyderabad, Hyderabad-500032, India

Abstract: In this paper, we present a novel N-gram (N> = 1) filtration technique for keyphrase extraction. To filter the sophisticated candidate keyphrases (N-grams), we introduce the combined use of: 1) statistical feature (obtained by using weighted betweenness centrality scores of words, which is generally used to identify the border nodes/edges in community detection techniques); 2) co-location strength (calculated by using nearest neighbour Dbpedia texts). We also introduce the use of N-gram (N> = 1) graph, which reduces the bias effect of lower length N-grams in the ranking process and preserves the semantics of words (phraseness), based upon local context. To capture the theme of the document and to reduce the effect of noisy terms in the ranking process, we apply an information theoretic framework for key-player detection on the proposed N-gram graph. Our experimental results show that the devised system performs better than the current state-of-the-art unsupervised systems and comparable/better than supervised systems.

Keywords: keyphrase extraction; weighted betweenness centrality; N-gram graph; normalised pointwise mutual information; NPMI; key phrases; N-gram filtration; statistical features; co-location; semantics; document themes; information theory.

DOI: 10.1504/IJDMMM.2016.077198

International Journal of Data Mining, Modelling and Management, 2016 Vol.8 No.2, pp.124 - 143

Published online: 22 Jun 2016 *

Full-text access for editors Full-text access for subscribers Purchase this article Comment on this article