Title: Detecting and ranking events in Twitter using diversity analysis

Authors: Daoud M. Daoud

Addresses: Department of Software Engineering, Princess Sumaya University for Technology, Amman, Jordan

Abstract: In Twitter and in other social media channels, detecting events is very important and has many applications. However, this task is very challenging because of the huge number of tweets that are posted every minute and the massive scale of the spamming activities. In this paper, we present an innovative approach for detecting events using data posted to Twitter. The proposed approach is based on the concept of user's attention by quantitatively modelling the diversity of hashtags using Shannon's index. Our method records the diversity values on an hourly basis time-series. Using statistical techniques, the method locates the intervals having diversity values that fall outside the range of forecasted ones (normal state). We also present the labelling and ranking techniques that are implemented in this research. Experimental results on a dataset consisting of 15 million Arabic tweets show that our proposed approach can effectively detect real-world events in Twitter.

Keywords: social media; event detection; diversity index; Twitter; Arabic; hashtags; time-series analysis; z-score; events labelling; events ranking.

DOI: 10.1504/IJBIDM.2018.088423

International Journal of Business Intelligence and Data Mining, 2018 Vol.13 No.1/2/3, pp.129 - 146

Received: 12 Oct 2016
Accepted: 09 Jan 2017
Published online: 07 Dec 2017 *

Full-text access for editors Full-text access for subscribers Purchase this article Comment on this article