Title: A lexicon-based term weighting scheme for emotion identification of tweets

Authors: S. Lovelyn Rose; R. Venkatesan; Girish Pasupathy; P. Swaradh

Addresses: Department of CSE, PSG College of Technology, Coimbatore, India ' Department of CSE, PSG College of Technology, Coimbatore, India ' Department of Windows Server and System Center (WSSC), Microsoft India Development Center, India ' Department of CSE, KMCT College of Engineering, Calicut, India

Abstract: Detecting emotions in tweets is a huge challenge due to its limited 140 characters and extensive use of twitter language with evolving terms and slangs. This paper uses various preprocessing techniques, forms a feature vector using lexicons and classifies tweets into Paul Ekman's basic emotions namely, happy, sad, anger, fear, disgust and surprise using machine learning. Preprocessing is done using the dictionaries available for emoticons, interjections and slangs and by handling punctuation marks and hashtags. The feature vector is created by combining words from the NRC Emotion lexicon, WordNet-Affect and online thesaurus. Feature vectors are assigned weight based on the presence of punctuations and negations in the feature and the tweets are classified using naive Bayes, SVM and random forests. The use of lexicon features and a novel weighting scheme has produced a considerable gain in terms of accuracy with random forest achieving maximum accuracy of almost 73%.

Keywords: emotion classification; Twitter; preprocessing; feature selection; dictionaries; lexicons; term weighting; random forest; SVM; naive Bayes.

DOI: 10.1504/IJDATS.2018.095216

International Journal of Data Analysis Techniques and Strategies, 2018 Vol.10 No.4, pp.369 - 380

Received: 15 Jul 2016
Accepted: 19 Jan 2017
Published online: 02 Oct 2018 *

Full-text access for editors Full-text access for subscribers Purchase this article Comment on this article