Title: Improving sentiment analysis using preprocessing techniques and lexical patterns

Authors: Stefano Cagnoni; Laura Ferrari; Paolo Fornacciari; Monica Mordonini; Laura Sani; Michele Tomaiuolo

Addresses: Department of Engineering and Architecture, University of Parma, Parco Area delle Scienze 181a, 43125 Parma, Italy ' Department of Engineering and Architecture, University of Parma, Parco Area delle Scienze 181a, 43125 Parma, Italy ' Department of Engineering and Architecture, University of Parma, Parco Area delle Scienze 181a, 43125 Parma, Italy ' Department of Engineering and Architecture, University of Parma, Parco Area delle Scienze 181a, 43125 Parma, Italy ' Department of Engineering and Architecture, University of Parma, Parco Area delle Scienze 181a, 43125 Parma, Italy ' Department of Engineering and Architecture, University of Parma, Parco Area delle Scienze 181a, 43125 Parma, Italy

Abstract: Sentiment analysis has recently gained considerable attention, since the classification of the emotional content of a text (online reviews, blog messages etc.) may have a relevant impact on market research, political science and many other fields. In this paper, we focus on the importance of the text preprocessing phase, proposing a new technique we termed lexical pattern-based feature weighting (LPFW) that allows one to improve sentence-level sentiment analysis by increasing the relevance of the features contained in particular lexical patterns. This approach has been evaluated on two sentiment classification datasets. We show that a systematic optimisation of the preprocessing filters is important for obtaining good classification accuracy. Also, we show that LPFW is effective in different application domains and with different training set sizes.

Keywords: sentiment analysis; natural language processing; POS tagging; feature weighting; word stemming; bag-of-words representation; tf-idf; Penn Treebank Tagset; support vector machines; naïve Bayes multinomial classifier.

DOI: 10.1504/IJDATS.2021.118022

International Journal of Data Analysis Techniques and Strategies, 2021 Vol.13 No.3, pp.171 - 185

Accepted: 01 Apr 2020
Published online: 08 Oct 2021 *

Full-text access for editors Full-text access for subscribers Purchase this article Comment on this article