Title: Are numeric ratings true representations of reviews? A study of inconsistency between reviews and ratings

Authors: Bidyut Hazarika; Kuanchin Chen; Muhammad Razi

Addresses: Business Information Systems Department, Haworth College of Business, Western Michigan University, Kalamazoo, Michigan, USA ' Business Information Systems Department, Haworth College of Business, Western Michigan University, Kalamazoo, Michigan, USA ' Business Information Systems Department, Haworth College of Business, Western Michigan University, Kalamazoo, Michigan, USA

Abstract: Consistency between review messages and ratings has been an implicit assumption of many product review studies. However, such an assumption has never been critically examined. Our study investigates the consistency between app reviews and their corresponding ratings to uncover insights on the nature of inconsistency and how that inconsistency relates to fee structure (paid or free) and product category. To examine this inconsistency, we collected data from the iTunes store. We used sentiment mining to measure the discrepancy between the review sentiment and the ratings of the app. Our findings show that the assumption of consistency between product ratings and product reviews do not always hold. Findings support the existence of inconsistency, but not for all product categories. As the groups of apps differ in their consistency patterns, future researchers are recommended not to lump all apps together. Doing so, could obscure the distinctive difference among these apps, thereby offering less insight from the collected data.

Keywords: product ratings; product reviews; review sentiment; app versions; iTunes; app categories.

DOI: 10.1504/IJBIS.2021.118637

International Journal of Business Information Systems, 2021 Vol.38 No.1, pp.85 - 106

Received: 12 Jul 2018
Accepted: 25 Apr 2019
Published online: 30 Oct 2021 *

Full-text access for editors Full-text access for subscribers Purchase this article Comment on this article