This is the first part of a series of blog posts on rating systems evolution. We will drill down to their core, explore their idiosyncrasies and return enlightened. Finally we will explain why free text ratings are so insightful and why they should be considered as a valid, viable and valuable - triple v bonus - alternative. Here we will focus on the most popular feedback mechanism to date, the star rating system.

From small online businesses to e-commerce giants like Amazon and eBay star rating systems are used in order to measure customer satisfaction and preferences. Truth be told star rating systems are the cornerstone of recommendation systems in pretty much every web industry. 

The reasons behind its success are quite apparent and dare to say no surprise to anyone. Whether looking for a product in an e-commerce website or struggle to get a good movie recommendation, star ratings fuel the decision making of every successful recommender system.

People enjoy using star ratings, machines can easily store and process them, but at the end of the day do star ratings work for either one of them?

With a series of blog posts we aim to drill down and weigh the pros and cons of the most popular rating system, wander around its core philosophy and emerge enlightened offering our take on more informative alternatives aka free text reviews.

Fistly we will briefly list the key features that paved the way for star system’s dominance around the web.

• Simplicity - one click is enough for a user to leave a rating.
• They provide a clear and fast way of getting both users’ feedback and aggregated results for the entire user base.
• They are granular enough for users and machines.
• Offers a clear definition of the best choice amongst products assuming ratings are well calibrated.

Considering the pros of the star rating system, it is obvious why it has been with us for ages and why it will stick around for some time in the future. On the other hand we should not turn a blind eye on its drawbacks.

• Star rating systems may be quick but not as insightful.
• Ambiguity/Uncertainty of scale - different scale implementation from website to website can boggle users and rating comparisons.
• Studies show that star rating system ratings are usually skewed - the average rating a 5-star system generates is 4.3, usually regardless of the object being rated.
• Suffers from a lack of a negative measure.

At this point we will elaborate a bit in order to understand its strengths and weaknesses.

On the plus side of star rating systems, key element is simplicity. Additionally star-rating systems provide a clear view of which is the best option for a variety of products, and ratings derived from these rating systems are easily digestible for humans and useful for machines - whether data will be used for simple analyses or more advanced AI recommendation models.

Is it possible though to grasp users’ preferences from simplicity and simplicity alone? Or how I like to picture the elephant in the room:

“What is the factor that separates a six-star rating from a 7-star one on a ten-star scale? How my rating criteria are compared to yours?“

Ambiguity/Uncertainty of scale is a serious consideration when considering star rating system success. If altering rating granularity will seemingly provide an easy solution to this problem, think again.

Three-star scales tend to solve some of the issues introduced by the five-star scale. Although as we will see below this is somehow a mixed blessing. Users tend to be less hesitant as the range of choices offered is reduced, introducing that way a bigger border between ratings. On the other hand three-star ratings are often less accurate when compared to five-star ones. Last but not least ten-star rating systems can, if not applied with caution, confuse rather than clarify one’s ratings.

Moreover, websites of the same industry like movie rating, for instance IMDB and Rotten Tomatoes definitely share audiences but use different rating systems, ten-star and percentage accordingly. Users must surely be annoyed while attempting to convert a seven-star IMDB rating to the corresponding Rotten Tomatoes percentage (quick hint: it is not 70%)

According to Wall Street Journal article On the Internet, Everyone's a Critic But They're Not Very Critical, the average rating a 5-star system generates is 4.3, no matter the object being rated. Lastly due to the fact that people are inherently polite and the vast majority have correlated online experiences pack mentality or alternatively “monopoly populism” kicks in as expressed in How Meaningful Are User Ratings? (This Article = 4.5 Stars!). This being said an average of 4.3 point of Amazon’s ratings is observed as well as Youtube’s shift from star ratings to a binary like/dislike rating system as described in Five Stars Dominate Ratings.

At this point anyone would consider whether the answer to all aforementioned concerns could be a binary, i.e. like/dislike, rating system. Evidence of success of binary rating systems are lying all around; Youtube, Reddit paradigms or the like versus dislike feature that characterizes most of the social media.

This post is the first of a series conducting popular rating systems analysis and finally displaying how Language based ratings in conjunction with Sentiment Analysis have what it takes to provide an effective and sophisticated alternative that incorporates getting a clean direct feedback from the user as well as extracting useful information about his experience.