Takeaways:
- Product ratings have become an essential part of the shopping process
- A new study by the UBC Sauder School of Business shows that when consumers look at star ratings, they tend to “round up”. This is because of a phenomenon known as the “visual completion effect.”
- Conversely, when ratings are in the form of numbers, they round down. This is the result of the left digit effect.
- How the stars get filled in also makes a difference, and can lead to consumers to pay more
- Studies have shown that a 0.2 difference in a rating (i.e. 4.5 versus 4.7) can lead to 200 percent higher sales
- The study is the first of its kind to test the accuracy of the formats independently of each other
Product ratings have become an essential part of the shopping process for consumers around the globe. But according to a fascinating new study from the UBC Sauder School of Business, whether they’re looking at star ratings or numerals, consumers are getting them all wrong — and the difference for sellers could be in the billions.
Retailers rely heavily on ratings to give consumers a sense of what they’re buying and help drive sales. Some use star ratings, while others use numerals; some use a combination of the two. So which format most accurately reflects what buyers have to say? The answer isn’t so simple.
For the study, titled Overestimating Stars, Underestimating Numbers: The Hidden Impact of Rating Formats, the researchers performed a series of experiments. In one, participants were asked to plot fractional numbers on an unmarked line; another tested how people visually process partially filled stars. In another experiment, respondents were given half ratings (1.5, 2.5, etc.) and asked how they might round up or down; another tested whether one format was more memorable than the other.
The researchers found that consumers tend to overestimate ratings that involve stars and underestimate ratings that rely on numerals — in particular when they involve fractional values rather than whole numbers.
UBC Sauder Assistant Professor and study co-author Dr. Deepak Sirwani says people overestimate star ratings because of a phenomenon called “the visual completion effect” — that is, when their visual system sees an incomplete image like a star that’s only half full, they instinctively fill in the remainder. As a result, when they see half-full stars, consumers tend to round up — so they perceive a 3.5 rating as closer to 4. They then partially correct the error, but not enough to land them all the way back at 3.5.
Conversely, when people see fractional numbers like 3.5, the “left-digit effect” leads consumers to round down — so they perceive a 3.5 rating as closer to 3. (This is the same phenomenon that leads retailers to price items at $3.99 rather than $4.00, explains Dr. Sirwani.)
“With fractional numbers like 3.99 we fixate on that left digit, which is three, and that’s why we start registering the number as 3. We make an upward correction, but it’s not sufficient,” says Dr. Sirwani.
“We evolved seeing three trees, three apples and three stones, so we can't immediately make intuitive sense of what 3.5 is. So the brain searches for a round number to base its estimation on, and for pictures the number that's available is rounded up. For numerals, the number that's available is rounded down,” Dr. Sirwani explains. “And why are these round numbers systematically different? It’s because for pictures, we complete the incomplete picture. For numbers, the left digit is available, and we focus on that left digit.”
Products whose reviews land on whole numbers — 3 or 4, for example — do not experience a bump up or down.
These processes happen unconsciously and in a matter of nanoseconds, adds Dr. Sirwani, so even people who understand how they work experience the same effect.
Interestingly, the way the stars are filled in also makes a difference. Businesses either use the proportion-of-area method — so 0.25 of a star is represented by filling in 25 per cent of the star — or the proportion-of-width method in which 0.25 of a star is represented by a star with 25 per cent of its width filled. The study showed that consumers were willing to pay significantly more for a product with a rating ending in 0.75 when that rating was represented with proportion-of-width stars rather than proportion-of-area stars or numerals.
Previous studies have compared the perceptions of various formats — star ratings versus bar ratings, for example — but the study, co-authored by Dr. Srishti Kumar of the Indian Institute of Management Ahmedabad and Dr. Manoj Thomas of Cornell University, is the first of its kind to remove that relativity and directly test which format most accurately communicates consumer ratings. It’s also the first to show that the way stars are filled in makes a difference. (The effect holds true for other shapes such as circles and squares, too, and is not specific to stars).
Dr. Sirwani says people’s incorrect perceptions can have a big impact, because when consumers underestimate or overestimate a product’s value, it can dramatically affect their purchase patterns. In fact, other research has shown that a tiny 0.2 difference in rating — for example, from 4.5 to 4.7 — can mean a 200 per cent increase in sales.
“Online shopping alone is a huge market, trillions of dollars around the globe, that relies on rating systems,” says Dr. Sirwani. “And the size of the effect that we calculate in our paper is about 0.15 or 0.17 — so it’s touching that 0.2 number. That means sales can triple.”
The fact that there is no standardization among retailers only adds to the confusion, he adds. For example, if you’re booking a getaway online, and the rating is 4.5 using stars on one site, and the number 4.5 on another, you might be more likely to choose the site that uses stars. Similarly, if you’re looking at a product that has a 4-star rating versus a pricier one that has a 4.5-star rating, you might go for the 4.5-star item because our brains perceive the rating as closer to 4.7.
So which format should platforms use in order to accurately represent ratings? Some, including online retail behemoth Amazon, are shifting to a combination of stars and numbers, although Dr. Sirwani says we process those the same way we process numeral-only ratings.
Dr. Sirwani notes that when using stars, the most accurate representation is a “visually complete star” — that is, a star that only shows the filled-in part of the star and not the empty remainder. So a 3.5-star rating would literally show 3.5 stars, without any additional space for the brain to fill in. Still, when it comes to how best to represent ratings, there is no clear winner.
“Whether it's pictorial formats or numbers, ratings matter, and platforms should probably think about it more carefully than they have,” says Dr. Sirwani. Consumers who overestimate a rating can end up disappointed in a product, he warns, while those who underestimate may not buy the product at all. “Stars are overestimated and numbers are underestimated, so based on their objectives, platforms should choose very wisely.”
Interview language: English
