A few years ago, I genuinely found value in user reviews. It was refreshing to find succinct reviews by consumers that didn’t necessarily focus on the nitty-gritty, but more on how fun games or movies were as consumers. In the years since though, I’ve increasingly found user reviews to be worthless, existing solely as some sort of extension of confirmation bias. People will heap mountains of undue praise on games they like, while others will review-bomb a game that has micro transactions as an act of digital protest. Worse is obvious bot reviews – done to increase positive public perception.
Both Metacritic and Steam have succumbed to this. Steam wants to do something about it – by making user reviews “more helpful.”
“Reviews should help paint a picture of what it’s like to play the game and how well the game has been enjoyed by the people who have already played it. A good review typically describes some of the factors that directly impact the experience of playing the game, which can include a wide variety of things like how well the matchmaking works, how buggy the game is, or whether the game represents a good value for the price.
While everyone that has played a game probably has opinions on how much they enjoyed the game or not, some people are better than others at writing thoughtful descriptions that are useful for other potential players. We think it’s fine for players to be able to indicate whether they liked a game or not, but not every review is useful to show in greater detail. A helpful review then is one that includes enough information to aid in your understanding of whether you are likely to find the game fun, or whether you should avoid the game and explore more alternatives.”
So how do they sort the helpful ones from the not so helpful ones? Valve says that while most people use the system as intended, there are many who effectively game user reviews – to whatever end.
“However, we found a small set of users on the far extreme that are clearly trying to accomplish something quite different from normal players, and are rating more than 10,000 reviews as helpful or unhelpful on a single game. This behavior is not only humanly impossible, but definitely not a thoughtful indication of how ‘helpful’ each of those reviews were. These users also tend to rate up just the negative reviews while rating down the positive reviews (or vice-versa) in an attempt to distort which reviews are shown by default.”
Because of how many reviews these users are rating, they each have a disproportionate amount of influence over the display of helpful reviews and cause certain reviews to appear more prominently than they should be”
They’re making two big changes to the user review system, which roll out in beta today.
- Firstly, our system will use a new method of calculating the helpfulness of each review, taking into account the users that are trying to manipulate the system. One way we’re doing that is by counting the helpful ratings on reviews differently for users that are far outside the norm. Ratings from users that follow normal patterns of rating will continue to be counted the same way that they have, whereas accounts that rate an excessive number of reviews on an individual game will see the weight of each individual rating count for less and less.
- Secondly, store pages will now show the default helpful positive and negative reviews in a similar proportion to that of the overall review score for the game. For example, if the game is reviewed positively by 80% of reviewers, then the ten reviews shown by default on the store page will be 80% positive, showing eight positive and two negative. This should keep the reviews shown on a game’s page from being so easily manipulated by a few determined players and should more accurately represent the overall sentiment of the people playing the game.
They’re actually great changes that I hope will paint a better picture of games’ worth than the current system. Hopefully, it can help put an end to ineffective review bombing.
Last Updated: November 21, 2017