On popular review aggregation site Metacritic, 1UP’s review score of an “A” for Gears of War 2 somehow translates into a “100”, MetaCritic’s highest possible ranking. I’m not sure how that alchemy works, but that score (plus a whole lot of positive reviews from other outlets) hasn’t stopped a group of rogue user reviewers from dropping the game’s user score significantly.
As of this morning just over 3,600 users had voted on the unreleased game, giving it an average user score of 3.4/10. This stands in stark contrast to the official review average of 94/100, which MetaCritic labels as “Universal acclaim.” The negative reaction from users was so extreme and unexpected that it warranted the following note from MetaCritic games editor Marc Doyle added to the bottom of the Gears of War 2 page:
“My advice for our faithful users is to focus your attention on the Metascore for this game and not the thousands of user votes, most of which have been submitted before said users have played the game. This is a gaming community, and if people want to stuff the ballot box, there’s not much I can do at this point. When we upgrade the registration requirements for participation on the site in the near future, this type of thing won’t happen. We’ll post the full legitimate user reviews upon the game’s release. As always, thanks for using the site.”
When Doyle was contacted for clarification by 1up news, he told them that the issue of unbalanced user reviews “hasn’t been a systematic problem” on the site. According to Doyle, it’s only really popped up recently and mostly for console-exclusive titles. Two other strong examples exist: Resistance 2 has a Metascore of 89 with a user score of 5.3/10, and Little Big Planet has a Metascore of 95 with a user score of 6.1/10. Doyle says the issue stems from the site’s foundation. User reviews were allowed to be entered before a game’s release because they “wanted people who had legitimately played the game ahead of its release to post them.” Stacked on top of that was a desire for an easy sign-up process. “The founders were really interested in not having people sign up for a really huge registration process just so they can participate on the site,” Doyle said, adding, “Obviously that’s been exploited.”
Doyle also explained exactly how MetaCritic will be looking to fix this problem through an enhanced sign-up process that will now require users to enter more information than just an e-mail address in order to get an account and vote on MetaCritic. “Even before the enhanced registration portion, we’re going to disallow voting on games before release,” Doyle said. Speaking to the relative importance of user reviews on the site, Doyle stated, “Our primary product is the critic score. That’s what we control; that’s what we can certify.” He added that the site may look at new ways to highlight user reviews in the future as they develop better means of certifying that the user reviews are legitimate. Until these fixes are implemented, though, MetaCritic readers will simply have to deal with the fanboy flame war affecting the user review scores.
Last Updated: November 5, 2008