Metacritic denies weighting system research

2 min read
16

Metacritic

We’re profoundly proud of our metacritic affiliation and as such out unbiased nature could be called into question when posting an article about the industry respected site. But since we’re not taking sides here I thought it was safe to do so.

A person by the name of Adams Greenwood has led some research on behalf of Full Sail University entitled ‘A Scientific Assessment of the Validity and Value of Metacritic’, in this research they attempted to place each publication in metacritic into a weighting band to see which publication Metacritic values more than others.

The entire list is available at gamasutra but crucially it’s not actually important as we are not listed in the outlets. Granted that’s because we’ve only been part of metacritic for 1 1/2 months and the research is based on the last 6 months.

Also metacritic has come out and stated the following in regards to the list

“There’s just one major problem with that: neither that site, nor the person giving the presentation, got those weights from us; rather, they are simply their best guesses based on research”

And followed up with

“We use far fewer tiers than listed in the article.

The disparity between tiers listed in the article is far more extreme than what we actually use on Metacritic. For example, they suggest that the highest-weighted publications have their scores counted six times as much as the lowest-weighted publications in our Metascore formula. That isn’t anywhere close to reality; our publication weights are much closer together and have much less of an impact on the score calculation.

Last but definitely not least: Our placement of publications in each tier differs from what is displayed in the article. The article overvalues some publications and undervalues others (while ignoring others altogether), sometimes comically so. (In addition, our weights are periodically adjusted as needed if, over time, a publication demonstrates an increase or decrease in overall quality.)”

So the researchers say one thing and metacritic says another… but really does anyone care?

Nothing is ever perfect and while some people may feel the metacritic score is slightly off kilter it would take a supreme leap of logic to believe that it’s entirely wrong. It’s a weighted average, if you can find me a score on metacritic that is patently false compared to all the critic scores then I’d give this argument some weight.

Until then this is nothing more than an Internet storm to keep media happy on quiet news days… and for that I am supremely grateful.

Last Updated: March 28, 2013

Gavin Mannion

I for one welcome our future robotic overlords

Check Also

Greta Gerwig’s Ladybird now the highest rated movie on Rotten Tomatoes

Debates on the greatest movie of all time (its Star Wars) are likely to always be inconclu…