Home Gaming Metacritic denies weighting system research

Metacritic denies weighting system research

2 min read
15

Metacritic

We’re profoundly proud of our metacritic affiliation and as such out unbiased nature could be called into question when posting an article about the industry respected site. But since we’re not taking sides here I thought it was safe to do so.

A person by the name of Adams Greenwood has led some research on behalf of Full Sail University entitled ‘A Scientific Assessment of the Validity and Value of Metacritic’, in this research they attempted to place each publication in metacritic into a weighting band to see which publication Metacritic values more than others.

The entire list is available at gamasutra but crucially it’s not actually important as we are not listed in the outlets. Granted that’s because we’ve only been part of metacritic for 1 1/2 months and the research is based on the last 6 months.

Also metacritic has come out and stated the following in regards to the list

“There’s just one major problem with that: neither that site, nor the person giving the presentation, got those weights from us; rather, they are simply their best guesses based on research”

And followed up with

“We use far fewer tiers than listed in the article.

The disparity between tiers listed in the article is far more extreme than what we actually use on Metacritic. For example, they suggest that the highest-weighted publications have their scores counted six times as much as the lowest-weighted publications in our Metascore formula. That isn’t anywhere close to reality; our publication weights are much closer together and have much less of an impact on the score calculation.

Last but definitely not least: Our placement of publications in each tier differs from what is displayed in the article. The article overvalues some publications and undervalues others (while ignoring others altogether), sometimes comically so. (In addition, our weights are periodically adjusted as needed if, over time, a publication demonstrates an increase or decrease in overall quality.)”

So the researchers say one thing and metacritic says another… but really does anyone care?

Nothing is ever perfect and while some people may feel the metacritic score is slightly off kilter it would take a supreme leap of logic to believe that it’s entirely wrong. It’s a weighted average, if you can find me a score on metacritic that is patently false compared to all the critic scores then I’d give this argument some weight.

Until then this is nothing more than an Internet storm to keep media happy on quiet news days… and for that I am supremely grateful.

Last Updated: March 28, 2013

15 Comments

  1. Admiral Chief Erwin

    March 28, 2013 at 09:12

    #ShamelessSelfPromotion
    #Approved
    #TongueInCheek
    #Lulz

    Reply

  2. Anon A Mouse

    March 28, 2013 at 09:54

    ” The article overvalues some publications and undervalues others (while ignoring others altogether), sometimes comically so.” ~ Clearly they where talking about LG in the latter part of this sentence.

    Reply

  3. Jim Lenoir (Banana Jim)

    March 28, 2013 at 10:10

    You guys have it all wrong.

    The only reviews that matter on Metacritic is the user reviews. When the People go on the rampage and find their revolutionary spirit, not even the golden calves of gaming are safe from the bile.

    I know I’m in the minority, but I really love the Metacritic users. Those guys are the true vanguard of truth and honesty.

    The current user aggregate for SimCity actually captures my views on the subject, similarly their love for that strange trucking simulator brings joy to my crusty old man heart…

    Reply

    • Admiral Chief Erwin

      March 28, 2013 at 10:24

      You couldn’t wait to use the word “Vanguard” in a comment couldn’t you? 😛

      Reply

      • Jim Lenoir (Banana Jim)

        March 28, 2013 at 10:26

        I’ve been waiting whole week 😀

        Reply

        • Admiral Chief Erwin

          March 28, 2013 at 10:32

          You bloody vanguard

          Reply

          • OVG

            March 28, 2013 at 11:10

            carguard :p

          • Admiral Chief Erwin

            March 28, 2013 at 11:14

            ballguard

          • Sir Captain Rincethis

            March 28, 2013 at 11:28

            0_O

          • Jim Lenoir (Banana Jim)

            March 28, 2013 at 11:46

            Wow that escalated pretty quickly… 😛

          • Admiral Chief Erwin

            March 28, 2013 at 11:58

            Like a baws.

            ITS ALMOST LOOOOOOOOONG WEEKEND

          • Jim Lenoir (Banana Jim)

            March 28, 2013 at 12:05

            Pickled fish and hot cross buns.. BABY!!!

  4. Sir Captain Rincethis

    March 28, 2013 at 11:28

    Nice that you still cover this Gavin! I assume that’s cus their cheque hasn’t arrive yet? 😉

    Reply

  5. matthurstrsa

    March 28, 2013 at 13:33

    Slow news day on gamasutra? Anyway, do some research, chat to your gaming buddies, read some reviews, then make up your own mind. If you just look at metacritic, and it gives it a 90 and you play it and don’t enjoy it, its your own damn fault.

    Reply

  6. mornelithe

    March 28, 2013 at 14:52

    How about the 50 they allowed to be aggregated into Uncharted 3? I do believe even you Gavin, called Bullshit on that one.

    Reply

Leave a Reply

Your email address will not be published. Required fields are marked *

Check Also

Call of Duty: Modern Warfare is being review-bombed because of a rewritten war crime

Call of Duty Modern Warfare may have a gripping mission featuring a stretch of desert road…