Game reviews are worthless (and scores too High)

The views expressed in this column are the author’s alone and do not reflect those of MyGaming.

Anyone who has been playing games for a fair amount of time may have noticed the slow but persistent increase in the mean of review scores over the past decade. Or perhaps you haven’t, it’s one of those things that can sneak past you when you’re not looking, like seeing a head of grey hair in the mirror and wondering how you didn’t see it start.

I’ll take my typical jab at Activision to illustrate my point. When you think of Call of Duty: Black Ops, what kinds of thoughts resonate in your mind? The words “disappointment”,  “okay”, “bugged”, “infuriating” spring to mind for me, along with recollections of an organization purporting to defend the rights of gamers describing Black Ops as if it were some kind of infringement of our basic human rights.

Perhaps that particular linguistic set is too harsh though, perhaps some of you think “overcriticised”, “underrated” and “quite fun”.  Certainly there were some who enjoyed it, although will readily admit it was a bit less than they had expected or hoped for.

Imagine each person, from each camp, were to rate this game out of 100. What do you think the average score would be? Imagine that number, and then consider this:

Call of Duty: Black Ops received a Metacritic score of 88 on PS3. Of 58 expert reviews, an average score of 88 was reached. A score of 88, to me, says outstanding. It says if I in any way consider myself a serious gamer, this is a non-negotiable purchase. 

However, on average, 402 customers who did make this purchase disagree. They gave the game a rating of 58, which in sharp contrast to the former score says nothing more than mediocre. 

“We told you what would happen if you gave a sub-80 score, comrade.”

I know it seems like every week I’m on Activision’s case about something, but Black Ops simply serves as an example of discrepancies that have been occurring for quite some time now. The fact is, game reviews are becoming almost worthless. I’ve become far more inclined to trust some good, honest user reviews of the games than that of the “experts”. 

Games which are receiving pretty lackluster reviews are receiving scores in the 70s and even 80s. This has fallen in line with our perceptions though. A game with a score in the 70s on Metacritic is considered a bit of a failure, and anything less than 80 isn’t viewed as a good way to be spending your money. The aforementioned score of 58 wouldn’t be mediocre – it would be a death sentence. Anything below 60 and your game is only being bought by those who like the box. Would you consider an examination score in the 70s and 80s as mediocre? Why should a video game review be any different?

So what the hell happened? 

It wasn’t long ago when a score in the 80s was something to be damned proud of, and scores cracking into the hallowed 90s were reserved only for the best of the best. The most disheartening thing is that games which aren’t backed by massive publishing houses or arriving on a tsunami of hype are often reviewed a lot more honestly. 

The weakened credibility of these reviews is a problem compounded by (and no doubt tied up in) the increase in their importance. Game review scores have become critical for a game’s success. It is something often discussed by publishers, with some publicly saying that a Metacritic score not in the 80s can be seriously harmful to a game’s success. 

With video gaming becoming a billion dollar industry, this engenders powerful motivation for dishonest practices – there’s a hell of a lot of money on the line. When good review scores can literally be worth millions upon millions of dollars, bribes are certainly not out of the question.

Okay, now remember you only get the other half when Big Rigs 2 hits 90 on Metacritic.

 There is, however, more to this than just money. A publication consistently giving a publisher’s games glowing reviews is likely to be given access to exclusive content and pre-release scoops on that publisher’s next big release – a revenue-generating privilege an honest reviewer is highly unlikely to receive.

There’s been evidence of this kind of thing in the hardware world – Nvidia has been accused of cherry-picking superior cards and sending them to reviewers who have given them favourable reviews in the past. Give a bad review and you get blacklisted; the next time a round of cards is being sent out early for review, don’t hold your breath for a package.

Make it stop

So what can we do to curb this? Unfortunately, probably not very much. As a more pedigreed breed of gamer, we can rely on more than a couple of reviews, but many people read these overbloated praise pieces as gospel. All we can really do is continue to support the honest reviewers out there, and bring attention to those well over the line of credible plausibility. 

Unfortunately, it can be somewhat of a challenge for an honest reviewer, when his/her score of 75 is seen as terrible, where the intention was “good, but not great”. If you review games, have the balls to stand by your ratings, even under assault of angry nerdrage and publisher scorn. Your true audience will appreciate and respect it.

Do you rely on game reviews to pick your games? Have you felt misled in the past? Do you think the majority of game scores are accurate? Share your experiences in the forums, or comment below!

Authors
  • Krystle

    Some interesting food for thought. Article well written, I really enjoy your work.

  • jimz0r

    Chris, I couldn't agree with your article more.

    I recently replayed Half Life 2, which has the highest score on Metacritic for a PC game (96). The reason for the replay was to see whether, after 6 years, the game was still up to par with all gaming titles I’ve played since. And the verdict? It is still the best FPS I have ever played! Hell, it’s still the BEST GAME I’ve ever played.

    And Black Ops (PC) receives an 81. On what grounds? The campaign? Nothing exceedingly special about it, so not that. So does the multiplayer redeem it? On release, which is when most of the reviews were gathered, it was the buggiest MP game ever to grace the PC!

    My point is that there is no way that a margin of 15% represents the difference between one of the greatest games ever made and Black Ops.

    There are precious few gaming critics left that will dare give a triple-A title less than 75% and those that do give high ratings seem downplay a game's flaws or ignore them entirely.

  • Patriarch

    Wow, thanks Captain Obvious! 😀

    Well written though. I've been complaining about this for a while now, totally crap games get in the 80% to 88% range and anything even remotely good gets over 90!

    But the worst part is that these new kids that started playing games while this situation was already going on expects to see these scores, just look on various forums and see their poorly-articulated comments like:

    "ZOMG this gaem only got lyk 79% most horibul thing evar I is totally not gunna buy it!!!11!!"

    I don't ever trust any mainstream reviews anymore, there are a few sites where you can really find out what the game is like, but they are few and far between.

  • MooseE

    I could not agree more, I find more often than not, game reviews are so biased, usually towards the big developers… look at Dragon Age 2 on Metacritic (yes even the CEO rated his own game 10/10, that's leading the crowds astray if you ask me)it received a score of around 80 – 85 in total from critics but users giving scores of 44… i am more interested in user reviews these days, than looking up critics reviews…

  • ReBoot

    I go to sites where i can check the scores given by the users/public. That way i can get a relative good average of the game's real potential

  • PenTanium

    That's why people pirate the games first and (some) will buy it if they are satisfied. Scores FTL? Yes.

  • StefKruger

    I do not want too advocate the devil too much, but surely it is misplacing one's trust if putting it the user review scores on random web sites. The one or two sane minded level headed user reviews will always be counter balanced by 500 juvenile misguided ones.

    The Facebook & Twitter generation with eveyone having something to say is very well and good, but there is a reason why critics and "field experts" exist.

    Example : If in the US the food and restaurant critic industry were be declared irrelevant and scrapped and replaced by a system of user scores, McDonalds would be receiving Michelin stars.

    The legitimate issue about reviewer scores (or maybe just the reader perception of them??), as you rightly said, is that they basically being bracketed in in the 50-100 range, instead of the 0-100 range, which is possibly cowardly and incorrect, but arguable. If the game boots up when inserting the disk, is that at least 20%? :)

    I digress.

  • Achmed

    I gotta say I completely agree with the article. Ive never bought a game on release date for a while. Alwys wait for the reviews a week or two afterwards.

    NAG Magazine and MyGaming are the only reviews I look at. Both I feel is very honest and for that I thank u guys And girls.

    Another nice feature about MyGaming is the user rating right beneath the critic rating. I find that super cool and downright awesome. Thanks for thinking ahead!!

    But i gota say user scores arnt alwys right, some kids can complain about nothing about a game and then just give it zero so thats very unrealible u know…

  • Chris Kemp

    I agree you shouldn't go off user score averages, because you get masses of people who'll rate MW2 a 1 because they're angry about no dedicated servers.

    Even if the anger is justified, it doesn't really reflect on the game (well not to the extent that its an auto-1).

    What I do like to do is look though the written user reviews, and find someone who seems genuine and honest, and not one of the legions of trolls. Those reviews are often worth a lot.

  • Horny

    1. Ditch the scores
    2. Play the free demo
    3. Decide for yourself (buy/don't buy)

    People are unique and so is their taste in games

  • Michael Reed (PCFormat)

    I would like to say that Chris Kemp hit the nail on the head. I am a reviewer and I can honestly say that the general industry trend is that any game not getting an 80+ rating is worthless. I am not going to punt my publication but we actively avoid following this trend at PCFormat.

    I interact with Nick Simmonds on a regular basis and regularly have discussions over hardware and games. We both feel strongly about the quality of the reviews in our respective publications. I respect the reviews and the reviewers on MyGaming and believe their comments and opinions to be both fair and honest.

    Caution however must be taken when looking at the reviews written by users/ Joe Public. Reviewers, like those who work for MyGaming, NAG, Lazy Gamer and PCFormat do it as a job. I would liken them to car journalists; these men and women review cars every day so they know the good and bad aspects of many cars. I would trust their opinion over a family members for example and definetly a random stranger. Taking the possibility that there has been some back-hands out of the equation (I do not believe for a second that any of the local reviewers in any of the previously mentioned publications/sites would even think about taking a bribe) I respect the opinions of a professional reviewer because they game and review for a living making them experts in their field.

    I am not a dedicated game reviewer but I do review games on occasion. When I reviewed Star Trek Online, a game which I love and still play, I gave it 75% because it showed potential and had some excellent aspects but I felt that there was room for improvement. I also felt that the game would be more appealing to Star Trek fans than those who had no interest in the franchise.I do not consider 75% to be a bad score, it’s not a great score but it’s an average score for an average game.
    In my mind a score of 60-70 is “Wait till it hits bargain bin,” 70-80 “If you have spare cash and some spare time buy this game,” 80-90 “Go out now and buy this game,” and 90+ means “Leave work now! Go buy the game and call in sick for the next week!”
    Reviewers must not be afraid to give a game 60%. Our first and only responsibility is to our readers (a view that I am sure every ethical reviewer holds) because you are the customer.

    The views that I have expressed here are mine alone and in no way reflect the views of Panorama Publications, its magazines or its staff.

  • Nic Simmonds

    Cool story bro.

  • TallShaw

    There seems to be a bubbling up of paranoia and distrust over games critics and their publications, magazine or websites.

    Why?

    There is a simple solution. Chose a publication, online or off, that shares your views on games.

    If their views consistently do not reflect your own, move on. Its your choice at the end of the day.

    And I agree with Michael Reed. I would trust a journalists impression of a game over a hater who is just gonna hate, i.e. a user review.

  • ReBoot

    Could someone please explain to me the big difference between the reviewers score (79) and the user score(45) regarding the game Dragon Age II, shown right here on Mygaming ? Are all these users idiots or "500 juvenile misguided ones" ? Or is Dragon Age II a flop …. or not ???

  • Nic Simmonds

    @ReBoot

    I think the example you listed is a perfect illustration of why user reviews should not be taken seriously at all.

    Sure, Dragon Age 2 may have been a disappointment to fans of the original, but 45/100? Really?

    The difference is that when James reviewed the game, even though he personally found it disappointing, he remained as objective as possible and acknowledged its weaknesses AND strengths.

    A game that earns 45/100 should exhibit broken gameplay mechanics, poor production values, a disastrous story and shoddy voice work to name but a few potential shortcomings. Any reasonable person will admit that Dragon Age 2 commits none of these crimes. I personally thought the tweaked combat made the game more fun to play. The voice acting was also solid, and the game is utterly gorgeous (high-res texture pack FTW).

    The only thing it really did wrong was venture a bit too far from its roots. This angered fans of the original, and resulted in a number of people giving the game 1/10, 2/10 etc, grossly slanting the average.

    It does have numerous legitimate flaws which when reviewed independently do detract from the overall score, but to say it’s a 45/100 game is totally ridiculous.

    I’ve said it before, and I’ll say it now. People need to pay more attention to the content of the review than the score. I’m often tempted to remove scores altogether in our review process.

  • Chris Kemp

    Thanks a lot to Michael and Nic for adding some great insight from a professional perspective.

    When I was recalling "reviews of old" that did the job right, PC Format was right there in my mind. A score in the 90s, a gold award, from PC Format for hardware or a game always meant a hell of a lot. I'm glad PCF is still holding up these ideals, indeed a lot of the South African publications are really carrying the torch for honest reviews.

    I love MyGaming reviews (genuinely no bias intended here) because their scores are never overbloated, I feel like they're a really honest indication.

    I feel the need to clarify a little what I meant about user reviews. I meant more along the lines of cherry-picking some great written reviews for an impression of a game, rather than the averages which are plagued by slews of "troll scoring".

    That being said, there are still good reviewers out there doing good work, and these are always my first port of call when looking at a new game. Unfortunately, this list is dwindling, and they can't cover every game.

    To the people out there doing good work the right way, thank you, I personally value your content a lot.

    Thanks again to Michael and Nic for their lengthy contributions, and to all the other posters here for sharing their views and experiences.

  • Opportunist

    Stopped caring about the actual scores ages ago. I only care about the issues mentioned in the review and then decide for myself whether they warrant putting off my planned purchase or not. For this reason I actually prefer overly critical reviews which is what GameSpot and IGN often get accused of these days. The more issues the reviews point out the better I can judge the game.

  • Wade

    Great article.

  • Eternal Engine of Linguistic Massacre… II

    Suffice to say that the review scale is flawed – it's pretty obvious when you consider that if you're using the percentile scale, 50% is perfectly average.

    I agree with you on your statement regarding the review's contents, Nic, although I challenge you then to either implement a more accurate metric or, as you say, remove the score entirely.

    The fault is NOT with reader who ignores the content of the review, but the reviewer or publication that fails to align the score with review's content.

    What rankles is that the content of a review is, when viewed with a critical eye, completely at odds with the posted score. I feel many publications do this to appease the developers and/or publishers, since aggregate services such as Metacritic – which are used by purchasers to gauge critical reception to a title – only take this metric into account.

    And let's be honest: bribes, exclusives and pre-release access aren't the only monetary rewards up for grabs. I happen to notice that that that filthy word – 'Advertising' – hasn't been mentioned once.

    Advertising is the bread-and-butter of gaming publications, and, I would argue, the primary source of revenue, and through it publishers wield significant influence. You can spend an entire review slating a game (Demonstrating your cred and I-Ain't-Afraid-Of-Sticking-It-To-The-Man'ness) only to give it 7/10 to appease the Great Advertising Gods.

  • boramk

    I always subtract 10 from the score.

  • Joemighty16

    I use reviews only as an indication. I primarly decide on buying a game on my own, and may only use a review as an indication of what “reviewers” think. Too often I had seen reviews that praise a popular game, but then nails an unpopular game, complaining about the exact same elements that were praised in the popular game.

    I do collect reviews of all my games, but focus more on the discussion than on the score.

    How do you judge a game’s worth? Do you follow the score given by a select few professional gamers, or do you follow the majority vote of the players? But then you also have the distinction between “good” and “enjoyable”. Just because Shakespeare is good, doesn’t mean I enjoy reading it. I play games for the enjoyment and therefore I have a diffirent rating system than, say, a professional reviewer who may focus on other aspects.

Top