Friday, January 2, 2015

Scores Always Lie

A+, two thumbs up, Buy-Rent-Skip, four out of five stars, 60% approval rating, 7.5/10, on a scale of Big Mother Truckers to Super Mario Brothers it's a Streets of Rage.

These are the kinds of rating systems that review websites and publications offer up on a regular basis.  They are then processed by aggregated review sites like Metacritic and the final results are spewed out in the form of "83" or some other number out of a hundred which, devoid of an explanation, tells the consumer absolutely nothing about the game itself.  In fact the only thing that can be garnered from review metrics is a vague idea of how reviewers felt about a specific game.

Recently on the Giant Bombcast there was a discussion regarding game cost vs longevity.  The conclusion was jokingly that Giantbomb should dump it's current review system and instead just have a screenshot, price and minimum-hours-to-complete label.  While this might sound like a silly suggestion it actually conveys more meaningful information that the star rating system currently in use.

Yes, it's a bit uncharitable, but as many people have pointed out before me the video game industry really only operates on a 7 to 10 scale.  Anything less than that is considered shovelware, and not worthy of a review score.  The one exception being Angry Joe, who really does use the 1 to 10 scale as intended.  Exacerbating the problem are inflated or bombed aggregated user metrics.  Most readers know what I mean; people who are on a publisher payroll, or else are friends/family of the developers.  These "user reviews" are written up like vaguely positive comments in order to push the game higher up the charts.  Counter to this is all the one's and zero's with equally unhelpful statements along the lines of "It sucks!" or "I hate it."  Of course there are reviewers who take the time to do a more in-depth analysis, but even these can come off insincere.  Classic examples include a non-trivial
number of Gametrailers' video reviews wherein the majority of the time is spent complaining only to conclude with a 9/10 rating.  Or conversely having nothing but glowing praise but then end with 7.2/10.  Unsurprisingly, the comment sections of these sorts of reviews tend to fill up with critics of their own.  The key difference being they're not criticizing the game, but rather the review, the person who wrote it, the website, and in some cases the industry as a whole.

The simple fact of the matter is, what makes a good game is highly subjective.  I'm sure we can all agree that inconsistent frame rates, lock-ups and inaccurate collision detection are all terrible, but even something as undesirable as glitches are not universally agreed upon as being a bad thing.  I can think of a few people who prefer Red Dead Redemption and Elder Scrolls V: Skyrim in their hilariously buggy form to say the current patched and polished versions.  Even what defines a clunky interface is not easy to get a consensus on.  I can't stand the button layout on Metal Gear games, but have no issue with old survival horror titles.  I'm sure that I'm in a small minority on this one, but I think fixed camera angles and tank controls suit certain aspects of the genre rather well.  Obviously others will rightfully disagree...

So, what's to be done about review scores?  Nothing really except to get rid of them entirely.  The medium has advanced to the point that they have long since outlived their usefulness to consumers, particularly when compared to "Quick Looks" and "LPs."  If you're like me though and still enjoy browsing reviews now and then, I think the best course of action would be to sum it up with one sentence using words like "play it," or "don't play it," along with conditional phrases starting with "if," "unless," "assuming" and so on.  That way you can get a quick impression of the overall review without cold impartial numbers or having to slog through an a several page essay.

No comments:

Post a Comment