Okay, I’m already starting to see a lot of drama over Tom Chick’s one-star review of Halo 4. (There’s a ton of it on Twitter. No direct links, but it’s not hard to find.) People are complaining about how he “couldn’t possibly think that the game was a 20%!” and how “he’s just trolling! and how “he’s ruining the MetaCritic average!”
Sorry, no. That’s all complete nonsense, and in a lot of ways, it smacks of people wanting to have it both ways.
You can’t have it both ways on scores. If you’re calling out score inflation and piously claiming that “a fifty percent is an average score” on your site, then you have to allow other sites to choose their scoring meters too. Chick’s site, “QuarterToThree” uses a film-like “star” system. Anybody who has read a movie review in the last few decades knows that a one-star film is not execrable, but definitely flawed, and will likely disappoint anybody but the most dedicated fans of the franchise.
I read the Chick review. That score is absolutely backed up by the text. He makes it abundantly clear that he is disappointed by what he played. He praised the original Halo’s “raw genius” and called the recent effort “a drawn-out retread without any fresh perspective”. You can agree, you can disagree, but that’s a one-star review.
And if you think that single star is a “20%”, then you don’t “get” measurements, because ordinal and ratio data aren’t interchangeable. The gap between 1 vs. 2 stars in a score may be enormous compared to the gap between 2 or 3 stars, depending on the reviewer, and there’s nothing wrong with that. If MetaCritic translates one star as a “20%”, and directly compares it to some other site that gives an equally disappointing game a 60%, then guess what? MetaCritic screwed up. Don’t apologize for them or make the same mistake.
You also can’t have it both ways on reviewers’ freedom. I’ve read game critics and game reviewers bitterly complain about fanboys’ angry reactions to their reviews over, and over, and over again. It’s why I’ve never really been tempted to write the things. Who needs the grief? Reviewers certainly don’t; they want to be able to express their opinions and criticism, and truly hate it when they’re accused of “trolling” or “bias” or being “on the take” or some such thing. They get especially peevish (and rightly so!) when their words are skipped over and they’re castigated for their scores.
Well, guess what? IF IT’S GOOD ENOUGH FOR YOU, IT’S GOOD ENOUGH FOR TOM CHICK. You may disagree with him. Lord knows I have in the past. I’ve read reviews of his that I thought were so completely out to lunch that I was tempted to put in a comment asking Tom to pick me up a turkey club, hold the mayo. But if you accuse him of simply “trolling”, instead of being an idiosyncratic reviewer that you disagree with, you’d best have a damned good reason. A reason that’s one hell of a lot more compelling than “here’s a set of reviews I disagree with and also look at that score!”
I’ve disagreed with Chick before. I haven’t played Halo 4 yet, but I may well think that he’s completely full of shit on this one as well when I do get the chance to play it. That doesn’t mean he’s “trolling”, any more than you are when you write a review that people disagree with. Go after Chick, and you’ve given the fanboys every legitimate reason to tear you limb from limb in comments. You’re no different than he is.
Mostly, though, you can’t have it both ways on MetaCritic.
Reviewers regularly respond to angry fanboys screaming about their scores lowering metacritic averages by saying that “Metacritic averages don’t or shouldn’t matter”…usually right before insisting that they don’t think about the effect on metacritic averages when they write their own reviews.
Thing is, the fanboys are actually on pretty good ground on this one. The dirty little secret at the bottom of all of this is that they have absolutely EVERY reason to pressure reviewers for better scores. MetaCritic averages do affect their favorite developers’ bonuses and job security. They do affect unit sales of the games they enjoy. They do affect whether beloved franchises get new installments. Rational arguments won’t work against that, because they are making the rational move. Scary, but true.
So the only way to stop the harassment is to make it perfectly clear that, rational or no, you don’t CARE about any of this. Reviewers need to make it clear that their only concern is expressing an informed opinion. Their scores are a reflection of the text, and the text is what it is. Full stop.
You need some demonstrate some solidarity to pull that off. The fanboys are going to notice if you start carving up some other guy on Twitter for hurting developers giving a bad score, and guess what? They’ll do the same thing to you! Sure, you might think that his “20%” (it isn’t a 20%, it’s a one-star) is totally different from your 6/10. You might insist that your score is sensible whereas his is nonsense. Guess what? It won’t help. They won’t believe you. They’ll hammer you for ruining a Metacritic score and throwing young developers to the welfare lines just as quickly as you hammered Tom Chick.
Don’t get me wrong. Disagreement is fine. Disagreement among reviewers is great. Disagreement among reviewers is what will help kill this toxic notion that you’re nothing more than MetaCritic input devices and that your words are a bunch of meaningless bullshit that distract from the all-important score. Disagreement among reviewers helps show that you’re people, not commodities.
Don’t confuse this with an endorsement of Chick’s opinions (hah!) or a condemnation of reviewers, either. Like I said, I don’t even write the things. Even if I disagree with a review, I still have immense respect for the people who stick their necks out like that. Some reviews are straight-up terrible, sure, but there are loads of reviewers and critics who demonstrate professionalism and care in a field that often doesn’t reward it like it should.
But, please, don’t fight this battle. Remember Franklin. You hang together or you hang separately.
Leveling Criticism 