Double Buffered

A Programmer’s View of Game Design, Development, and Culture

Is MetaCritic Scientism?

Posted by Ben Zeigler on July 6, 2008

On this Friday’s 1UP Yours, Denis Dyack spends an hour discussing NeoGAF, Technopoly, The Player of Games, Peter Drucker, and many other things. Many of the points he makes about the social utility of message boards (NeoGAF is of negative social worth) make sense. However, he spends some time talking about Scientism, a topic I care some about. Scientism is the misuse of scientific terminology, ideas, and theories in a non-scientific, socially-negative way. For instance, the use of phrenology for criminal prosecution or inappriate use of IQ scores. He claims that MetaCritic is nothing but a misuse of statistics, and is endemic of the greater ills in our society. He equates it to the use of 11+ tests to decide professional future.

This makes absolutely no sense. As I mentioned 2 weeks ago (sorry about the delay, stupid crunch time), MetaCritic score is a totally valid, although limited, measure of game quality. We’re not talking about a single superfluous score, or some complicated easily-manipulated formula, it’s just an average. If averaging subjective scores is somehow “Scientism”, many incredibly useful, valuable metrics are. Rejecting the aggregate opinion of reviewers or users just seems like total elitism to me. It’s a statement that because we have no absolutely perfect way of integrating that opinion, that opinion is completely worthless and should not be available to anyone. Combining this argument with one of his other ones (that magazines and websites should never express any negative criticism about a preview build), Denis comes accross as a smart guy with an excellent analysis of message board mentality. But, he also comes accross as someone who doesn’t think that anyone besides himself has a right to assess and validate his (and his company’s) creative work. This kind of attitude becomes very evident, and will do nothing but encourage people to attack him personally.

UPDATE: Shawn Elliot makes the same points I do but better, in a blog post.


2 Responses to “Is MetaCritic Scientism?”

  1. Noah Kantrowitz said

    While I have no problem with Metacritic directly, the whole idea of expressing the quality of a game (or anything else) as a fraction seems off to me. Averaging many of these scores together does help, but that doesn’t change the fact that the base metrics are entirely arbitrary and often scores are more a quantification of gut reaction than actual ejoyment or quality.

  2. JZig said

    Gut reaction is generally a pretty good estimate of overall enjoyment and quality, and better than more reasoned, thought out measurements. I’ll pull up some cites from Blink if you care (or you can borrow my copy, it’s a good book on gut reactions)

Sorry, the comment form is closed at this time.

%d bloggers like this: