Double Buffered

A Programmer’s View of Game Design, Development, and Culture

Posts Tagged ‘magazine’

The “Problem” With Game Reviews

Posted by Ben Zeigler on June 19, 2008

There’s been a lot of discussion over the last month or so about the game reviewing system. Stephen Totilo at MTV did a whole week of coverage on the subject, which I suggest you go browse. The spark for this discussion was the overwhelmingly glowing reviews of GTA 4, and has continued with arguments over Metal Gear Solid 4. Reviews are on the industry’s mind, and everyone thinks they know what’s wrong with them. So being egotistical and analytical, I thought I’d conclusively answer this.

The basic problem with game reviews is that no one agrees on what they’re supposed to be. Some people think they should be an objective analysis of the value of a game, in the vein of a Consumer Reports review. Other people think they should be commentary on a game’s artistic and cultural value in the vein of a book review. Should they have a score? Should the score be objective or subjective? Should they be exhaustive, or only focus on the particularly important (good or bad) parts of a game. Should they be written for a general or specific audience? Should they have a byline or be from “The Magazine”?  Should they include vague, poorly described categories like “Fun Factor”?

I think some of these extremes are clearly wrong. Consumer-report style value-only reviews are not what most reviewers or readers want. Games are not toasters, and players read reviews as much after the fact as before. Everyone wants at least some commentary and analysis. However, you can go too far, as I think book reviews (traditional newspaper ones) do. Every time I read a book review in a newspaper I come out with a clear picture of the reviewer’s innermost feelings and desires, but not a damn clue about if the book is any good. At some point professional book reviewers went down the path of full subjectivity, and I find amazon book reviews to be way more useful (and fun to read) than professional ones. Whenever a magazine tries to take out scores, everyone hates it, so we need some sort of score. I love the build up of context that comes from a well written review, but at the end of the day I want to get what I (indirectly) pay experts for in the first place: an opinion. Oh, and you know what sucks about institutional reviews? You can’t tell if you actually have anything in common with the reviewer. In fact, smaller genres often get underrated because they’re not correctly identifying the audience of the game (sometimes the publisher is at fault here).

So, we want a review that has a good mix of objective analysis, subjective opinion, and commentary. There should be some sort of clearly defined rating system, and enough information to help us decide if the review is relevant to our interests. Does this remind you of any other type of review? Yup, this is exactly what a well written movie review is. When Roger Ebert writes a review for the Sun Times, it is completely obvious that his star rating is his completely subjective opinion and not some sort of “composite” number that is derived from Sound and Fun Factor. No one calls up the Sun Times to complain that their movie got a bad review, they either complain to Roger (ineffectually) or chalk it up to difference of opinion. But, fanboys and developers across the world ask, why does one reviewer’s subjective opinion matter? Can’t game reviewers somehow come up with a provably “correct” rating for a game? And if they can’t why should we care about the aggregate at all?

The answer is that experts tend to be really good at rating things. Half of Blink by Malcolm Gladwell discusses how the gut instincts (ie, subjective score) of experts tend to be very good predictors of enjoyment by non experts. However, what experts (and humans in general) almost universally suck at is justifying that gut instinct. When we try to figure out where our rating comes from, we attempt to craft an explanation of the specific way we cleverly came up with that score. This is the process that generates the text of most game reviews (“The game is great, but the sound effects could use more punch”). Once you write a bunch of explanatory text, it then makes sense to adjust the score to be more “correct”, which generally pushes it away from accuracy. My feeling on this is that the more emotional a game experience is (there’s no way those 10 scores for GTA 4 were objective, the game has many obvious flaws), the more subjective and useful a game score is. However, any text justifying that score is going to basically be a waste of time, and should be better spent on commentary.

So if movie review scores (and theoretically game review scores) are just subjective opinion that may not apply to you, why is an aggregate score useful? Well, if a consumer shares sufficient traits with the “average” game reviewer, the aggregate score is a very useful approximation of their own estimated rating for a game.  In general, the hardcore gamers who are the initial adopters, the free advertisers, and the fanboys share a lot in common with the average game reviewer. If a game is well reviewed, it is very likely that it will generate positive gamer goodwill towards the developer and publisher and build a reputation of quality. Given that review scores have a direct effect on the future sales of related games, I think it absolutely makes sense to use metacritic scores for things like royalty payments. If you, as a game developer, can’t make a well reviewed game (thusly hurting the future sales of related games targeted at hardcore gamers), you either need to stop trying to sell games to hardcore gamers, adopt a publisher-free funding model, or stop making shitty games. As for magazine publishers? You should just make your reviews as much like good movie reviews as possible, and ignore all of us whiny developers.


Posted in Game Development | Tagged: , , , , | 3 Comments »

So Long, Computer Gaming World

Posted by Ben Zeigler on April 9, 2008

Last week, I linked to the GFW Podcast. This week, Games for Windows: The Magazine is no more. I’ll admit that I haven’t particularly enjoyed the magazine over the last 3-4 years (although apparently it got better recently), but I used to love it back when it was Computer Gaming World. It was one of the first gaming magazines I ever read, and I mourn it’s passing. The podcast lives on (and they kept all the editors for online stuff), but the legacy of CGW is basically at an end.

In memorial, I pulled out the oldest copy of the magazine I have: Computer Gaming World, March 1993, Issue 104. The cover featured Spectrum HoloByte’s The Iron Helix, a game I don’t even remember. Based on the article, it appears to be one of those exciting CD-ROM FMV games that were popular in the early 90’s. The quality of the magazine is pretty shitty, thus proving that I was not very considerate of of my personal posessions when I was 10 years old. I’m amazed the cover is still attached, actually. Here’s assorted highlights of awesomeness:

  • Page 12 features an ad for a play-by-mail (snail mail) game called Portinium II. “Receive over twenty double-sided pages of high-quality Laser Graphics output per turn. $10.00 per turn. 3-week turnaround.” I remember thinking to myself that this sounded like a really stupid idea.
  • Page 14 is an overview of the exciting new world of CD-ROM FMV games.
  • Page 31 is an ad for Wolfenstein 3d. “More like an interactive movie than an arcade game.”
  • Page 33 is an ad for GEnie, and features a picture of an incredibly stupid-looking guy with the title “this one wants to rip your lungs out.” Online gaming ads haven’t changed much.
  • Page 36 is a review of Star Control 2, and features an awesome picture of a Shofixti. “Star Control II has been placed on this reviewer’s top ten list of all time”. Still true.
  • Page 60 is a review of Eric the Unready, a fun parody text adventure that fans should pick up. The page featured a picture of a pig.
  • Page 78 features a horrible review of Stunt Island. It’s structured as an interview with a fictional movie director. Wow, what a bad idea.
  • Page 84 complains about the lackluster endings of the majority of PC games. Yeah, that hasn’t changed.
  • Page 102 is a company profile of Bethesda Softworks. “For example, one finds more expensive towels in the shower room at Bethesda’s offices than one finds in most hotels.” This article praises the management practices of the company, but it made me think that Bethesda would never get anywhere, due to it being an incredibly boring place to work.
  • Page 104: “The solution for piracy is just around the corner with CD-ROM software.”
  • Page 124 is a review of Comanche Maximum Overkill. Voxels!
  • The last page is an add for Subversion 1.0. It’s a submarine strategy game.

I miss the 90’s era of PC gaming. It’s not the same these days. Also, get off my lawn.

Posted in Game Culture | Tagged: , , , | 8 Comments »