Double Buffered

A Programmer’s View of Game Design, Development, and Culture

Archive for March, 2009

GDC09: Post Mortem: Mission Architect for City of Heroes

Posted by Ben Zeigler on March 30, 2009

Here are my notes from the GDC session “Post Mortem: Mission Architect for City of Heroes” by Joe Morrissey from NCSoft. This talk had particular interest to me, because it’s based on some technology that I was tangentially involved with while Cryptic was still working on City of Heroes. The structure of the talk was to give an overview of the design of the new Mission Architect (user-generated content) feature for City of Heroes, as well as show some early feedback from open beta.

The Design of Mission Architect

  • Each player can publish 3 “stories” of 5 missions each, with up to 25 objectives per mission. This can be done at any point, from level 1 on.
  • The content is constructed from 7000 pre-made interior maps (no level customization), a hundred dialog states, and 10 large mission goals.
  • On top of that, players get to customize all text in the mission, pick what the players fight, and add NPC helpers.
  • The original design came out of a tool for developers to quickly make simple missions (via a tool that exported to excel that exported to text that exported to game. I always disliked the CoH mission pipeline)
  • The original design for the architect UI was to build it on the character creation UI, because it’s one that the players already understand. They put a lot of effort into showing errors, having tooltips, and having in-game tutorial text

Playing Architect Content

  • For finding existing content, you can filter on played count, rating, or morality (hero, villain). You can sort on rating, date, or length. It automatically gives you a page with some default filters, but there are also “developer’s choice” and “hall of fame” pages. Finally, you can do a keyword or ID search if you want something specific. Joe mentioned players were advertising their missions via ID number, which actually strikes me as a fairly bad idea.
  • Playing the created missions gives you equal XP to playing normal missions. However you get redeemable “tickets” instead of normal item drops for killing things. (This sounds good to me, because default minion drops in CoH were often lackluster)
  • To “Control the content”, they reward extra story slots for successful authors, allow private player comments creators, and use an automatic language filter for IP and profanity. (I find it interesting that Joe explicitly said the goal was to control content instead of nurture/encourage it)
  • Content flagging works by automatically banning content when it receives a certain number of flags. At that point creators can make changes and re-upload. On a second automatic ban, CSR manually checks the content and either perma-bans the content (so the creator has to start from scratch), or marks the current version as “unbannable” so it stays up until creator edits it. They mentioned they would track “grief voting”, but it didn’t sound more concrete then just making the identity of flaggers known to CSR so they can check for problems. (I’m very curious how this works out)

Feedback on the System

  • When they originally announced the feature, they had some PR/community relations issues. Players didn’t understand the scope of the feature, and they had to go back and put more effort into specifically explaining the limits and capabilities of the system.
  • The hard part of implementing the feature wasn’t writing it from scratch, it was dealing with making a 6 year old game deal with an entirely new concept.
  • To deal with potential exploits, they reduce XP if a mission has NPC helpers. End of mission rewards are based on what you actually did in the mission instead of being flat per mission. The examples made it sound like this was just an extra multiplier on kill xp you gained, but it was unclear.
  • One ongoing discussion during development was between “Fun to Play” vs “Fun to Create”. Should they focus on features for the missions, or the UI/process for creating it? This was an ongoing debate, and end result was better due to both sides being represented.
  • They have gotten more content then they expected, and know they need to add “multiple layers” of filtering. That’s being actively worked on.
  • One conceptual difference they didn’t anticipate was that players are not developers. The pipeline was originally designed around working on things and then “shipping” them in final, uneditable form. This has now changed, but took some rethinking.
  • They’ve noticed the existence of a new role for players, a “critic” who looks at existing content and attempts to aggregate it in forms useful to the average player. There is nothing formal built in to the system to encourage/support this, but they’re thinking about it now.

This was a interesting talk from the perspective of seeing how teams can go about adding UGC to an existing product. From the presentation I feel like they’ve done a good job of working through the core features of what is needed for a successful UGC system, but I didn’t see much evidence of learning from the experiences of other teams that have tried previous forms of UGC. This definitely seems like a solid feature for CoH, but I hope it gets the ongoing support it needs to be successful, and doesn’t end up like other highly touted features in the history of CoH development.


Posted in Game Development, GDC 2009 | Tagged: , , , | 2 Comments »

GDC09: The Cruise Director of AZEROTH: Directed Gameplay within WORLD OF WARCRAFT

Posted by Ben Zeigler on March 30, 2009

Here are my notes for the GDC Session “The Cruise Director of AZEROTH” presented by Jeffrey Kaplan from Blizzard. The first segment was thoughts on the quest design in World of Warcraft, and the rest was about specific mistakes the quest team felt they made in the original WoW. There was a lot of interesting and relevant info packed into a nice digestable lecture:

Directed Gameplay in World of Warcraft

  • The goal of directed gameplay in the design of WoW was to improve immersion within the context of an open-world design.
  • Directed gameplay can be achievements (750 in WoW, according to Jeff “Some just suck”), UI via tutorials or navigation information, and Quests, which are the primary source of directed gameplay in WoW.
  • Players of WoW complete 16.6 Million quests per day, and 8.5 Billion between June 2007 and March 2009.

Original Design of WoW Quests

  • The original design of WoW was to be content driven, with the flavor of an open world. Original goal was to have 600 quests at launch, to compete with Everquest’s 1200 (which included the first few expansions).
  • Then in internal Alpha, Blizzard employees said (according to Jeff) “What the Fuck” about the number of quests, and they eventually had to scale up to 2600 quests at launch, 5300 through Burning Crusade, and 7650 today.
  • A big goal was to improve quest accessibility. This came through in putting ! above quest givers (which pissed off the hardcore, who thought finding quest givers was “gameplay”),  integrating a quest log, orienting the log to give “cliff notes” for the player, and showing the rewards ahead of time.
  • Players should never have to try and discover the core game experience. When people can’t figure out what to do, they get bored and quit.
  • Showing the rewards ended up not being that important, because players learned to trust the developers: Quests in WoW were by design the “smart” way to play, and players felt safe knowing they could trust they would be playing efficiently

Mistakes in WoW Quest Design

  • The “Christmas Tree Effect” of giving too many quest givers was determined to be bad. They now give you a max of 7 available quests. He feels this discouraged players from reading quest text and getting involved in the world’s fiction. (My view is that it’s a lost cause to get players to care about the fiction of non-custom quests, and personally enjoy trying to optimize my questing)
  • “Too Long, didn’t read”. If the text is too long then NO ONE will read it, even people who want to. WoW quest text has a hard limit of 511 characters, and they have to fight for it when designers and programmers want to increase it.
  • Medium Envy. Quests that try to hard to mimic other mediums (film, books) tend to not work out too well.
  • Mystery in quest objectives. In today’s world there is no way to effectively do mystery for quest mechanics. Players will figure it out anyway, but they’ll just get irritated if the developer forces them to check a FAQ. Put the mystery in the fiction, but NEVER so players don’t know what they can do next.
  • Poorly paced quest chains. There is one 14-level quest chain in WoW that is way too long. This makes players lose trust in the designers and think too much about how the quest was designed.
  • Gimmick quests without polish. You should direct players towards the fun that is your core mechanic, not waste time half-assing bizarre vehicle missions (I would argue that in certain cases it can be good to distract players from your core mechanic)
  • Bad flow in a zone. If you give quest chains that are too deep, or are too uniform in objective, the zone starts to feel boring and players quit. The WoW quest team now explicitly graphs out mission flow for a zone, and uses that to evaluate how fun a zone might be.

Specific Collection Quest Mistakes

  • Creature density issues (too few, too many, hard to find, large travel distance). If it’s hard to GET to a quest objective, that has to be factored into the balance or players will be irritated and distrust the developer.
  • If you have a collection quest that takes too many items or types of items (that irritating 19 part quest in Stranglethorn), it screws the players by confusing them and using up valuable backpack space.
  • “Why collect THAT?” There has to be a reason you are collecting things. If you’re collecting “gnoll paws”, why doesn’t one always drop after a kill? Why don’t 4 drop? Why isn’t it just a kill task in the first place? Collection quests need to have some logical payoff, such as being combined into a potion that you give to someone.
  • WoW has a standardized collection quest drop rate of 35%. This leads to good and bad streaks, so in Lich they changed it to a progressive percentage, where it gets more likely to drop if you haven’t gotten one in a while. This removes bad AND good streaks, so they had to raise the drop rate to 45% or so to match with player perception. (This is a good start, but I think a better study into the actual psychology of drop rates is warranted)


  • They have 5 full-time quest designers, as well as help from encounter designers. There were 60 developers on the original WoW team and 140 now, but they can’t really scale the quest team. They prefer a small, manageable team who communicates a lot.
  • They don’t have any sort of magical quest testing setup, other than that each designer must play their own content.
  • The original design of quests encouraged players to travel between zones frequently. Eventually they realized this was a bad idea due to how long the zone travel times are in WoW, and now try to only use breadcrumbs to leave zones when it makes sense. (This still needs a lot of work, especially in early zones)

One take away for me was that Jeff kept talking about how important it was to go back and fix up old, badly designed quests, but that CLEARLY has not happened to any significant extent. They know the problems, but even a company like Blizzard appears to enjoy making new stuff much more than fixing old broken stuff. I’ve been thinking that this is just an inherent bias built into creative people, and I don’t know the right way to combat it. Other than that, the talk did a good job of getting across some of the specific design decisions of World of Warcraft, and did it in a way that was clearly targeted at devs instead of fanboys. Despite seeming a bit nervous on stage, Jeff gave a great presentation.

Posted in Game Development, GDC 2009 | Tagged: , , | 4 Comments »

GDC09: From COUNTER-STRIKE to LEFT 4 DEAD: Creating Replayable Cooperative Experiences

Posted by Ben Zeigler on March 29, 2009

Here are my notes for the GDC Session “From COUNTER-STRIKE to LEFT 4 DEAD: Creating Replayable Cooperative Experiences” presented by Michael Booth from Valve. As a summary, half of the talk broad overview of the design of Left 4 Dead, and half of it was discussion of specific methods for achieving the randomized AI behavior. The first half was a rehash of various stuff I’ve read other places online, but the second half was all new to me:

Initial Design of Left 4 Dead

  • Why Left 4 Dead? The developers (at this point pre-Valve-purchase) identified a gap in the market for a cooperative shooter, and saw that it would be a good format for building a community around. They decided to fully commit to a game that requires 100% cooperation, and guide the design to fit that.
  • The entire survivor team was treated as a single player. This means that players should be happy if anyone makes it out alive, and the design needed to penalize players who abandoned their teammates. However, the team had to do this in a way that would feel natural, because artificial teamwork never seems to work with the somewhat cynical gaming crowd.
  • The zombie apocalypse setting was a logical solution to this problem, because “stay together or die” fits naturally with the fiction of the genre. Players already understood why they had to work as a team.
  • The team started with just zombie wanderers and hordes, and then they added in the special zombies to fix specific gameplay issues that arose from the original playtesting. The hunter was the mechanism for enforcing death to lone wolves. The smoker was designed to break up overly coordinated groups so they would still experience the fun of panic. The boomer was designed to make players think before firing.
  • The moments of incapacitation that special infected added were very useful in building cooperation. It forced players to work as a team, and gave them the opportunity to “be the hero”. It turned out players really ended up loving this mechanic, and it drove people to keep playing the game.
  • The Tank and Witch were designed to create points where the players would have to temporarily change their strategy completely, and were modal in the same way as a traditional boss.
  • The original goal of vocalizations was to help with situational awareness, but it was found that giving the characters personalities helped to create a sense of camaraderie. This encouraged players to work as friends, by working within the established social relationship of the existing characters.
  • One concept they discovered midway through was the importance of Dramatic Anticipation. Originally the Boomer was a bomb that exploded when shot, but this was found to be too penalizing to beginning players (they would shoot one and kill most of the team, which would then make the veterans ostracize them, killing the community). They changed it so the boomer would draw a zombie horde, which actually ended up feeling a lot cooler, because the dread/excitement that occurs when you wait for the horde to come. They then pushed that concept with other details, such as the car alarm and tank sound effects.

Structured Unpredictability

  • Overall the goal of gameplay was Structured Unpredictability. The basic idea is low probability + high drama = memorable. The goal was to create awesome possibilities that people would remember, but to keep it structured so players could learn some skills for dealing with specific situations when they did randomly arise.
  • A big part of this is the adaptive dramatic pacing. This originally came about from studying the pacing of CounterStrike. CS has a very “spiky” pace, where there are periods of quiet tension followed by extreme excitement. It was very unpredictable, but sometimes lead to dull games when both sides decided to be cautious. The goal of Left 4 Dead was to bring this pacing to Co-Op play, but without the possibility of boredom.
  • The algorithm for pacing first estimates the emotional intensity of the team. The solution they settled on increases intensity with damage received, and then decays over time if the players are not actively fighting. The algorithm uses the highest intensity of a single member of the team, as opposed to an average.
  • The 4 phases of dramatic pacing are Build Up, where it spawns things until someone hits high intensity. Then it sustains it for a bit, then fades it, then goes into “relax” mode for 35-40 seconds or until they reach farther in the map.
  • Basically, the system is always attempting to spawn things in Build Up, but if the players are in relax/fade it will refuse to spawn things, thus giving them a breather.
  • The specific method of spawning works by keeping track of an “active area” around the team. It then uses information in the pre-computed “navigation mesh” as well as the concept of “flow distance” which is how far along in the map the players are.
  • Normal infected spawn in areas that are currently not visible, and have NOT already been checked (so there is some gameplay advantage to searching. This was news to me). They normally spawn behind the players, to force them to keep going forward.
  • Special infected spawn in areas that are currently not visible, but CAN spawn in areas that have been checked. They often spawn ahead of the players.
  • Bosses spawn according to their own system. It splits a level into segments, and for each segment it will spawn a tank, a witch, or none. It never spawns the same thing twice in a row, but otherwise is just random and is not attached to the intensity system.
  • One important concept was that there are no preset spawns. Preset spawns force players to memorize spawns, which hurt cooperation by penalizing new players. Instead of preset spawns or complicated algorithms, it works via several layers of very simple algorithms.
  • Other new features that helped cooperation were changes to VoIP, an in-game instructor, voting to kick players, split screen play on the XBox360, achievements, and the new lobby system.
  • The robust survivor AI (derived from CS bot AI) was a huge win, because they could always balance for exactly 4 survivors, and the drop-in-drop-out play meant that setting up online games was much less irritating. Also, it was very useful for testing, because they could run a group of AIs at high speed to test out levels.
  • In conclusion, random internet players WILL cooperate if you provide the proper structure for them to do so.


  • Because their primary focus was on the cooperation aspect, gameplay features like ammo scarcity and multiple character classes ended up not as priorities. There is work that can be done here to add depth, but needs to be balanced against cooperation.
  • Michael said that all of the AI and balancing was a black box entirely controlled by software. He felt this was a problem now that they were part of Valve, and they were fixing it. (I’m not sure how bad this really is, as long as you have a responsive programming team)

The most important take aways for me were that Left 4 Dead implemented their entire dynamic AI system as a layered set of extremely simple, playtestable algorithms. Also, they put a lot of effort into designing to encourage cooperation and a healthy community (as opposed to prioritizing gameplay depth), which obviously succeeded given the size of the Left 4 Dead community. Also, the identification of the power of spiky pacing is something that I wish every game understood. This was a great talk, and I would recommend everyone check out future talks by Michael Booth.

Posted in Game Development, GDC 2009 | Tagged: , , , | 3 Comments »

GDC09: Meaning, Aesthetics, and User-Generated Content

Posted by Ben Zeigler on March 29, 2009

Here are my notes for the GDC Session “Meaning, Aesthetics, and User-Generated Content” by Chris Hecker, of EA/Maxis. It was a broadly-focused talk about things both directly and tangentially related to User-Generated Content (UGC).  It reminded me a good bit of Chris’s talk from last year, in that I wouldn’t say it had a clearly defined focus, but did a good job of bringing up a variety of interesting points:

Categorizing User-Generated Content

  • Two axis of categorizing User-Generated Content: Aesthetics vs Behavior, and Parametrization vs Creation.
  • Parametrization means that the space of possibilities is small enough that it is theoretically completely explorable, and sliders are the normal UI for this. An example of this kind of content is the City of Heroes costume editor, where the possibility exists of a “Random” button which is guaranteed to create a valid costume (incidentally, the City of Heroes random button has issues that mean it won’t actually explore the entire theoretical space)
  • Creation becomes the correct approach when there are too many dimensions of behavior to explore using only sliders and pickers. Creation-based UGC ends up creating create<->test cycle. The length of this loop affects the overall usability of the system, where long cycles are harder to use and approach just being full development tools.
  • Given those ideas, what are the limits of what is considered UGC? Are the machinations of player corporations in a game like Eve UGC? What about something like high-level fighting game tournament play, where the excitement comes out of the specific tricks players invent.

And Now For Something Completely Different

  • At this point, Chris’s talk was interrupted by a surprise Russian Space Minute featuring Will Wright. He went over the development of the Russian Buran space shuttle competitor, and it was both informative and hilariously out of context.

Who Makes UGC?

  • Who really makes UGC? According to wikipedia statistics, 2% of users perform 73% of total edits. However, that’s not a useful statistics because it’s about number of edits, instead of amount of actual content. Analyzing the Alan Alda article on wikipedia, only 2 out of the top 10 content contributors (by amount of text added) were even registered. 1% rule is a lie.
  • The old broadcast model of communication has broken down, and has split into two competing models: crowdsourcing ( vs curated (Pandora). Both have benefits and issues, but crowdsourcing is the currently more successful model
  • In Salganik, Dodds, Watts 2006, the authors did some research into song rating by groups of people. Research subjects were split into several groups, and in each group half the subjects would see the ratings of others when making their rating, while half would rate the songs only based on their own thoughts. Across the groups, the subjects who didn’t see the other ratings were fairly stable, while the subjects who saw the ratings of others tended to make much more random ratings, that would stabilize based on the initial ratings in their particular group. Seeing the ratings of others made the end results more random, which is counterintuitive.
  • Dan Gilbert (TED Talk) has researched the Free Choice Paradigm for Dissonance Reduction, or what is also called synthetic happiness. Research worked by having people rate 6 paintings, and offering them a print of their two middle choices (which were rated very closely initially). Then, several months later they were asked to re-rate the paintings. The mediocre painting they choose became much more liked, while the one they didn’t choose before became despised. Even more interesting, they performed this on people with inability to create short term memories, and the research still worked. Choosing something inherently and automatically makes you enjoy it more. (I call this the “fanboy” effect, where the system you buy becomes the system that is obviously better. It’s not just for people on forums, it’s basic human psychology)

How Do Games Mean

  • An important near-term focus of game research is figuring out How Do Games Mean? Ie, what is the correct method for a game to provide meaning?
  • One approach is the “Message” model, as discussed (and discouraged) by Frank Lantz and Jonathan Blow. This means that the creator adds explicit packets of meaning to their games, which they then transmit to their audience, much like passive media.
  • Another approach is the “Immersive” model, as discussed by Steve Gaynor and tale of Tales. This consists of building a world that contains various bits of meaning, but NOT explicitly showing them to the player. The player is then free to find them themselves, or ignore them as desired.
  • The last approach is the “Looking Glass” model, as used by Marc LeBlanc and Doug Church. The idea here is to completely abdicate authorship to the user, to get off the stage and let them actually create their own meaning.
  • But do we abdicate enough authorship as game designers? Walton Ford is a painter who creates works with lots of layered meaning, and also adds extensive labelling to point out the intricacies of the work. However he has been trying to wean himself off the labels, because they feel too directed, and leaves no room for interpretation.
  • Is it time for an “Interpretative” model of meaning in games?
  • For instance, there is the difference between beautiful (safe) and the sublime (scary). UGC pushes expression, and leads to things like weird double monkey creatures in Spore, which are not beautiful but have the scaryness of the sublime.
  • What are games as a platform? Games are a platform for meaning.

Sorry about the notes for the end of the lecture, I was getting tired and hungry by that point, and the lecture was running long. I may have reconstructed them incorrectly. Also, throughout the lecture, Chris demonstrated various Spore creatures that somewhat illustrated his points, but I was too distracted by how cute the walking chair was to accurately note which ones he showed. Overall, it was a definitely a talk worth going to, although I can’t help but wish the disparate threads had come together a bit more satisfyingly in the end.

Posted in Game Development, GDC 2009 | Tagged: , , , | 2 Comments »

GDC09: ‘Winging It’ – Ups, Downs, Mistakes, Successes in the Making of LITTLEBIGPLANET

Posted by Ben Zeigler on March 27, 2009

Here are my notes for the GDC Session ‘Winging It’ – Ups, Downs, Mistakes, Successes in the Making of LITTLEBIGPLANET. It was a general postmortem on the development process of LittleBigPlanet, with a focus on specific mistakes they made. The session itself was also winging it a bit, it was presented via a custom Mac app that they apparently wrote on the plane ride over. Throughout, they showed various videos and early screenshots of LittleBigPlanet, on a weird zooming grid space. It was presented by Alex Evans and Mark Healey of Media Molecule. These notes are categorized by topic instead of chronologically, because the talk changed topics fairly often.

Development Process

  • They had to deliver a set of monthly milestones for Sony. Milestone were initially delivered in the form of videotaped studio walkthroughs with content shown. Milestone requirements were generally worked out the prior month, so iteration time was fairly short
  • Spent 3 months working on a “beanstalk” level, that was beautiful but never really worked because the vertical orientation of the level lead to severe gameplay issues. Eventually got painfully cut.
  • The Tech Director spent a lot of time working on a “ribbon” 3d play space, where it was linear gameplay but presented as full 3d. This lead to complication in the code base without providing benefit to designers. Eventually was entirely excised from code base.
  • They kept an actively maintained Design Doc that was used as a teaching and summary tool, but was not at all considered final. It was more a “snapshot” of the current thinking about the design of the game.
  • Tried using Lua for their scripting language, but ended up using a proprietary one becuase they couldn’t serialize the state of the game satisfactorily using Lua. They could load old versions of saved games on new versions of the code, which was complicated but deemed worth it.
  • In general, they worked on one branch of the game, but it worked because they had a small, highly competent team. They put a prime focus on backward compatibility with all of their code changes. They don’t really have a specific strategy for scaling up, working on that now.
  • Features such as the look of Sackboy came about originally as a fight between various artists with lots of ideas. Then, they synthesized the best concepts for each of the designs into one, and then the chief character artist owned Sackboy from then on.
  • Original design was for Sackboy to always be directly facing the camera, to simplify the physics interactions. However, the artists deemed this unacceptable, and eventually one of them coded up a prototype to prove the system would work with side-view movement as well. It mostly worked so coders fully implemented that solution.

Editing Tools

  • Initial version of editor tools was entirely physical and exactly the same as playing the game. This was one of their strongest initial principals. Moving away from entirely physical effort took a lot of prodding, and eventually it happened when part of the team coded up a prototype to try and convince the rest of the team, which eventually worked.
  • Once they moved to a more abstract editor, they had  a variety of UI problems, especially involving placing and scaling objects.  They tried to optimize button count and text, but the problem was actually the basic metaphor used by the editing tools.
  • The final editor design ended up being an extension of the only fun part of editing before, placing stickers. This lead to the integration of very complicated CSG tech for allowing arbitrary material shapes, but this was deemed to be worth it despite the risk.
  • At point in production, all editing of shipped gameplay started taking place using entirely the console editor. This forced the designers to never cheat, so lead to improvements to console editor as well as entirely consistent, physical gameplay. Players knew they could do exactly what the designers did

Community Tools

  • Original design for community tools (search, etc) were very much using the Web metaphor, and used a lot of images and text. This had problems with display and interface.
  • Shipped community tools shifted dramatically the opposite direction towards ease of use and simplicity. They felt they probably shifted too far, and worked post launch to bring back some of the missing functionality.
  • Discovered that anything more than 2 clicks from the front page had a dramatic falloff in participation. This lead to a positive feedback loop where the highest played levels STAYED the highest played levels, and new content had a hard time breaking through.
  • The quality of content during the internal Alpha was lower then they were hoping. Content quality during the semi-open Beta was significantly better then they expected. Moral is that you can’t rush into decisions about community, sometimes it takes a while for things to develop.
  • During Q&A I stammered through a question about the priority of Share compared to Play and Create. They admitted that it didn’t get nearly the attention as the other two, because Create was operational so late, and they didn’t ever come up with a method of testing Share in any meaningful way other than just doing the betas. No way to do it without just trying.

I enjoyed this talk, and a good bit more than some other people I talked to. It was fairly unstructured and it did come off as a bit unprepared, but I thought the content was clearly worth it. I found the details about their development process and various mistakes very useful. It was definitely worth my time, especially as someone who’s interested in UGC.

Posted in Game Development, GDC 2009 | Tagged: , , | 4 Comments »

GDC09: Advanced Data Mining and Intelligence from Large-Scale Game Data

Posted by Ben Zeigler on March 27, 2009

Here are my notes from the GDC 09 session Advanced Data Mining and Intelligence from Large-Scale Game Data. As an overview, it was a discussion of academic analysis of Everquest 2 log data, co-presented by Dmitri Williams from USC and Bruce Ferguson from Sony Online Entertainment. I very well may have written something down incorrectly, so feel free to correct me in the comments.

Research Process

  • Data analyzed came from a combination of database, chat, and gameplay log files, as well as survey data collected directly be the researchers. Data had to be cleaned of personal information, but still collatable. Survey data could be combined with log data to compare reporting to actual behavior.
  • An in-game item was provided as incentive for completing the surveys, and this proved to be more effective than more traditional small cash payments for research participation.
  • Data was analyzed by a team of one half-time coder, and a group of 20 researchers with varying degrees of involvement.
  • Storage needed ended up being about 3x the data set, due to analysis techniques. Analysis was performed on beefy machines using some custom code on top of SQL databases.
  • The NSF and US Army provided most of the funding for the project, a total of $1.5 million. Rationale for funding was to study how this could help team dynamics.

Demographic Results

  • In general, play time increased as age increased. Much of the time spent playing EQ2 appears to have come at the expense of Television, and not at the expense of social or other activities.
  • Female players were fewer (80-20), but tended to play the game more on average, and enjoyed their time more. However, they tended to under report play time more (3 hours under reported vs 1 hour for males)
  • Hardcore roleplayers took up about 5% of sample size. In general they tended to be unhappier vs non roleplayers, but this appears to be explainable by the fact that a larger percentage are from marginalized social groups, such as those with disabilities. Unhappiness seems to lead to role playing, instead of the other way around
  • The data was useful for basic economic analysis. Because they had access to 100% accurate measures, it was good for studying the relationship between things such as GDP and Price stability
  • Group size and performance was measured. Solo play was 16% of content, with average XP of 68 per content-unit. 6-player groups was 23% of content with average XP of 78. Average XP was lowest for 4 player teams, despite them still being fairly popular. This could be a gameplay balance problem, and is worth looking in to.
  • Political leaning was studied. Players with more moderate political beliefs generally ended up in better groups than those with more extreme beliefs, plausible that extreme beliefs could alienate group members.

Gold Farmers

  • Just started on research into Gold Farmers. Research was requested by Sony, and they have not done any explicit study of the actual harm of Farmers.
  • 4 Types of accounts: Collectors who get the money and materials, Mules who hold it, Spammers who advertise, and Traders who do the final transaction. Each had different properties
  • As an early filter, many gold farmers were from Alaska or Antarctica (first in list). Simple guessing could filter out a large %
  • More advanced methods were discussed (Regression analysis, Brute force pattern matching, Network analysis) but nothing conclusive was available because research is still ongoing.

Overall, I quite enjoyed the talk. The mix of academic info and practical knowledge was perfect for me, and the speakers were engaging. It was a good use of my time and I will seek out any future talks on this subject.

Posted in Game Development, GDC 2009 | Tagged: , , , , | 3 Comments »

GDC09: Plans For Awesome

Posted by Ben Zeigler on March 22, 2009

This week is GDC week, and I’m looking forward to it. I’m heading up Wednesday and Thursday and have a conference pass (Friday’s sessions looked a bit light, and there’s a lot to do at work), so I’ll be packing in as many sessions as possible. I’m going to skip the keynotes, because there normally isn’t anything you’ll get out of those that won’t be on kotaku later. Okay, Nintendo does like to give out free games at keynotes, so you might get something cool at Iwata’s. Here’s my preliminary schedule:



My plan is to blog my notes from the sessions I end up going to go, so look for that next week. Oh, and if anyone knows where all the cool parties are this year, drop me a note.


I posted links to my notes from the talks I attended up above. Hope they’re helpful

Posted in Game Development, GDC 2009 | Tagged: , | 1 Comment »

Is John Romero making Marvel Universe?

Posted by Ben Zeigler on March 18, 2009

I was reading kotaku the other day, and was interested when I read an article about Marvel’s new MMO deal. As some people may be aware, Cryptic Studios was at one point working on an MMO entitled “Marvel Universe Online“, so I was obviously curious where it ended up. The kotaku article mentioned that Gazillion Entertainment was the publisher, which confused me because I’ve never heard of it. VentureBeat has the story on the company as a whole, apparently it’s been in stealth mode as “NR2B Research”, and currently owns 4 studios: NetDevil, who developed Auto Assault and is working on Jumpgate and Lego Universe (who was previously thought to be an independent company), The Amazing Society who does some casual MMOs, Slipgate Ironworks which was founded by John Romero and is working on some super-secret original IP MMO, and “Gargantuan in San Mateo, Calif., making the Marvel Universe PC and console MMO game”. A quick googling will show that Gargantuan has absolutely no web presence, and the Gazillion site now confirms they were formed in 2009 in San Mateo, Ca. So, who is Gargantuan and how are they able to make a massive MMO like Marvel Universe from scratch?

BigDownload has more details on Gargantuan/Marvel Universe. Actually, that link is NOT the same link I saw last night when I was researching this. There was originally a different article available at this url: that had more details about the game. However, that article appears to now have been pulled for whatever reason. Anyway, on the pulled article, there was some more discussion about Gargantuan, and it listed a set of employees who previously worked at other companies and were now working at Gargantuan. I recognized several of the employees as people who were recently working at Slipgate Ironworks, which is ALSO located in San Mateo, CA.

So we have an entirely new MMO studio formed from scratch in San Mateo, Ca, part of the same company as a DIFFERENT studio that is also in San Mateo, Ca, and having employees from the previous company. This is based entirely on public statements on the web and is entirely speculation, but it looks to me like Slipgate has now split into two studios, one to work on Marvel Universe, and one to work on original IP (the Gazillion site confirms what Slipgate is working on). Now, we have no way of knowing which studio John Romero is working with (or even if he’s still there, who knows), but there is a DISTINCT possibility that John Romero is, right now, working on a Marvel Universe MMO.  Anyway, if you were curious about the random dance of setting up MMO companies, this may have been useful. I wish Gargantuan a lot of luck, entering into the MMO business, even with what sounds like a really solid staff, is not for the weak of heart.

Posted in Game Development | Tagged: , , , , | Comments Off on Is John Romero making Marvel Universe?

Archiving Virtual Worlds

Posted by Ben Zeigler on March 12, 2009

Via episode 2 of Robert Ashley’s absolutely brilliant “A Life Well Wasted” podcast (best described as This American Life for video games), I was introduced to the “How They Got Game” project at Stanford. Basically the goal of the project is to try to archive and document the history of Virtual Worlds. Instead of just archiving the software (which is largely pointless for MMOs), the focus is instead on things like in-game-replays, transcripts, videos, and etc. The goal is to treat Virtual Worlds exactly like you treat the real world, so people can look back in 100 years and see what the online world was like. An example would be the last moments of EA-Land (previously Sims Online).

Episode 2 of ALWW features a 10-minute except from an interview with Henry Lowood, one of the chief historians on the project. Robert also put together interviews with a Pinball archivist and a private collector of misc. gaming stuff, but I loved the Virtual Worlds section and was hoping for more. Luckilly, Robert just posted the full interview, and it’s great. Robert is a very competent interviewer and Henry is a compelling speaker, so it works out really well. If you’re at all interested in the social nature of Online Games, you should definitely give it a listen.

Posted in Game Culture | Tagged: , , , , | Comments Off on Archiving Virtual Worlds

Downtime Doesn’t Matter

Posted by Ben Zeigler on March 1, 2009

I was listening to this week’s episode of the Stack Overflow Podcast, and there was a bit about Twitter. Jeff Atwood mentioned how if you didn’t design your database properly, you could end up a reliability laughingstock, like Twitter. Joel Spolsky then sarcastically replied “Right, and now Twitter has disappeared and we all use Pownce.” According to conventional knowledge, the massive downtime problems Twitter had throughout 2008 should have killed the service and made people leave in droves. That obviously has not happened, and Twitter has crystallized as the leader in persistent-IM (to the point where I broke down and got an account). Why didn’t everyone stop using it?

The key is that Twitter isn’t a product, it isn’t an endpoint. Twitter is a communication medium, and is something you use to reach a community. This means that it’s hard to switch services once you’ve gotten a bunch of friends, because you’re locked in. However, I think there’s more to it than that because this kind of “lock in” tends to inspire way less hatred than say a 2 year contract. Because you become emotionally attached to who you’re communicating with, when the service works you’re greatful for it and see it as a facilitator. Also, unlike say a phone, you never rely on Twitter for critical life-changing communication, so it’s ONLY used to build communities and much less for purely practical communication.

What else matches the profile of a service that connects you to a community, but is not used for strictly practical communication? That’s right, MMOs and other online games. As I’ve mentioned before, I believe that subscription-based MMOs are uniquely good at building communities, so I think the Twitter phenonom goes a long way to explain something that has mystified me for years: Why did no one quit when WoW ran like shit? When I originally joined WoW, and to a lesser degree now, there were very long queues and heavy server lag when doing things like picking up items or trading on the auction house. But it just became even MORE popular during that time. You could argue that the awesomeness of WoW overruled the horrible effects of lag and downtime, but I think it’s more likely that the lag and downtime just wasn’t that big a deal. People were willing to stick around and work through it, because the community, and their characters, were worth it.

So, the question is: Can downtime and crash bugs ever significantly hurt an MMO? I think technical design can kill an MMO (for instance, if you were to build an MMO that didn’t have chat… that would be a problem), but my hypothesis is that as long as your client is functional enough to perform the basics of community building, downtime just doesn’t matter. I do think tech issues hurt Anarchy Online (they all happened at launch, before people could build communities), but that game still makes money. Age of Conan had a wide variety of code bugs, but they aren’t why the game is dying. The MMOs that have failed have all failed due to business (no market, most often) or design decisions, and as long as a game gets good enough to actually SHIP, the quality of the code behind it really doesn’t seem to matter.

Posted in Game Development | Tagged: , , , , | 3 Comments »