Double Buffered

A Programmer’s View of Game Design, Development, and Culture

Never Balance Cool Against Useful

Posted by JZig on June 29, 2009

One of the more important parts of most RPGs (both online and not) is having cool rewards, often in the form of items. Great loot design can elevate a repetitive activity to be extremely satisfying (Diablo 2 is the best example of this), or can drag down an otherwise well designed game. To have a good loot system, you need a decent pool of desirable items, and systems for allowing players to acquire the item they want. There are a bunch of interesting ways to grant access to loot (quest rewards, random drops, crafting, auction house), but all design effort put into those systems is a waste of time if the end result isn’t actually compelling to the players. So, what makes an item compelling enough to drive player desire? I think the real value of an item is a combination of two components: Utility and Coolness.

I’m using Utility in the mathematical sense. Basically, the question is how will a particular item help the player achieve their goals more efficiently. Many players (especially the higher-level players) are trying to min/max their character and all they care about are the raw numbers. If you don’t give them hard numbers, they’ll come up with their own (possibly flawed) ways of figuring out what the best gear is. Also, once these players have come up with a system for evaluating item utility, any item ability that is complicated or hard to quantify will be considered of dubious value. In short, these players like Stats. The higher stats an item has, the more they want it.

Coolness is a bit harder to quantify, and refers to a cluster of different things. First of all, there is the simple issue of how cool it LOOKS. Also, items that are specifically rare (ie special mounts in WoW) will be highly valued regardless of utility. More relevant to combat design is how fun an item is to USE. Items that have cool effects (procs, conditional bonuses, active abilities with a long recharge, etc) can vary the combat in interesting ways and keep a game fresh, even if they don’t necessarily improve a player’s overall efficiency. They can even encourage a player to try out completely new tactics, essentially creating a mini-game nested within a larger game. More casual players, as well as players who crave variety, are big fans of items with cool abilities. However, it is difficult to strike a balance between making situational effects useless and making them too powerful (where combinations of effects can break the balance entirely). If you do it right, players will appreciate the unique abilities but won’t be able to abuse the system.

Okay, so there are two reasons why items are interesting: the quantitative improvement offered by Utility, and the qualitative improvement offered by Coolness. They both provide value to an item, but the perception of that value depends heavily on the player. Hardcore number crunchers will devalue cool effects for not being obviously useful to efficiency, while casual players will devalue pure stats for being boring (I think World of Warcraft equipment has actually gotten significantly more boring over time as they cater to the hardcore more). Given that we have two dimensions of value that are difficult to compare, how do we build an economy around them? You could try to estimate the average economic value of stats and cool effects and attempt to directly balance them against each other. This seems to make sense, but I believe it is the wrong approach.

If you balance Utility against Coolness, what you end up with is a system where the people who want stats will pick the items with the best stats but no coolness, while the players who want variety will pick the items with bad stats but lots of cool things. This has two horrible problems. The variety seekers ends up with a set of complicated conditional effects but will be significantly worse at everyday efficiency, which means they’ll fall behind their friends at levelling and be irritated at the game difficulty. The stat seekers will end up with no interesting combat effects, which means they’ll get bored of the combat more quickly. Both players get what they think they want, but are more likely to be dissatisfied over the long time.

The solution is to not balance utility against coolness, but to scale them both up as the value of an item increases. As a basic example, a “common” quality item should have mediocre stats and be fairly boring. If your baseline is fairly boring it gives you more space for improving your higher quality items without having to get too crazy. Then, a “uncommon” item should always have better stats and have some sort of interesting conditional ability. All “rare” items should then have better stats and be more interseting than all “uncommons”. Put another way, instead of constructing an item from one pool of “item points”, you instead construct it from the dual currency of “stat points” and “cool ability points”, which vary per item level and quality. This system results in items that are valued approximately equally by both stat-seekers and variety-seekers (and the majority of players who are in between). This means everyone strives for items that are both fun AND useful, and so everyone is happy! Well, everyone who can pay is happy, and everyone else is looking forward to being happy. In the world of MMOs, that’s even better.

Posted in Game Design, MMO Design | Tagged: , , , , , , | 2 Comments »

The Players Are Wrong, But Listen Anyway

Posted by JZig on June 16, 2009

When you’re deep in Beta, or you’re just taking the unusual step of actively seeking out what people think about your game, you’re going to interact with direct user feedback. Now, unlike feedback that has been filtered through OCR (which sometimes improves it and sometimes obscures it), direct user feedback tends to be either VERY FORCEFUL or just a bit confused. Based on my observations of fellow employees it seems to be standard policy for a new developer to seek out feedback, realize that some of it is hateful or confused, and then conflate the rest of your player base with the minority (well, or majority depending). The human brain really likes to make those kind of assumptions, so it takes conscious effort to get over the impression that not everyone who disagrees with you is a worthless human being. It turns out that most of the direct feedback you get from users DOES have value, if you just know how to mine it. Anyway, here’s my informal guide to Actually Learning From Users, broken down into helpfully pedantic steps:

1. Note Context

The context of the feedback should have a large impact on your interpretation of it. For the MMO-playing audience this can be broken down into Official Forums and Email (except for those bizarre companies that think they’re a waste of time), MMO Forums (F13, Fires of Heaven, Massively), and General Forums (Kotaku, Joystiq, etc). Feedback on the official forums tends to come from either low post-count users with specific issues or high post-count users who are part of the forum community. MMO forums tends to be full of cynical posters with a high level of knowledge and distrust, but with the occasional great suggestion. General forums are full of people who sort of heard something about your game and are a good way to get a feel for how well your marketing (formal and word of mouth) is working.

2. Ascertain Motive

Once you know the context of a posting you can make a quick guess as to the motive of the poster. The first motive is the always exciting Trolling. From my personal experience this is actually pretty rare on official boards, but this happens a lot on the general boards. If the goal of a poster is to instigate conversation or argument without actually contributing anything, go ahead and ignore them. A related but much less evil motive is Conversation. These posters will have very high post counts and are an integral part of the community, but most are useless for feedback purposes because message board posting, much like Blogging, is all about volume instead of content. A small number of high-volume posters graduate past Conversation into Aggregation. These users (I was one at one point) actively enjoy collating feedback for developers, and some are very good at this. During City of Villains Beta I trusted the bug-aggregate threads far more then I trusted our Q/A department, sad to say. Past those, another motive is Confusion. These users are often new and have a lack of knowledge. Because of this they will often be ostracized by the other members of a community, but to a developer they’re invaluable. If a number of posters are confused by some part of the game, there are no plans to make it better, and your game isn’t Darkfall, you Have A Problem. Either the system isn’t being explained well enough, or more likely your system is just too damn complicated in the first place. If players don’t at least have the illusion of understanding a system, they’re going to be constantly second guessing their choices and be too worried to have any fun.

The last motive is Advocacy. These users have a very specific goal, which is to get something about your game changed. These are the trickiest to deal with, because the first instinct is to ignore people who want something as being selfish and self-serving. But this is only part of it. Users who constantly gripe about their class will always exist no matter how well balanced your game is. The usefulness of Advocacy feedback comes in specificity. If a player feels that his class is gimped overall but doesn’t provide detail, that post is largely worthless. However, if a player specifies that a specific ability seems underpowered, and non-regulars agree (regular complainers will agree that everything is broken), it’s worth taking a look at. The odds are good there’s either a design bug or misleading player feedback. Oh, and as for those players who REALLY care about the way that one CERTAIN cape looks and it NEEDS TO BE RED or else they’ll QUIT FOR REAL, those players should be humored but largely ignored. MMOs are so much about personal identity that it’s inevitable you will highly annoy detail oriented people who can’t have exactly what they want. The only people who threaten to quit are those who are highly invested in your game: you should worry about the ones who don’t give a shit either way.

3. Extract Value

Any feedback that passes the Motive check (specific advocacy, content aggregation, and confusion) contains some value to a game developer, and I refer to this as Valid Feedback. Now, I use Valid instead of Good for a reason. The posts contain useful information but there’s absolutely no guarantee that it’s correct information. For instance, a confused user will often make weird assumptions about the cause of a bug because of their lack of background. The useful kind will still report what confused them, but if you see completely illogical feature requests or feedback it’s probably from someone who is confused but doesn’t want to admit it (because of the social ramifications of admitting to being confused). In these situations you have to ask “Why would they say this?” instead of “What did they say?”. Let’s say a user angrily posts that you’re evil for removing his favorite power. Instead of ignoring him because the power still exists in the source data, you can check to see if something weird broke in the runtime to hide it from the UI.

This same principal needs to be applied to Advocates. When someone discovers something about a game that makes them unhappy, the average player will not do a very good job of figuring out why they’re unhappy. Instead, they will grasp around a bit until they find a vaguely plausible cause, and then flog that horse until it’s dead. This often results in a petition of some sort, many of which won’t make any sense. As a developer you can either ignore them for being incorrect, or you can embrace their opinion as being valid but misdirected. For instance let’s say a large number of players complain about the fact that you nerfed a particular self-healing power which was obviously broken and overpowered. You can say the players are selfish exploiters, or you can take a deeper look and wonder if the players are only upset because that broken self heal was the only way to compensate for the occasional player mistake during unforgiving combat. Players choose to post out of genuine frustration, but instead of reasoning why they tend to just rationalize.

4. Repeat

These rules are a bit verbose, but after reading a few hundred posts you can start to filter them really quickly. Most feedback will end up not being useful, which should give you enough time to think about what’s actually important. Basically, if enough sincere people post asking you do something, then there is very likely a real problem. However, the solution to that problem is unlikely to be what they originally asked for (although specific posters with deep knowledge and analytical skills are better at this then your Q/A department. Learn who they are). Extracting out the Valid feedback from the chaff is not a skill that comes naturally, but I think it’s invaluable for making a compelling long-term product. Directly following user requests and completely ignoring them are two different paths to ruin for an MMO, and the solution is to seek the middle path of interprating that feedback without being slave to it.

Posted in MMO Design | Tagged: , , , | 5 Comments »

“How We Decide” If a Game is a 9.5

Posted by JZig on June 8, 2009

A few weeks ago I was listening to episode 4 of Out of the Game, and they started talking about “How We Decide” by Jonah Lehrer. At this point I realized I had been given a copy of it as a present, and given that it’s a book on psychology endorsed by Shawn Elliott I put it at the top of my reading stack. I’m glad I did, because I quite enjoyed it. A quick summary is that it’s a more complete, better written version of “Blink” by Malcolm Gladwell. It combines a few really good anecdotes about quick decision making  (apparently when caught in a quick moving forest fire the right solution is to light a SMALLER fire directly in front of you) with an overview of current research into both conscious and unconscious decision making. I recommend it to anyone with an interest in psychology.

The book builds what seems to be a really solid framework for making good decisions in a wide variety of contexts. There are two main processes we go through for making decisions: emotional and logical. They’re good at solving different kinds of problems, and are very complementary. The unconscious brain is very good at pattern matching and evaluating statistical models, and presents the results as input into your conscious brain as “gut feelings”. However, it is bad at dealing with falsehoods or irrelevant information. The conscious brain is good at simulating outcomes to solve problems and at regulating emotion, but it is very capable of thinking too much and incorrectly overriding emotional inputs. The general formula is to use your conscious brain to filter information and monitor emotional state (the best decisions get made while you are in a moderately excited state, as opposed to entirely dispassionate or enraged), and then let your unconscious brain think about it for a bit. The one that “feels right” will more often than not be the right decision.

I wrote last year about psychology and Game Reviews, and there’s a study in How We Decide that directly supports my thoughts. in 1990 Timothy Wilson put together a study comparing the ability of college students to rate jams to the ability of jam experts from Consumer Reports. When simply asked to rate the jams, the college students showed a correlation coefficient of .55 with the experts, which is reasonably high and shows that the expert’s choices matched fairly well with the average college student. Then, Wilson asked a different group of students to analyze why they preferred certain jams using elaborate questionnaires and a wide variety of categories. This group of college students showed a correlation coefficient of .11, which is essentially meaningless. After further study it turns out what happens is that the jam “reviewers” would try to describe individual components, such as “spreadibility”, that didn’t really affect their overall enjoyment of the jam. Then, as they evaluated all of these categories they tended to revise their preferences to match with what they had written in the review. The reviewers had overthought the problem and in the process had modified their initial preferences to match their specific analysis, as opposed to analyzing their preferences.

A similar study was performed by Wilson with paintings (a Monet, a Van Gogh, and 3 humorous cast posters). One group of women was just asked to choose their favorite painting, and 95% chose the Monet or Van Gogh. The second group was asked to explain why they liked the poster they chose, and that group was split 50% between the fine art and cat posters, because the cat posters had more content available for explanation (the subjects were not trained artists).  The book goes through a litany of other studies, all showing that when you try to carefully think through a complicated decision you end up making poor choices.

Let’s see, these studies are all about situations with a complicated decision and the need to generate explanatory content. In all of them, people who explained their decision before making it made worse decisions. Yeah, that sounds a lot like game reviews. If we assume that game reviewers are trained experts (most are) who have consciously trained themselves to be a good judge of games, and are not influenced by extremely strong emotions at the time of rating, their initial gut assessment of a review score is likely to correlate very strongly to the actual enjoyment of a game. However, if a reviewer rationally dissects a game into components they will be likely to rate a game higher or lower then it actually warrants, because it has “bad replay value” or something. So, game reviewers, stop thinking so much about the game and just pay attention to your emotional state while you’re reviewing a game. You’ll make better decisions.

Posted in Game Development, Random | Tagged: , , , | Leave a Comment »

UFC 2009 Undisputed: Actually Very Good

Posted by JZig on June 1, 2009

I just finished a 2 hour session of playing random people over XBOX Live. This is the first time I’ve played for longer than 20 minutes against complete strangers, and the reason I kept playing was that I was playing UFC 2009 Undisputed (UFC being the Ultimate Fighting Championship, a Mixed Martial Arts competition brand). It’s a licensed game published by THQ and not developed by Relic, which means that by previous evidence it should be a flaming pile. Despite all of that, it’s actually very good. It’s my favorite non-traditional fighting game (traditional fighting games being everything derived from Street Fighter 2) and although it isn’t perfect, it does some things better than any other game I’ve played. There is no game quite like UFC 2009, and in today’s world of refined experiences it’s great to see something new.

The inspiration for this game came very clearly from the UFC PPV/television product. The first components to the quality of UFC 2009 is presentation. First of all the fight trappings are spot on. The graphic packages are identical to what you’ll see in a real UFC fight, and the recorded audio announcing by Joe Rogan and Mike Goldberg is the single best announcer track I have ever heard in a game. It seems to use a mix of existing commentary and specially recorded bits, and it meshes quite well. UFC 2009 has really reminded me why presentation matters in games: it sets the emotional context for your actions. The fighter models and animations are great on average, and they mostly did a good job on the likenesses. I worked my created character up the Middleweight division (losing a few heartbreaking matches to last minute submissions) until I finally fought Anderson Silva for the Title. Bruce Buffer announced that I was challenging for the title, the announcers talked about previous Anderson Silva title matches, and I felt pumped. After I managed to knock him out with a solid uppercut from the clinch, and they replayed the highlights of the match, I felt extremely satisfied. If you’re familiar with the UFC product the presentation of this game will get you in the exact right emotional space to really care about what is going on, and there is nothing more you can ask from graphics and sound.

In addition to analyzing presentation, it’s obvious that the developers (Yukes) spent a lot of effort analyzing the flow and composition of real UFC fights and faithfully converting them into gameplay mechanics. The main components are striking, submissions, and transitions. For striking, they decided to keep it simple, and map each limb to a face button, with directional input to modify the attacks. Attacking without modifying gives you safe strikes that will do some damage, while modifying it makes your strikes more damaging but ALSO more dangerous for you. Best of all, the damage modeling uses realistic physics and collision detection, so strikes do damage based on momentum changes of body parts. The primary way you get knocked out in the game matches exactly with real life: Moving forward for a strong strike while your opponents hits your exposed area with a strong counter. You can also end a fight by hurting them enough that they get knocked down and stunned, where you finish the fight by standing over them and punching until the ref stops you, which is also exactly like real life. Submissions are fairly simple. In most positions you can attempt a submission at any time by clicking the right stick (on defense you have to grab an arm as a counter), and you will then button mash against your opponent. The interesting thing is that your current stamina (which goes down when you attack) has an effect on this, so it’s quite common for a fighter in the game to get overzealous in their striking and lower their stamina, at which point they are more vulnerable to a submission. Again, this is exactly like real life and the submission and striking components directly counteract each other as ways to win a fight.

The other major component is the transitional/positional game. Previous UFC and Pride games had fairly decent striking and submission components, but the positional game in UFC 2009 is extremely deep and satisfying. There are 3 standing positions, 4 clinching positions, and 12 (I think) ground positions. This sounds complicated, but the rules for each type of position are very similar, and they form clear progressions. Basically, each in each position you can actively choose to strike, go for a submission, or attempt to transition (by rotating the right stick). Also, you can defend against strikes or transitions. Striking an opponent who is guarding against transitions is very effective, as is transitioning against an opponent who is blocking strikes. So, the whole system quickly turns into an extremely psychological guessing game. Should you go for strikes that do long-term damage, or do you transition to mount to try and win the fight right away? Or you can block an opponents transition to make them waste stamina and become vulnerable to a transition.  There is no other game I can think of that places this much explicit emphasis on positional control and transitions, and learning this part of the game will definitely give you a deeper understanding of the actual principles behind MMA fighting. The fact that the combat is deep, systematic (as opposed to being overly based on special moves), and maps very closely to actual MMA fighting is why I love the game.

Outside of the core fighting mechanic and presentation, the game is a bit more hit and miss. The career mode is pretty comprehensive and gives you a fairly fun time-management layer to deal with when training your fighter. However, it features some of the most irritating menus in the history of gaming (when it auto saves, which is like every 2 minutes, it pops up SIX different save dialogs, all with slow transitions between them). The Online play is very playable (the slower pace and higher strategy of the game work well online) and has some nice ranking features, but right now an irritatingly large number of people will disconnect immediatly before losing (I have lost 3 wins to this, to add to the 7 I have that registered). These are issues that can be fixed in updates or next year’s version, but the core game is absolutely worth it. Like few other games in this generation of consoles, UFC 2009 Undisputed is a game that could not have existed 5 years ago (when the last UFC game came out). Improvements in physics have lead to a very realistic striking game. Improvements in graphics have lead to the ability to duplicate the presentation of a UFC fight without having to clutter it up with too many meters (do turn on the stamina meter, that cue is too subtle). The improved popular success of MMA and the UFC has allowed the developers to build an extremely deep combat system that can rely on lessons learned from watching real fights instead of having to copy existing fighting games. UFC 2009 Undisputed (name still kinda sucks) will definitely go down as one of the most unique and best-executed games of 2009, at least on my personal list.

Posted in Game Design, Game Development | Tagged: , , , , | Leave a Comment »

Valve Actually Made a Mistake

Posted by JZig on May 26, 2009

Since the mostly-fictional events of my last post, the Team Fortress 2 Sniper vs. Spy update was actually released. Well, before that the awesome details of the Jarate were released, including a sweet comic. After the masterful PR and hype, I was going into the update with very high expectations. And then the unthinkable happened: Valve made a moderately sized mistake, which is the first mistake I can remember them making since… Condition Zero probably.

In the new update they dramatically changed the method of unlocking new items. Previously you would unlock items by doing a certain number of class-specific achievements. So if I played a lot of Medic and did a combination of fun and tedious achievements, I would get to use new items. This system had some problems, namely that some of the achievements were poorly designed. Some were incredibly difficult to get naturally, which lead to players creating “Achievement Farming” servers were people could go and quickly get the difficult ones. The dedicated players who knew what they were doing could get the items in a few hours, while the uninformed masses might never see them. I agree there was a problem.

But the solution Valve implemented has serious issues. The achievement unlocking has been removed. In it’s place they added random drops. Now when you are playing TF2 you will randomly be awarded an item. The goal of this system, as explained by Valve, is to allow players of all types to acquire the class-specific items. They also wanted to discourage farming and encourage people to do what is actually fun, namely playing the game. These are admirable goals and I fully support the system in concept. However I am extremely frustrated at the current implementation of the system for the following reasons:

  • The drop rate seems too low and too random. I’ve played a LOT of TF2 since the update (around 15 hours) and have gotten a total of 4 items. On the other hand other people I have played with have gotten 10-15 items in the same period.
  • Out of the 4 drops I got, 3 of them were duplicates of items I already had. Valve has said that trading is coming, but until that comes in duplicates just feel like Valve is making fun of me.
  • It’s inconsistent per class. All previously-available items can still be unlocked by achievements, but the new items can’t. This means that even once you put in trading the economy is going to be screwed because 80% of items will have negligible demand. Duplicates of everything but the new items will still feel like an insult.
  • The actual drop process is not psychologically satisfying. In an MMO you loot things from dead corpses or chests. This is an inherently satisfying experience because there is a direct link between your action (killing something or finding a chest) and the reward. TF2 items are given to you at random intervals and not as a result of a specific action. Because of this the action-reward cycle is broken. The part of your brain that associates reward with choices doesn’t work at all on TF2 drops and thus they do not feel rewarding.
  • The process is very opaque. They’ve said it’s based on play time, but no one can figure out the exact system. Some players get them by remaining in spectator mode, which has lead to a lot of players sitting in spectator in the server I frequent. When it comes to anything involving random chance, the human brain tries to find patterns. Everyone is going crazy trying to figure out how it actually works. Combining a low random chance and an opaque system leads to an extremely frustrated brain as it tries to grasp at straws.
  • I don’t have anything to strive towards. With the achievement system I could plan out my play and work with a set of sub-goals that I know will lead me towards my ultimate goal. Now I have no control over getting items. Things just happen. When they don’t happen to me I don’t have an explanation, so I just get bitter.

The current system really doesn’t make any sense from a psychological perspective. They’re “rewards” but don’t actually feel good to get. They’ve stymied my free will by making all of my choices pointless. I’m actively discouraged from playing TF2 because I feel like I won’t get anything good THIS time either. This is all very unlike Valve, who normally think through the psychological impact of their decisions. I think a clue to the background to this decision lies in the following bit from the blog: “For the last number of months we’ve been working on using the Steam Cloud to store a player’s inventory. With that finally in place, we were able to deploy a new system focused on the giving of items to players.” This means that Valve has been wanting to do this for a few months but were unable to do so because of technical issues. Having seen this happen before on other projects (ask me about CoV design), my guess is that when the tech was finally done everyone was tired of dealing with the irritating system that never got finished, so it got sent out a bit half baked. The fact that it wasn’t talked about in the time leading up to the release also hints that people were internally frustrated with the whole thing. Anyway, I’m sure they’ll fix many of the issues I pointed out above, but I think I’m done with TF2 until they do.

Posted in Game Design, Game Development | Tagged: , , | 1 Comment »

Everyone At Valve Has Been Fired

Posted by JZig on May 19, 2009

Last week Valve announced the next free update for Team Fortress 2, which would update the Sniper class. Things started fairly normal, with the announcement of a Hunting Bow and a new map type. On the third day they announced a second upgrade for the sniper, a shield that prevented one back stab per life. This leads to a fair amount of complaining from players of the Spy class on message boards across the internets. They were bitter that the sniper was given an ability that directly countered them, when spies were the explicit counter to snipers in the first place. Around this point a few players noticed something subtle: the page for the shield update had changed a few times over the day, and at some point included a picture of a de-cloaking spy directly behind the Sniper. Conspiracy theories spread, but people thought Valve was just screwing with the hardcore fans, which they’ve been known to do before.

Finally, from out of the mist of confusion, the Spy Update backstabbed the Sniper Update. For once the conspiracy theories were right. The spy gets two new items (a decoy, and permanent invisibility if you don’t move) and a HILARIOUS spread of products from MANN CO. (”We Sell Products And Get In Fights”). I very much want a camera beard, as I need to infiltrate Canada shortly. The Spy Update reveal was pulled off in a clever way that very much fit the attitude of the game. I figured that was enough excitement for one update.

But then late Saturday night Valve accidentally leaked Meet The Spy. Apparently they accidentally uploaded it to their youtube stream and did not properly set it as private. The error was quickly discovered, but in the meantime a few people had managed to download the video and it instantly spread around the internet. It spread so quickly because it was a damn fine short film (that you should watch in HD, link later). It features slapstick comedy, dramatic fight scenes, and the absolute best “Your Mom” joke I have ever witnessed. Valve obviously had planned to release the video Monday, but their thunder had been stolen by an accidental leak. Most companies would go down one of two roads: clamp down harshly on the leak and try to stop distribution, or go on with the big announcement as if nothing had happened.

Instead, over the course of today Valve decided to take what had happened and fucking run with it. The official release of Meet the Spy was modified last minute to add a reference to the leaked video (the hilarious alarm board now includes a reference to “Leaked Video” as well as a shit-ton of other references). The Spy achievement also list includes a bonus “Valve Corporate Achievement” entitled “Welcome to the internets: Fail to understand what ‘Private’ means on YouTube.” In case that wasn’t enough, Valve’s Robin Walker vowed to get to the bottom of the leak before working on any other games. He then fired Marc Laidlaw and several others. Finally after wearing down Greek intern Stavros Xanthis, Robin declared the case officially Closed. Everyone at Valve has now been fired, and we will also miss everyone who gets hired tomorrow and then fired again. It is unknown how long Robin Walker’s tyranny will stand, but rumors of him being Red Spy are just speculation. We need to find out if he speaks French and enjoys older women from Boston.

The lesson to be learned here is that when something screws with your careful plans, you take control of that thing, warp it to your every demand, and channel it into a concentrated stream of Awesome. That is how you do PR.

Posted in Game Culture, Game Development | Tagged: , , | 3 Comments »

Solving an Impossible Puzzle: MMO Controls

Posted by JZig on May 11, 2009

My last post discussed in the abstract the various benefits of a substantial MMO Beta, and I now have a very concrete example from my own experience. Our game is at the point in development where all the major features are in, but there is still a lot of work left to make the game fun and polished. 2 months ago one of the areas that needed the most help was the combat controls. It’s something I care a lot of about personally and enjoy doing, so I volunteered to jump on that particular grenade.  It’s a tough job because controls are subjective, involve the interaction of multiple software systems, and critically affect the final game experience. No one can agree when controls (which include targeting, ability use, and camera) are good enough, but everyone knows when they suck. After spending 5 weeks on them, I can confidently say that the controls Don’t Suck. I’m not claiming to be an expert, but there’s not much documentation on the subject so I thought I would share some thoughts on the process.

Gather Intelligence

Because Controls interact with many different gameplay and design systems, I knew I had to do some research before starting to code. The first part of this was to see what we already HAD. I had been playing the beta for months and was aware of previous attempts to design the controls, and after looking around I saw there were remnants of  multiple control schemes implemented over multiple years. There was a lot of good code and many good ideas, but only one actually working control scheme, that had some issues. Additionally, there was a 90% complete back end for switching control options per-character, which wouldn’t require much modification. There were a lot of useful bits and pieces, but nothing resembling a controls system.

The second part of research was to figure out the goals of our controls system. There were 3 major sources of input here: what I wanted, what other developers wanted, and what the end users wanted. I’d been thinking about combat controls a fair bit so that part was easy. To represent the desires of other developers, I tracked down design docs and email threads about the older prototype designs and broke down what the goals of each were. It turns out everyone was trying to make the perfect default control scheme, but no one had quite gotten there. It’s normally fairly difficult to get direct feedback on controls from end users, but the game was in a beta so I just asked them. I collected the existing bug reports about combat controls and read through a few hundred complaints on the beta forums. The primary user complaints were the fact that they often accidentally attacked non-threatening objects while they were being killed by enemies, that it was hard to predict what the targeting would do, and that combat on the mouse/keyboard took more hands then they actually had. These were serious issues that had to be addressed, but for the time being they went in the back of my mind.

Prepare for the Unknown

After my week of research I now had a clear idea of what my resources and requirements were. I could have immediately started coding up the perfect targeting system that would magically satisfy all end users and fellow developers, but I didn’t.  First, I’m smart enough to know I won’t be able to come up with the perfect solution to any problem as complicated as combat controls. Second, several talented developers had already tried and failed to Solve the Puzzle. I knew that I had to redesign our code to deal with the fact that there was no perfect solution. It had to be designed for flexibility.

My first step was just to remove all of the broken stuff that obviously didn’t work. There was a fair amount of this left over from previous prototypes and removing cruft makes modifying the rest much easier. Then, I hooked up the previously-written control options back end to an in-game UI. This way it would be extremely easy to tweak individual settings without having to modify code or a data file. Finally, I moved all of the existing control options into the unified options back end. Previously they were modified by changing data files or running specific in-game commands, but now they were all modified in the exact same place. Making these changes took another week and didn’t add any new functionality, but all of the existing functionality was hooked up and ready for iteration.

Try Stuff, See What Sticks

At this point in time, I had a long list of suggestions and complaints and a system that allowed me to easily try different combinations of options at run time. I prioritized the list of suggestions (the ones I hadn’t already fixed), and just started implementing them. It only takes an hour or so to add your average control feature, which is less time then you’d spend arguing over rather it’s a good idea or not. I put in the ideas that I thought were good, the ones I wasn’t sure about, and the ones that might lead somewhere new. Important additions were the ability to specify the order of tab targeting, to change the priority of different types of targets, to automatically select or deselect on various conditions, and to modify the camera based on various actions. In between adding new features, I was constantly trying them out and seeing how they felt.

After playing around with a few odd combinations I stumbled on some pretty cool ideas. If you enable the option to show your automatic target, disable the conversion from auto to selected target on attack, set the target priority to be whatever is closest to the center of the screen, and set right click to mouselook toggle, you end up with a playable hybrid MMO/FPS control scheme. Likewise, if you enable the ability for melee attacks to ignore out of range selected targets and set the target priority to nearest, you end up with a hybrid MMO/Brawler targeting scheme that ends up being really fun for melee classes.

Publishing your First Draft

After a week of iterating personally, the control scheme options were coalescing into 3 clusters: the traditional MMO cluster that focused on precision and predictability, the Action cluster that focused on smart automatic targeting, and the Shooter cluster that used mouselook-based targeting. These would be the initial presets that players would have access to, so I made a best guess at initial settings for the two new schemes. I set the default scheme to be the same as the old settings so I would be able to compare the feedback from people who modified their controls to the control group of people using the old settings. I posted to the boards explaining the design goals and let the options loose on the Beta community. The feedback was illuminating.

That week the feedback fell into two groups. Players who had discovered the options posted generally nice feedback, but that was overwhelemed by the feedback from people who had heard there was control changes but hadn’t found the options menu. These players complained bitterly about the lack of something that was now available. There was literally no one who preferred that option to be off, so I enabled it by default and removed the option from the menu to avoid confusing people. Several other options I had added ended up being confusing and useless, so I removed those. Others fit into a middle ground where some people liked them and some didn’t, so I left them disabled but kept the ability for end users to override them. Using the feedback from that week’s beta testing, I made a bunch of modifications to the defaults for next week’s test.

Polish and Iteration

The feedback after I updated the defaults was significantly improved. There were significantly fewer negative comments on targeting than before. There was still a lot of specific feedback about missing options and confusing text, so I knew my job wasn’t done. By having week-long iteration cycles with direct player feedback, I was able to very quickly focus on what else needed to be done to make the controls system as good as it could be. After two more weeks of changes and feedback, the iteration cycle was over and I was happy.

6 weeks ago, the combat controls were one of the most commonly argued about features on the beta message boards and were a source of significant player rage. After last week’s playtest I took a look at the feedback threads and there was not a single negative comment about the combat controls, which shocked me. There were also very few positive comments. This could be because players don’t really like them, but lack of discussion is really the ultimate goal of controls: if no one notices the controls, they’re perfect. I’m sure the combination of rising player expectations and bugs will cause future complaints about controls, but it’s very satisfying to dramatically improve the experience of thousands of players over such a short period of time. That’s why MMOs need Betas, and that’s why I like working on them in the first place. It’s not about solving puzzles, it’s about making (virtual) life better for real people.

Posted in Game Development, MMO Design | Tagged: , , , | 1 Comment »

What is Beta Testing Good For?

Posted by JZig on April 21, 2009

A few days ago, TenTonHammer posted a comprehensive article about what Beta Testing means to MMOs. I got it off Scott Hartsman’s twitter, and it has a lot of quotes from both developers and users, regarding what Beta really means these days. To summarize, both developers and players generally agree that Beta is less about gathering player feedback, and more about marketing. I will 100% agree that OPEN Beta (or pre-order Beta, whatever) is entirely about marketing and load balance testing. A month out from release is too late to really do any significant changes to the game. However, at least at companies that are open to feedback, a fairly large Closed Beta is required to make a high-quality product. Other than the obvious benefit of having players do free Q/A, here’s why Beta is vitally important:

  1. Gets your Community up and running. You can’t start a community from nothing, so if you want a solid and supportive community at launch, you need to put in a lot of work before launch. A big part of this is getting together the right community team, and there’s really no way to figure out what will work without just TRYING it. So, during Beta both the OCR team and Devs can directly interact with the players and set up a rapport that will hopefully carry through launch. Both the CoH and CoV betas had good developer-community communication, and I definitely think it followed through past launch. For this reason, I think using different community tools (*cough*Warhammer*cough*) for Beta and Live is a stupid idea.
  2. Helps resolve developer choices. The development of an MMO involves a LOT of people, and reasonable people can disagree on various features and design decisions. Sometimes it works out that it’s fairly easy to implement two versions of a feature, but everyone knows that a choice has to be made before launch, for either player confusion or implementation issues. If the developers structure it correctly, it can be very helpful to use Closed Beta as an extremely large focus test. So, you can give players a set of options and see what the reaction is, both via data mining and direct feedback. In cases like this, the players in Closed Beta will have a direct effect on the final product, and I know that’s happened before at Cryptic.
  3. Tracks down compatibility issues. This is kind of boring but absolutely crucial. MMOs are the most complex type of video game, and PCs are stupidly complicated. This means that you’re guaranteed to have a shitload of obscure technical conflicts, and you’re going to have to spend incredibly tedious hours resolving 90% of them (you never fix all weird technical conflicts, you just have to live with that). It’s important to make sure your Beta includes a very wide variety of computer setups, and inevitably it means that some percentage of you users just won’t get to play for a few weeks. I have to say I was mildly irritated by some of the user comments in the article on this subject. Very likely the reason the client doesn’t run on your computer (if it does run on most other people’s) is that the PC platform is stupidly complex and prone to failure.
  4. Drives your post-launch content. By the time you get to large scale Closed Beta, it’s generally too late to make dramatic content changes in time for launch. However, this does NOT mean that user feedback during Beta is useless. Remember, an MMO tends to change a lot in the first big content release after launch. This is because the feedback from Beta users ends up directly affecting the design decisions and content additions that go into content that comes out 2-3 months after launch. For instance, a LOT of the content in the 40-50 level range in CoH (and CoV) was directly built in reaction to shortcomings that became obvious during Beta testing. Again, this is an area that Beta players had a direct impact on the future of the product.

If you don’t run your Closed Beta with an eye for gathering feedback from testers, you’re going to end up with a game that has a disconnected and isolated community, unpopular design decisions that the developers dont particularly love, a cavalcade of horrible compatibility issues, and no plan for post-release content updates. If you run your Beta as an advertising campaign with incredibly strict moderation and no developer feedback you’re taking a huge risk. I could name a few recent MMOs that have had large initial launches and exactly those kind of post-launch problems.

Posted in Game Development, MMO Design | Tagged: , , | 7 Comments »

2008 Game Developer Salary Survey

Posted by JZig on April 10, 2009

I opened up this month’s Game Developer, and it’s salary survey time again! I thought I would take a look at the numbers and compare them to last year’s survey. First of all, the tone of the piece was a bit different. Jill Duffy, who wrote the article and was a senior contributing editor, mentioned that she was being laid off in the intro to the piece. Despite that depressing start, salary numbers lag a bit behind the economy, so wages were actually up this year. Next year is the one where we’ll really see the fallout. Most of the trends are basically the same as last year, so no point in going over the general trends. One thing worth noting is that the US sample size was only 1,879 this year (down 600 from last year) so statistically it’s a bit more suspect.

One area I wanted to revisit was trends in gender pay equality. Here’s the 2008 gender pay equality chart:

Field Percentage of Females Female Compensation Difference
Programming 3% $10,249 less (12%)
Art 10% $8,456 less (12%)
Design 6% $18,324 less (26%)
Production 21% $10,079 less (12%)
Business 14% $28,704 less (26%)
Q/A 14% $7,500 more (+16%)

Right, so, except for Q/A (either the demographics of Q/A changed dramatically in the last year, or something weird is going on statistically) and to less extent Business, this is pretty bad news. The gender disparity in Programming, Art, and Production rose by 3% which is disappointing. The gender disparity in design rose by a ridiculous 11%. Female game designers now make $18 thousand less a year than male game designers, and game designers don’t really make that much on average. All of the trends I mentioned last year about females having a hard time getting top-level design jobs seems to be even worse this year. I find that more than a bit depressing.

Posted in Game Development | Tagged: , , | Leave a Comment »

Why OnLive Won’t Work

Posted by JZig on April 5, 2009

So the big topic of general interest at last week’s GDC was OnLive. Lots of people, both gamers and developers, were excited about the possibility of playing PC games without a PC. It’s supposed to solve piracy and system requirements issues in one go. Well, I tried it out in person at GDC, and I can say conclusively that OnLive’s plans will never work. But, it’s not for the reason you may think. The technology is viable, it’s just the economics that make no sense.

There was a bit of a line at the OnLive booth, but I waited patiently before I got to play BioShock over it for a few minutes. The experience was actually better than I had expected. The image quality was perfectly fine in my opinion, although it got worse when you moved the camera rapidly. There was noticeable lag, but it was still playable. It felt almost exactly like playing an FPS on too-high settings. Basically it was like playing BioShock on high quality settings at about 15 frames per second. It’s not an experience I would consider paying money for, but I can see how it would be acceptable for slightly less picky gamers. So, OnLive can successfully play games remotely 50 miles away from San Francisco to Santa Clara (where the servers were), using a dedicated line, and to a quality level acceptable to an average user but not to the hardcore. That’s actually a pretty impressive accomplishment.

But it isn’t enough to actually make money. Time to do some math. According to what they’ve said publicly about their business plan, they plan to stream games over the general internet, as long as you are within 1000 miles of a server (1000 miles would be significantly worse lag then I experienced), and have a 1.5 Megabit sustained connection (5 Megabit for HD, which we’ll just ignore because it’s even sillier). Ignoring the issue with ISP bandwidth caps (which could be a serious problem) this is mostly reasonable. However, it leaves out a big part of the equation: the servers and bandwidth on OnLive’s end are not free. In fact, they are very expensive.

Thick-client Western MMOs are a space I am reasonably familiar with. Out of your $15 subscription fee, somewhere around $5 goes directly to hosting and bandwith (the rest towards development, support, and licensing). MMO bandwidth is split between patching (let’s say 1 Gigabyte a month per player on average) and runtime (usually pretty light, 100 kilabit/s MAX). At 50 hours of play a month (which is about average) that ends up being around 3 Gigabytes of bandwith per month. On the other hand, 50 hours a month of 1.5 megabit streaming is 30 Gigabytes per month. So, bandwidth costs for OnLive are 10x what they are for MMO hosting. The other part of the cost of running an MMO is purchasing and powering servers. An average MMO server can host a minimum of 50 concurrent players (a lot more in the case of games like WoW which are light on server use). These are fairly buff machines (but not special in any way), so let’s say that either you could buy 5 cheap desktops for that price, or host 5 client instances of OnLive games, giving a maximum concurrent OnLive users per server at about 5. This means that OnLive will need to buy and run 10x as many computers as an MMO.

The two components of hosting are both about 10X worse for OnLive than they are for MMOs (and that’s a very generous number). Multiplied by the wholesale hosting cost of $5 a month for MMOs, this gives an estimated wholesale cost per user of OnLive to $50/month. Remember, this is wholesale and before they make ANY money off of this. So assuming a minimum margin of around $10/month, anyone who wants to use OnLive is going to be paying at least $60/month for it. This is why they haven’t announced pricing yet, because they know what the reaction would be. So, the audience for OnLive is people willing to pay $60/month for pc gaming, but not hardcore pc gamers who care about latency.

This business plan doesn’t make any sense. On the high end, it fails miserably compared to Steam, which is a lot cheaper and more satisfying for the hardcore. On the casual end, why pay $60/month when you don’t care about graphics and are happy playing browser games? There are only two ways to make this work: either contract directly with ISPs (which would negate the bandwidth costs) and get it subsidized through already-expensive cable bills, or sell the company for lots of money up front. Considering that OnLive was created by Steve Perlman, who is the same guy who sold the largely useless thinclient technology WebTV to Microsoft for $425 million, I think it’s obvious what the corporate strategy is: Get bought before you have to launch the service and hope no one digs too deeply into the realities of the business plan.

Posted in Game Development | Tagged: , , | 5 Comments »

« Previous Entries