This week's column looks at how developers really gather and use the feedback of their players. Sanya's peek behind the development curtain appears every Friday.

The flaws with user initiated data are many. The feedback is limited to those members of the playerbase who are highly motivated to post. The term “silent majority” isn’t a cliché for nothing, y’all. For example, every class has its group of people screaming that their Foozle is underpowered. When no one of a particular class is screaming about their Foozle, it is drastically overpowered and the entire class should probably be removed from the game.

There was a point early in City of Heroes where I went to the forums to look something up about my class, and I noticed that the forums were eerily silent. No activity at all. My spidey senses went haywire, and just for yucks, I checked the other class forums. Indeed, they were all packed full of people complaining. Experience told me that either my class was so awful that no one was playing it at all, or I was going to get nerfed down to the atomic level. The hammer fell two patches later. Owie.

Read more here.

Good read.

I liked the story about the golf course - the field of green that inspired imagination, even if the original implementation was incidental.

I would heavily agree with your call that developers should be more open as to how players' feedback has been handled.

I don't believe I've ever had a "blue" as we call them on the Blizzard Forums, look by my threads and say anything. But oh well.

Sadly writing this won't change a thing. The people who post on these forums and think the devs should only change the game based on their particular feedback will just miss the point completly.

The truth is people need to stop thinking they are so important. Even if a game is smaller then WoW, has a hundred thousands players or so, you as an individual are not important. So when you are mad that a dev didn't take time to respond personally to your post you have no need to get upset. They might have read it, they might not have. They might have agreed they might not have. They may have datat to support it, they may have data to oppose it. But they are never required to respond to you personally.

I also don't agree that release notes need to be highlighted with what was changed due to different forms of feedback. The changes are there for a reason, if you are an intelligent enough person you can figure out the reasons. The devs don't need to hand out cookies by highlighting something and saying "See we changed this because you talked about it". That's just feeding back into the player mentality that "I as an individual are the most important person here, and the devs should listen/respond to my every word".

 Reactive communication is vital in the current MMO world. Just take a look at FunComs own AoC boards or even better at their test boards.

The developers interact frequently with the players. They talk about difficulties and why they do what they do. All stuff that appeared after AoC got its new General Manager.

The result is that yes people still complain but there is a general good mood there and the complains are often focused and productive. A completely different world from the days when funcom just stonewalled and only send out sugared rose colored press statements from time to time that had no relation to the game experience the players had.

Sometimes it takes a change of leadership to improve things. I remember in DaoC where there were key elements of the game that people complained about forever and the players never ever got any decent reply (you could almost feel Sanya's frustration in her words on the Herald and boards back then). Then Mythic began to develop WaR and a lot of the big shots got moved to that project leaving a new dev team on DaoC and within 3 months you saw issue after issue the people had complained about for years be solved one by one and even today the game is still being played by dedicated fans.

In relation to Sanya's talk about the golf course and the lack of a similar area in the new enviroment, I remember suggesting a big neutral island without keeps in the middle of the ocean as a playground for that kind of play that I did on the IGN boards and 6 months later there were one . . . . . which then weren't  _that_ cool teaching me that idea's you think of yourself could easily not be that good and that is why you need many inputs and opinions on game elements before you implement them. :)

Player ideas are good but they need to get processed.

Also keep in mind that sending your feedback to Customer Support will most likely get you a "Thanks for your feedback!" standard response while your suggestions will end up in the CRM's digital bin as a single person's opinion will most probably be regarded as rather unimportant.

You will be better off with becoming part of a focus group (if such thing is in place), or with providing feedback on the game forums where other players and maybe community representatives may discuss it.

Developers definitely should at least get sharp-eared if there's "overwhelming  player feedback" on a certain topic, but then properly investigate (in-game data, in-game experience) before taking any decision that the silent majority might probably dislike.

A dev team with passion for their own game and a clear focus on what'll be good for it and what not, plus player feedback taken from the meta-game (e.g. forums, official or not) and from in-game (e.g. from focus groups, or from playing your own game) can be very successful in doing what's good for the game as a whole and thus for the majority of its players.

I instantly think Guild Wars. The ArenaNet guys have been doing a great job for that matter.
 

Just don't send your feedback to CS...

A good read Dana.

If I was running a MMOG I would be focusing on usage data.

I would not ask my Community Manager to collect data from forums, I would not host official forums myself, and I would very rarely get player information from Fanfaires.

Why? Because as you mention the vocal minority will stay both vocal, and a minority. Actions speaks louder than words.

I would not have an issue with mentioning the statistics in the change logs though, so that people could sit and be vocal on their guilds forums reguarly. But as long as the gameplay was decent and content would chug along nicely the minority likely wouldn't be going anywhere anyhow.

I would like to see feedback from the gamemakers. I would also like to see slight changes over the swing for the fences type fix.

I do post occassionally but never that often on a game I play. I enjoy playing then my complaints will be small. Most will only be about one power or ability that might not be working. I don't worry about being the best Class/sub-Class/damage dealer/tank/healer or anything like that. I want a fairly even playing field but don't expect perfect balance. I see expansions/updates that add new abilties and/or powers to classes and the balancing includes going from start to finish on the fix to a class. I will never understand that and never respect that. Doesn't mean that it is wrong but I sometimes think that needed perfection by the activists to be #1 is too much.

Great read!

Good ole Emain Macha

Good article.

I especially like the point about looking at all relevant data: not just what the folks are saying on the forums, but also what the "numbers" data is saying.  Many times, players whinge about this or that...even as they take full advantage of whatever it is they're upset about this week has to offer.

The fact is that, in my opinion and experience, players ask for things without considering what it is that they're asking for, and then push the issue on a game's forums until they get what they asked for...only to complain that what they got wasn't what they wanted.  It seems to me that, in many cases, some players (the "vocal minority") come across as wanting to be the only people who have a say in what happens to/with a game, without considering what the "silent majority" would like, or even what is humanly possible given the way a game is coded.  I don't expect too many folks will stop whinging after reading this, but at least Ms. Weathers has done a good job of putting some information out there for folks to consider.

After SWG, I'm very iffy about focus groups.

Using fanboys only or using people who barely knew what a PC was to get the feedback they wanted.

Instead of getting good feedback, they only got the feedback they wanted, the feedback that agreed always to their actions.

I dun trust focus groups any more, because of the ability to pick and choose only participants that fit the devs scheme.

Tho I have to admit, I was invited to participate in the focus group that OK'ed the NGE. But to participate, I would have had to pay dearly to get there, since SOE who claimed in their invitation "All expenses paid", suddenly realised that I was european...

But my invite came after I had been a good lil fanboi for a while on their forums... Well, that and I was part in a guild that did get dev attention and was used as a posterboy(guild) in SOE fan faires and gatherings.

Loved the read, but as an old DAoC player I knew exactly what you were talking about with the "golf course"...and you are sooooo far off base as to what "went wrong" with the replacement that I cringe to even read it....but that's another thread and one I would love to discuss over several pitchers of beer sometime.

A fair read, I don't know how dev's feel about the subject.

I generally believe that many developers do read forums regularly. I don't think that anyone is too small to respond to though. As far as a real response, well that is up to the person/s forming the response.

I know from experience about putting my personal thoughts out on a closed beta forum, and getting a personal response from dev's. Neither of them being necessarily good, even if my original intent was to throw up a red flag and try and reign some people in on what was reality. I wasn't shouted down by any other players, but I was told to be quiet by the dev in a personal message. It was odd to me that  I must have struck a nerve, cause there was no public response just a person message.

In the end I believe my views were generally reflected by the playing community at the time. Laughably, now that the community in the game is leaving in droves and the dev that asked me to politely be quiet got fired, and the game looks like it very well might be on the chopping block. I'd really like to believe that since my thoughts were read and responded to in writing, that if these thoughts were applied to the game in a constructive manner that the game would be doing far better now. That's a matter of objective opinion I think.

Really, if we want to be noticed, and we're asked to be part of a closed beta for our opinions it's our right and obligation to form informative and realistic responses. We also need to put our thoughts out there regularly, that's what it's all about on forums.

It's nice to be noticed, but I would rather see a posetive response in the actual game where it truly matters.

Decent read, although I can't say I beleive Blizzard does all that well with it's feedback in all three catagories relating to rogue stuns. I have yet to hear 1 person on the wrong end of a rogue stun say it's a balanced mechanic, even other rogues. Sorry if I'm going off topic here, but, really, I can't count how many times I've died from being stun locked, regardless of what class I'm playing. Eh, well I could QQ forever about the imbalances of WoW, but this isn't the place for it. Thanks for the info.

I should add my own worst experience with this sort of thing, although I suspect it only gets this bad in very very small indy companies.

When DEVs do read the forums, there is a chance - if they are basically unstable egomaniacs - of getting into a nasty cycle of insults, derision and negative feedback with players. The "it would all be fine if they just played my game the way they were supposed to" syndrome. Of course, this is because the dev is always "too close" to the issue; they will always see more of their vision for how things were supposed to be than the reality that the players see.

I agree whole-heartedly that a larger degree of transparency is needed with respect to what changes are being considered, why they are being considered and based on what "feedback" were they suggested. SOE and their NGE is a poster child for my reasoning behind those desires.

The one thing I do see happening is that gamers are voting more and more with their wallets. Sure, we laugh and joke with devs on forums, but the majority of us who pay attention to the happenings of the industry as a whole (as opposed to those who keep their heads buried in one game) see the snake oil salesman mantle being donned by most dev houses. Gamers are starting to realize their place if they haven't already:

1) share holders

2) publishers

3) game company execs

4) measurement against WoW / Blizzard

5) new customers

6) existing customers

A good read as always, but for once I don't entirely agree. I think Sanya's previous heavy involvement in community relations colors this take somewhat.

As a player who has been actively communicating concerns and opinions, through forums and in-game feedback, I've never had the impression that I wasn't being listened to. I can see the devs reply to things I've said in forum threads; I've had emailed responses; I can see that issues I've brought up or joined in with others to draw attention to have been changed in-game, or have not been changed but have been heavily discussed and reasons given for maintaining the status quo.

Moreover, having played regularly on Test servers in various games, particularly EQ2 where Test is the server where I have all my regular characters, I am used to seeing developers and QA people actually in-game, talking in the in-game chat channels, organizing testing groups, asking for feedback, initiating live discussions of current issues. And in betas, including betas for expansions for Live games, this happens tenfold.

So, I have never had the idea that developers don't listen. Indeed, it's my personal opinion that they actually listen too much and that most MMOs would be improved by designers having the courage of their convictions, following their personal vision and not playing to the crowd.

Then there's the JBoots example. Firstly, I played EQ from before the first expansion and people did not either all have JBoots, nor did they all try to get them. I certainly never bothered. The quest was too long and fiddly for my tastes and in any case i was perfectly happy with the normal run speed. I had characters both in and out of guilds and while some people certainly did obsess about getting JBoots, others couldn't care less.

I also think that its a misleading example for the lesson Sanya's trying to draw from it. Yes, it's true that those people who wanted JBoots did want increased run speed, but to conclude from that that there was inevitably a problem in the underlying game design either with generic run speed or with travel time overall is unreliable.

Firstly, JBoots were a prestige item at that stage. Having them showed that you were at a particular point of understanding and ability within the game, because as you say to obtain them took a degree of commitment and application of resources. Secondly, and more importantly, in every MMO I have ever played, a large number of people will always aspire to have the fastest movement possible. The motivation for this, I believe, comes not so much from a practical desire to get places faster as from fast movement being a marker of achievement in itself.

You could conclude from usage data that as "everyone" is doing this long, awkward quest to get an item that makes them run faster, the base run speed in your game needs to be ramped up. Or you could conclude that, by having a relatively sedate base run speed you are able to offer various desirable run speed upgrades, which will act as considerable motivators.

I prefer a system that gives you very little at default but provides you with many and various ways to improve. I'd like designers to be able to stand outside and above the players and offer us not what we want, nor what we'd like, but what we need. I'd like to have confidence that, while my opinion might be listened to and certainly would be respected, the people making the game actually know better than I do what is good for the game.

And if they don't (and I can think of some that clearly didn't) then they really shouldn't be making games in the first place.

Originally posted by TormDK

I would not ask my Community Manager to collect data from forums, I would not host official forums myself, and I would very rarely get player information from Fanfaires.

Why? Because as you mention the vocal minority will stay both vocal, and a minority. Actions speaks louder than words.

There's one very important piece of player feedback that a lot of devs don't get to see because they simply don't bother to ask for it - that's the one asking a player why he has decided to leave or didn't want to continue playing after a trial period had expired. Some have a little feedback box on the accoiunt cancellation page, a few others send an email asking for feedback, most just don't bother. Without feedback like that, they are just assuming they know why people are leaving.  I'd have thought that was very important information...

Originally posted by Bhagpuss

 I'd like designers to be able to stand outside and above the players and offer us not what we want, nor what we'd like, but what we need.

I completely agree with your entire post, Bhagpuss.  But I wanted to highlight the above sentence because I have felt this way for quite some time.

The DAoC expansion history is a perfect example of giving players what they wanted and were asking for and not what they needed.

There's one game in particular that I'm thinking of that has several forum sections: one for suggestions, some for specific classes, and one dedicated to discussion of upcoming patches. Long story short: the suggestion and class-specific forums almost NEVER draw an official dev response (in the form of an actual post, I'm sure SOME of the ideas there get added to the game, but they never acknowledge it, which is just as bad), yet even the most IQ-challenged troll usually gets a dev response within a few days. End result: the 'vocal uber-minority' get whatever nitpick they decide to vent on responded to quickly, and the ACTUAL feedback forums are viewed as the places where ideas go to die, so fewer and fewer people bother. Worst of both worlds, imho.