Sanya Weathers's MMO Underbelly: User Feedback

Good read.

I liked the story about the golf course - the field of green that inspired imagination, even if the original implementation was incidental.

I would heavily agree with your call that developers should be more open as to how players' feedback has been handled.

I don't believe I've ever had a "blue" as we call them on the Blizzard Forums, look by my threads and say anything. But oh well.

Sadly writing this won't change a thing. The people who post on these forums and think the devs should only change the game based on their particular feedback will just miss the point completly.

The truth is people need to stop thinking they are so important. Even if a game is smaller then WoW, has a hundred thousands players or so, you as an individual are not important. So when you are mad that a dev didn't take time to respond personally to your post you have no need to get upset. They might have read it, they might not have. They might have agreed they might not have. They may have datat to support it, they may have data to oppose it. But they are never required to respond to you personally.

I also don't agree that release notes need to be highlighted with what was changed due to different forms of feedback. The changes are there for a reason, if you are an intelligent enough person you can figure out the reasons. The devs don't need to hand out cookies by highlighting something and saying "See we changed this because you talked about it". That's just feeding back into the player mentality that "I as an individual are the most important person here, and the devs should listen/respond to my every word".

 Reactive communication is vital in the current MMO world. Just take a look at FunComs own AoC boards or even better at their test boards.

The developers interact frequently with the players. They talk about difficulties and why they do what they do. All stuff that appeared after AoC got its new General Manager.

The result is that yes people still complain but there is a general good mood there and the complains are often focused and productive. A completely different world from the days when funcom just stonewalled and only send out sugared rose colored press statements from time to time that had no relation to the game experience the players had.

Sometimes it takes a change of leadership to improve things. I remember in DaoC where there were key elements of the game that people complained about forever and the players never ever got any decent reply (you could almost feel Sanya's frustration in her words on the Herald and boards back then). Then Mythic began to develop WaR and a lot of the big shots got moved to that project leaving a new dev team on DaoC and within 3 months you saw issue after issue the people had complained about for years be solved one by one and even today the game is still being played by dedicated fans.

In relation to Sanya's talk about the golf course and the lack of a similar area in the new enviroment, I remember suggesting a big neutral island without keeps in the middle of the ocean as a playground for that kind of play that I did on the IGN boards and 6 months later there were one . . . . . which then weren't  _that_ cool teaching me that idea's you think of yourself could easily not be that good and that is why you need many inputs and opinions on game elements before you implement them. :)

Player ideas are good but they need to get processed.

Also keep in mind that sending your feedback to Customer Support will most likely get you a "Thanks for your feedback!" standard response while your suggestions will end up in the CRM's digital bin as a single person's opinion will most probably be regarded as rather unimportant.

You will be better off with becoming part of a focus group (if such thing is in place), or with providing feedback on the game forums where other players and maybe community representatives may discuss it.

Developers definitely should at least get sharp-eared if there's "overwhelming  player feedback" on a certain topic, but then properly investigate (in-game data, in-game experience) before taking any decision that the silent majority might probably dislike.

A dev team with passion for their own game and a clear focus on what'll be good for it and what not, plus player feedback taken from the meta-game (e.g. forums, official or not) and from in-game (e.g. from focus groups, or from playing your own game) can be very successful in doing what's good for the game as a whole and thus for the majority of its players.

I instantly think Guild Wars. The ArenaNet guys have been doing a great job for that matter.
 

Just don't send your feedback to CS...

A good read Dana.

If I was running a MMOG I would be focusing on usage data.

I would not ask my Community Manager to collect data from forums, I would not host official forums myself, and I would very rarely get player information from Fanfaires.

Why? Because as you mention the vocal minority will stay both vocal, and a minority. Actions speaks louder than words.

I would not have an issue with mentioning the statistics in the change logs though, so that people could sit and be vocal on their guilds forums reguarly. But as long as the gameplay was decent and content would chug along nicely the minority likely wouldn't be going anywhere anyhow.

I would like to see feedback from the gamemakers. I would also like to see slight changes over the swing for the fences type fix.

I do post occassionally but never that often on a game I play. I enjoy playing then my complaints will be small. Most will only be about one power or ability that might not be working. I don't worry about being the best Class/sub-Class/damage dealer/tank/healer or anything like that. I want a fairly even playing field but don't expect perfect balance. I see expansions/updates that add new abilties and/or powers to classes and the balancing includes going from start to finish on the fix to a class. I will never understand that and never respect that. Doesn't mean that it is wrong but I sometimes think that needed perfection by the activists to be #1 is too much.

Great read!

Good ole Emain Macha

Good article.

I especially like the point about looking at all relevant data: not just what the folks are saying on the forums, but also what the "numbers" data is saying.  Many times, players whinge about this or that...even as they take full advantage of whatever it is they're upset about this week has to offer.

The fact is that, in my opinion and experience, players ask for things without considering what it is that they're asking for, and then push the issue on a game's forums until they get what they asked for...only to complain that what they got wasn't what they wanted.  It seems to me that, in many cases, some players (the "vocal minority") come across as wanting to be the only people who have a say in what happens to/with a game, without considering what the "silent majority" would like, or even what is humanly possible given the way a game is coded.  I don't expect too many folks will stop whinging after reading this, but at least Ms. Weathers has done a good job of putting some information out there for folks to consider.

After SWG, I'm very iffy about focus groups.

Using fanboys only or using people who barely knew what a PC was to get the feedback they wanted.

Instead of getting good feedback, they only got the feedback they wanted, the feedback that agreed always to their actions.

I dun trust focus groups any more, because of the ability to pick and choose only participants that fit the devs scheme.

Tho I have to admit, I was invited to participate in the focus group that OK'ed the NGE. But to participate, I would have had to pay dearly to get there, since SOE who claimed in their invitation "All expenses paid", suddenly realised that I was european...

But my invite came after I had been a good lil fanboi for a while on their forums... Well, that and I was part in a guild that did get dev attention and was used as a posterboy(guild) in SOE fan faires and gatherings.

Loved the read, but as an old DAoC player I knew exactly what you were talking about with the "golf course"...and you are sooooo far off base as to what "went wrong" with the replacement that I cringe to even read it....but that's another thread and one I would love to discuss over several pitchers of beer sometime.

A fair read, I don't know how dev's feel about the subject.

I generally believe that many developers do read forums regularly. I don't think that anyone is too small to respond to though. As far as a real response, well that is up to the person/s forming the response.

I know from experience about putting my personal thoughts out on a closed beta forum, and getting a personal response from dev's. Neither of them being necessarily good, even if my original intent was to throw up a red flag and try and reign some people in on what was reality. I wasn't shouted down by any other players, but I was told to be quiet by the dev in a personal message. It was odd to me that  I must have struck a nerve, cause there was no public response just a person message.

In the end I believe my views were generally reflected by the playing community at the time. Laughably, now that the community in the game is leaving in droves and the dev that asked me to politely be quiet got fired, and the game looks like it very well might be on the chopping block. I'd really like to believe that since my thoughts were read and responded to in writing, that if these thoughts were applied to the game in a constructive manner that the game would be doing far better now. That's a matter of objective opinion I think.

Really, if we want to be noticed, and we're asked to be part of a closed beta for our opinions it's our right and obligation to form informative and realistic responses. We also need to put our thoughts out there regularly, that's what it's all about on forums.

It's nice to be noticed, but I would rather see a posetive response in the actual game where it truly matters.

Decent read, although I can't say I beleive Blizzard does all that well with it's feedback in all three catagories relating to rogue stuns. I have yet to hear 1 person on the wrong end of a rogue stun say it's a balanced mechanic, even other rogues. Sorry if I'm going off topic here, but, really, I can't count how many times I've died from being stun locked, regardless of what class I'm playing. Eh, well I could QQ forever about the imbalances of WoW, but this isn't the place for it. Thanks for the info.

I should add my own worst experience with this sort of thing, although I suspect it only gets this bad in very very small indy companies.

When DEVs do read the forums, there is a chance - if they are basically unstable egomaniacs - of getting into a nasty cycle of insults, derision and negative feedback with players. The "it would all be fine if they just played my game the way they were supposed to" syndrome. Of course, this is because the dev is always "too close" to the issue; they will always see more of their vision for how things were supposed to be than the reality that the players see.

I agree whole-heartedly that a larger degree of transparency is needed with respect to what changes are being considered, why they are being considered and based on what "feedback" were they suggested. SOE and their NGE is a poster child for my reasoning behind those desires.

The one thing I do see happening is that gamers are voting more and more with their wallets. Sure, we laugh and joke with devs on forums, but the majority of us who pay attention to the happenings of the industry as a whole (as opposed to those who keep their heads buried in one game) see the snake oil salesman mantle being donned by most dev houses. Gamers are starting to realize their place if they haven't already:

1) share holders

2) publishers

3) game company execs

4) measurement against WoW / Blizzard

5) new customers

6) existing customers

A good read as always, but for once I don't entirely agree. I think Sanya's previous heavy involvement in community relations colors this take somewhat.

As a player who has been actively communicating concerns and opinions, through forums and in-game feedback, I've never had the impression that I wasn't being listened to. I can see the devs reply to things I've said in forum threads; I've had emailed responses; I can see that issues I've brought up or joined in with others to draw attention to have been changed in-game, or have not been changed but have been heavily discussed and reasons given for maintaining the status quo.

Moreover, having played regularly on Test servers in various games, particularly EQ2 where Test is the server where I have all my regular characters, I am used to seeing developers and QA people actually in-game, talking in the in-game chat channels, organizing testing groups, asking for feedback, initiating live discussions of current issues. And in betas, including betas for expansions for Live games, this happens tenfold.

So, I have never had the idea that developers don't listen. Indeed, it's my personal opinion that they actually listen too much and that most MMOs would be improved by designers having the courage of their convictions, following their personal vision and not playing to the crowd.

Then there's the JBoots example. Firstly, I played EQ from before the first expansion and people did not either all have JBoots, nor did they all try to get them. I certainly never bothered. The quest was too long and fiddly for my tastes and in any case i was perfectly happy with the normal run speed. I had characters both in and out of guilds and while some people certainly did obsess about getting JBoots, others couldn't care less.

I also think that its a misleading example for the lesson Sanya's trying to draw from it. Yes, it's true that those people who wanted JBoots did want increased run speed, but to conclude from that that there was inevitably a problem in the underlying game design either with generic run speed or with travel time overall is unreliable.

Firstly, JBoots were a prestige item at that stage. Having them showed that you were at a particular point of understanding and ability within the game, because as you say to obtain them took a degree of commitment and application of resources. Secondly, and more importantly, in every MMO I have ever played, a large number of people will always aspire to have the fastest movement possible. The motivation for this, I believe, comes not so much from a practical desire to get places faster as from fast movement being a marker of achievement in itself.

You could conclude from usage data that as "everyone" is doing this long, awkward quest to get an item that makes them run faster, the base run speed in your game needs to be ramped up. Or you could conclude that, by having a relatively sedate base run speed you are able to offer various desirable run speed upgrades, which will act as considerable motivators.

I prefer a system that gives you very little at default but provides you with many and various ways to improve. I'd like designers to be able to stand outside and above the players and offer us not what we want, nor what we'd like, but what we need. I'd like to have confidence that, while my opinion might be listened to and certainly would be respected, the people making the game actually know better than I do what is good for the game.

And if they don't (and I can think of some that clearly didn't) then they really shouldn't be making games in the first place.

Originally posted by TormDK

I would not ask my Community Manager to collect data from forums, I would not host official forums myself, and I would very rarely get player information from Fanfaires.

Why? Because as you mention the vocal minority will stay both vocal, and a minority. Actions speaks louder than words.

There's one very important piece of player feedback that a lot of devs don't get to see because they simply don't bother to ask for it - that's the one asking a player why he has decided to leave or didn't want to continue playing after a trial period had expired. Some have a little feedback box on the accoiunt cancellation page, a few others send an email asking for feedback, most just don't bother. Without feedback like that, they are just assuming they know why people are leaving.  I'd have thought that was very important information...

Originally posted by Bhagpuss

 I'd like designers to be able to stand outside and above the players and offer us not what we want, nor what we'd like, but what we need.

I completely agree with your entire post, Bhagpuss.  But I wanted to highlight the above sentence because I have felt this way for quite some time.

The DAoC expansion history is a perfect example of giving players what they wanted and were asking for and not what they needed.

There's one game in particular that I'm thinking of that has several forum sections: one for suggestions, some for specific classes, and one dedicated to discussion of upcoming patches. Long story short: the suggestion and class-specific forums almost NEVER draw an official dev response (in the form of an actual post, I'm sure SOME of the ideas there get added to the game, but they never acknowledge it, which is just as bad), yet even the most IQ-challenged troll usually gets a dev response within a few days. End result: the 'vocal uber-minority' get whatever nitpick they decide to vent on responded to quickly, and the ACTUAL feedback forums are viewed as the places where ideas go to die, so fewer and fewer people bother. Worst of both worlds, imho.

Great read! Keep up the good work.

Good read.

By the way, just wanted to show what could be the first in-game tool for public opinion - in Aion, at the current NA/EU Beta, you actually receive a questionnaire in-game at level 21, with a few questions like Rate Aion's combat/crafting/etc, What you look forward at release, What you think of mechanic XYZ, etc. There's some potential in there for some "The Developer Cares" showing by actually making these questions in-game.

Maybe even makes their job easier, because along with the automatic data collection they also get the user opinion feedback objectively through in-game tools instead of forums.

Originally posted by EricDanie

Good read.

By the way, just wanted to show what could be the first in-game tool for public opinion - in Aion, at the current NA/EU Beta, you actually receive a questionnaire in-game at level 21, with a few questions like Rate Aion's combat/crafting/etc, What you look forward at release, What you think of mechanic XYZ, etc. There's some potential in there for some "The Developer Cares" showing by actually making these questions in-game.

Maybe even makes their job easier, because along with the automatic data collection they also get the user opinion feedback objectively through in-game tools instead of forums.

They did this in the WAR beta. The questionaire part, that is.

And this article explains why I think Turbine (D&D Online section at least) is the best MMO developer.

Contrary to what nay-sayers try to troll out, they do every part of that. And do pretty well. It is seen in beta now to the unmatched extent.

quality  vs reality problem

John, IQ 100

Tomas, IQ 200

Marcus , IQ 130 

John likes TV,  Tomas like TV, Marcus hate TV

John likes porn and pop culture.   Tomas like porn, pop culture and science. Marcus hate porn, and like pop culture.

You want to make John, Tomas and Marcus happy.

So you obsviusly create a POP culture TV program.  Because thats what the date shows. And you are right. 

But you are not delivering Porn, or giving something for the people that hate TV. 

You have killed excellence, embracing mediocrity. 

Thats why almost *all* TV is trash.  Because is the minicom common denominator where the group is big. The bigger the group, the crappier and trashier the mcd. 

 In my opinion these game developers listen too much to their gaming audience. Many great games designed by people with great vision have went to trash by listening to what little Johnny thinks a game should be. They are the ones that make it their career to make a great game, I dont' care to play a game little Johnny decides what is fun.

The developers need to stick to their game visions and of course get feedback whether or not people like it. What is bad is when game developers have a game many people like but change the winning formula because of people crying on the forum. In most cases the ones on the forum do not care about whether or not the game is fun or captive, they are crying because of wanting something somone else has without playing to get it. 

 I love reading these articles. Getting a glimpse into the industry and how it works is really cool. I think it's very eye-opening for many others as well who think they know how the MMO world turns, despite never going further than an official game forum.

Feedback via text, forum, webform is a WASTE OF TIME.

When it comes to having a player get their point across the old saying holds true.

Money TALKS! Bull$#!t WALKS!

Don't like an MMO game? Then don't pay the $15 per month to play it. Sooner or later, the development team will get the hint, when their bean counters are filing for chapter 11.

Originally posted by NameWasTaken

Feedback via text, forum, webform is a WASTE OF TIME.

When it comes to having a player get their point across the old saying holds true.

Money TALKS! Bull$#!t WALKS!

Don't like an MMO game? Then don't pay the $15 per month to play it. Sooner or later, the development team will get the hint, when their bean counters are filing for chapter 11.

Ok...but if you don't tell them why you don't like it how are they suppose to ever figure out what you do like and make a game with those aspects...

Ok, being a Sociologist by trade and a gamer by hobby, I felt inclined to reply to this article in some detail. I have worked quite some time on feedback control studies, and IMVPO there are a few misconceptions here.

Of course, the most simple player misconceptions is "I said I am not happy with feature X, it did not change therefore Game Companies never listen". It is the very human feature of a) concluding from personal experiences to matters per se and b) generalizing singular events to general rules (Company X did not listen = all game companies never listen). Quite a trivial observation, some may say, but given forums like this are usually full of these basic issues, I think it is valid to point them out at the beginning. In MMos it comes in simple forms. "I like hardcore therefore hardcore is the future guarantee for success."

The problem is, companies get a lot of feedback based in that sort of thinking, and truth be told, NONE of us is entirely free of this thinking. It is practically impossible to avoid this altogether. Our concepts are often based on the expectation what others think. If you look at any game forum, you will find a most problematic issue with feedback: People tend to "follow the general stream". Some people are just unsure of their judgements in the inside, so they read some postings and reviews and tend to get influenced into that way. Thats no matter of being weak willed; it happens to me frequently, and I count myself as not easily persuaded. On critical topics, there is always a tendency that some "ruling opinion" is dominating forums, and people with differenciated opinions shy away from posting and further taint the feedback.

Now sure, you can gather data from mails or surveys, but I must admit in the many betas I have participated I have seen that VERY rarely. Even then, the opinion about stuff is formed a lot in your private friendship circles and forums. The greatest problem with how people like something in Social Science is always, that asking the target audience is a tricky and often misleading thing. For instance, if you want to find out about the situation of the working class in an under-developed country, just going and asking them is not working. Many people who never thought about things in detail, may not have the words for something, and then tend to say "all is ok." When you make surveys with people on any social topic, say "how happy are you with you life in general", the majority will then answer they feel good about it. When you then ask about every detailled aspect, politics, environment, money asf, and ask the same question AFTERWARDS, the result will usually be much more negative, because by then they had time to think about all detailled stuff. It is just one example how difficult "just asking gamers" is. For instance, if you ask people after they had some bad day in the game because a quest was bugged or they had a quarrel with other gamers, they will answer polls much more critical, as if they just had a blast of a day in the game.

This also counts for usage data. Sure, you CAN watch the gamers doing stuff. But it is quite difficult to judge from that how much players really like it. You can watch people do the quests or hunt at some place, but how can you know another way would not be more welcome? Many gamers will do something they actually do not like, because the reward is so good. They will for instance repeat raids over and over, because they are greedy for some rare armor, but that says nada about whether or not they actually like the act of raiding itself. Same goes to soloing. You may discover more and more people solo. But is that because they enjoy soloing more, or is it that grouping just isn't organized well enough? Or maybe the community in the game is just not there. I recall games like SWG had a strong community because of many of the social ingame features, which resulted in people much more willing to group (in the old days). On the other hand, when you play Kunark in EQ2 level 70-80, you will find 90% soloing. Not because they like to solo, but because the system doesn't make grouping profitable. Now just watching those habits can easily lead to the wrong conclusions that (a) everything is fine - people do what expected and (b) people love to solo, lets add more of it. Both are conclusions from observing player habits, and still they are wrong.

The entire question of the "silent majortiy" isn't really to avoid. It would take tremendous work to get their feedback, and there comes another issue. Programmers and designers are "queens". I worked with programmers, even in something as mundane as programming cash desk computers, and even there every programmer was a bit of a Diva in "his" ideas how to implement stuff. I am just quite certain that a substantial amount of decisions is never placed to serious gamer feedback. Game companies may take small trivialities serious. But most programmers and designers are just way too certain that THEIR idea is right and will entertain the people best, that there just is too little will to ask people in the big themes. Take SWTOR's story. Or Warhammers two factions. There are very basic non negotiables, and these are just extreme design decisions. I have seen changes based on feedback come only very slow, if at all. Sure, in small terms, that may be the case. But in every major design questions, devs (in the broadest sense) are Divas. They are very sure what they design is right. They are sort of resistent against council by their very nature. They are experts, the usually love their job and regard their games like a parent their kids. The relationship of creator to product is very high in the entire entertainment industry, and gaming is no exception. You don't go to Spielberg and expect him to make a survey on how to make his films. And most game devs, programmers and designers feel the same way. The DIFFERENCE between film and game is, film had highly developed, changed and diversified in the last 30 years, while games more or less still function in the same way as 25 years ago. Games are a very conservative thing, when it comes to "how to make".

The game industy is IMPOV very conservative and narrow, usually sticking to the known and proven. Just look at how stereotypical, predictable and copy cat-like almost every single character or story is. Mostly it is, know one, know them all. Another point is, game developers are usually bad artists. Sorry, but that is for me at least in Western Companies, a fact. Most of them are programmers and have little to no artistic feeling, and I always have this urge to send them all to the TV show "queer eye for the straight man" or what the name is. Or just hire more women. (Sorry if that sounds prejudiced.) How much you can do with a game company with GOOD artistic feeling, is proven by the world design of LOTRO, which is just superior. Whoever designed that visual world was really good, whereas we have a world design like Vanguard or WAR which is just sterile and dead, which has nothing to do with Polygons or that WAR is based on an IP. LOTRO is too. Or look at races, haircuts, robes, armor types, faces, whatever. So many games just have a very bad sense of art, and only few games really are visually artistic, like GTA IV, to name something entirely different. In such terms, feedback is void, because you can say that an hundred times. Preaching to game developers about artistic matters is mostly in vain, simply because they either HAVE a sense for it, then you don't need to mention, or they DON't have it (mostly so), and then they will not understand you. The Vanguard devs never understood that despite they polygon details Telon is just a cold and sterile world in most places. (Just see the changes of Khal, which SOE made. THEY knew how to make something visually good, if you compare Khal before and after.)

You can apply this example on many parts of a game. Feedback does not result in change mostly, because of these mechanics. The companies will usually think, they know better and give only the smaller things to player feedback. Now if that is wise or not is another topic. WOW surely seems proof that you can basically ignore what the gamers say, and still be a success in money. And thats the crux. A game company today is mostly a company to make money, just as selling a laundry machine or a sort of youghurt would be. But for the GAMERS, the customers, a game still by and large is something very personal. It is "ours", our hobby and something with which gamers identify a lot. That is what differs games from yougurts and laundry machines. But today we have large companies with stock holders and bean counters who rather want to sell Sims 645 and Soccer Manager 2053 to make big cash. They care less and less today for those who feel gaming as a part of their life, and aim more for people who buy their games, play it a week or two and then toss it away. For extreme casual gamers, to say it blunt. WOW is proof for that. The most time consuming parts - crums for the hardcore - like raiding and daily quests and faction grind, are the most mindless and repetitive parts. Blizzard does not invest into such gamer dreams like story, new haircuts or really new meshes, when 10 of 11 million players are satisfied with less. That McDonalds is succesful, doesn't say anthing that McDonalds food is good.

Yes, transparency is lacking. Badly lacking. So often we see almost zero feedback from the companies, or we get some PR blah blah, which essentially says nothing. What game companies lost today is, that games are not like any other product. Much more than a film or a book, a game is something gamers feel as "their own". And that isnt entirely unfounded. You can passively consumate a film or a book, but a game needs work, effort, and espcially a MMO is a huge investment of time into a character. So there IS some moral ground to say, we the gamers should have some say in where the train is headed. The time and work people invest into their chars is often tremendous, so its not like a movie which you watch in the cinema in a 2 hours session and be done with it.

Finally, let me say this. As in TV and once in film, we have game companies who feel that in order to reach masses they have to make cheap, easy stuff. Sims & WOW. I just deny this is true. You can see the same, wrong philosophy in movies. Some movie makers like George Lucas have always argued, a popular movie can only be 90 min. Until Lord of the Rings proved the opposite. They have argued complex, strenous stories will not make popular movies. Such things are cliches. You do not HAVE to make McDonalds entertainment to reach the masses, you CAN attract many people even with elaborate and demanding and innovative stuff. But my guess is the bean counter & stockholder structure in game companies will usually quench all reasonable use of feedback. Insofar we will see the same standard game models recycled over and over for likely a very long time.

^ Do you really think in most cases they have much say in how things get implemented? Speaking as a lowly programmer myself, virtually all of the time your manager tells you what to implement, and he in turn gets his marching orders from management or the bigwigs at PR or whoever. "Doing things your way" is a programmer's pipe dream, unless you're talking about one-man shows or garage based companies.

Point is, your perception of the "diva" programmer who insists on doing things "his" way is largely incorrect. Decisions are made by higher ups 100% of the time, unless you're talking about a tiny company of 3-4 people where the programmer is also a partner or something like that. Most programmers are simply cogs in the machine, rats on the treadmill. Sure, occasionally you hear from big name studio people in the press like Carmack from id, but the vast majority of us are just peons.

Customer feedback gets filtered through like 10 layers of management crap before actually reaching the programmers, and by that time it's already in the form of "do this, then do that". That's the reality of it, unless you're talking about smaller no-name studios... but those aren't the guys making MMOs now, are they?

Originally posted by Elikal

Finally, let me say this. As in TV and once in film, we have game companies who feel that in order to reach masses they have to make cheap, easy stuff. Sims & WOW.

I agree with a lot of what you've said, but right here you've gone off the rails. WoW and the Sims are only cheap and easy in retrospect. WoW was the most expensive MMO made for a good few years there and I refuse to be believe a title as complex yet streamlined as the Sims was 'cheap' to develop.

Even their clones have been expensive (not that anyone has really truly tried to clone the Sims in as much detail). But the irony is that even as new MMOs have come out and attempted to do something different - WAR started out very differently to where it ended up - they get shouted down by their players that they don't want the new thing, they want the old thing just better. So player feedback ends up driving things to the middle and away from innovation as surely and publisher who wants a guaranteed hit.

On Sanya's article, I just have to comment on game companies using surveys and focus groups. I've never been part of a game company focus group, but I've seen their surveys and they are awful. Poorly constructed, unbalanced scales (the irony of the graphic for this column using a 3-positive, 1-negative scale didn't escape me), often slated towards certain topics over others (e.g. PvP might get more of a focus from an exit survey, while someone might be exiting for completely PvE reasons) and so on.

As for focus groups, if they really are used as you detail, I can't see them giving back much quality information as well. There are much better research techniques available, but I'm sure it is easy to convince devs that sticking 20 people in a computer lab, letting them play for an hour then getting them to fill out a questionnaire while others watch will somehow give them solid player-based feedback to work from.

They have the two best resources for player feedback at their finger tips ( forums / game itself ) and they don't know how to use them.  I guess its probably more of the " burnout " factor, since they probably just spent years on the project, and don't really want see it on their desktop anymore.

Another great article, and Sanya is now clearly my favorite writer here on MMORPG.com!

I worked for a director a few decades ago, who held to the motto "always leave them wanting more".  Simply said, if you completely pay off a users expectations, there is no reason to come back for the next event (film in his case).  I see this as applying quite a bit to MMORPG design.  To use a metaphor, "a truck going up a hill is supposed to make some noise", meaning that any game that defines itself as being 'challenging' is going to give the people on the low end of the capabilities crowd enough struggle that they will whine and complain.  As such, I take "feedback" with a grain of salt.  It's a matter of consider the source.

In my beta (actually in pre-beta right now) I've got players who whine about [insert item here], while most of them have no real issue with that same item.  Conclusion:  it works.

On the other hand, when almost everyone from a hand picked focus group (with little to no communication among themselves) comes to me and says ... "Ken, this [blah] isn't going to work", then I have a real problem and its time to fix things.

Overall, its quite a balancing act.

Ken

Interesting read on all the complaining we do. :)

I think linking patches in vague terms to player suggestions could work, in fact I have seen this happen a couple of times. If I was a community manager I would stay in touch with the best of the beta players, the ones who actually put in problem reports, they would be a goldmine of information.

here's what I think as the best way to do it : ingame surveys, much like what Aion just did. first, you're limiting the feedback to just the people interested in the game, not the random 13 years olds and all other kinds of forum trolls, basicly people who can only bash the game no matter what. but the important thing is - you HAVE to listen to that feedback. just an example - for several months now the WAR community (whats left of it actually) is screaming how AOE should be changed, while the producers letters are clearly showing Mythic doesnt give a **** about the feedback and decided to go a completely different way (i.e. reducing range instead of reworking the skills)

and yeah, forums (especially fansite ones) are NOT the right way to do it, too many people running those sites are simply moderating EVERY negative feedback and are deluded that they are helping the game by doing so. typical example is WHA, even worse example is aionsource, which founder got replaced by ncsoft demand. too bad people cannot understand that simply not talking about a issue would not make it disappear.

If you had spent over a year making a game, you would know if it was good or bad, wouldn't you? If you actually played it?

I've always wondered about this.

Originally posted by badgerer

If you had spent over a year making a game, you would know if it was good or bad, wouldn't you? If you actually played it?

A game as a whole is hard to judge - and by the time it's together it's too late anyway.  But when it comes to individual features, this is the part that boggles me the most.

Even though there are many parts of WoW I'm not particularly interested in, I can always see how SOMEONE would like pretty much every feature in it.  WIth many other games, I look at some aspect of the game (or sometimes, more than a few aspects) and I have no idea why ANYONE would EVER want that.  And I don't just mean that the implementation just didn't work well, or that it's buggy -- I mean I literally have no idea what the Hell they were thinking when they first decided to do it this way.

I always get this picture of a coke-head at a board meeting rapid-firing a half-baked pitch about how he wants the game to have thisandthisandthisandthis...  (Either he's some rich guy's son who got WAY too much money to play with, or he's some crazy genius who once had a game idea so far outside-the-box that it caught the gaming public's interest and sold like hotcakes -- 15 years ago.)

Then the people he hired to do his thinking for him do their best to turn that mess into design requirements and hand them down to the poor programmers. 

I think that's where the real disconnect is.  Nobody wants to tell the guy who's paying their salary:  "Hey dude, we can make this work, but it just isn't any fun."  That's where the feedback loop is missing.

The inability to listen to user feedback has doomed many games.  Let's use DAoC as an example.  It was a pretty good game until the Atlantis expansion.  Had a growing playerbase and it was fun.  It also had a plethora of class for the alt-aholics. 

Unfortunately Atlantis introduced overwhelming powers for the dedicated player, which is a major no no when it comes to pvp games.  The forums were rampant with complaints.  Changes forthcoming were minor.  If you did not subject yourself to the huge grind, you were superfulous in pvp.  The playerbase voted with their feet and left the game in droves.   DAoC never recovered.

SWG is another example.  Don't have to go into that one.

You had better know where your bread and butter lies when it comes to introducing new things.  Too drastic a change will start an exodus that will be hard to stop.  You had better have your ear to the ground and be prepared to scale things back when the feedback gets loud.

As to the comment about programmers, their ability to freelance is very limited.  You deviate from the design and most often will be required to redo your code.   No programmers freelancing has little to do with bad design decisions that management refuses to retract.  It is the designers

Originally posted by Trucidation

^ Do you really think in most cases they have much say in how things get implemented? Speaking as a lowly programmer myself, virtually all of the time your manager tells you what to implement, and he in turn gets his marching orders from management or the bigwigs at PR or whoever. "Doing things your way" is a programmer's pipe dream, unless you're talking about one-man shows or garage based companies.

Point is, your perception of the "diva" programmer who insists on doing things "his" way is largely incorrect. Decisions are made by higher ups 100% of the time, unless you're talking about a tiny company of 3-4 people where the programmer is also a partner or something like that. Most programmers are simply cogs in the machine, rats on the treadmill. Sure, occasionally you hear from big name studio people in the press like Carmack from id, but the vast majority of us are just peons.

Customer feedback gets filtered through like 10 layers of management crap before actually reaching the programmers, and by that time it's already in the form of "do this, then do that". That's the reality of it, unless you're talking about smaller no-name studios... but those aren't the guys making MMOs now, are they?

I underlined what makes great games. I know the guy above said that some programs are like divas but I disagree. I think you guys make the most fun games out there when you are working with your ideas.

---

Originally posted by qombi

---

Originally posted by Trucidation

^... Most programmers are simply cogs in the machine, rats on the treadmill. Sure, occasionally you hear from big name studio people in the press like Carmack from id, but the vast majority of us are just peons.
Customer feedback gets filtered through like 10 layers of management crap before actually reaching the programmers, and by that time it's already in the form of "do this, then do that". That's the reality of it, unless you're talking about smaller no-name studios... but those aren't the guys making MMOs now, are they?

---

 
I underlined what makes great games. I know the guy above said that some programs are like divas but I disagree. I think you guys make the most fun games out there when you are working with your ideas.

---

Programming is like paiting a mural. A genius can get a bunch of normal painters, and make then draw faces, and horses, explosions... following his orders.

If these normal painters are not genius, wen you zoom-in on the details, you will see mediocrity. But If the genius is really a genius, wen you zoom-out you see a great mural.

Wen you work with a guy that is not a genius, and the painters are not a genius... you get something that is boring on all levels.

Wen you work with genius programmers, and a director that is not a genius, you can see genius details, but the whole thing suck.

So.. good games need a good director. The better games need the better programmers.

And the big the team, the smaller the contribution of the individual programmer. Trucidation is telling us than in a big project the contribution of the programmer is soo small, the oportunity to make a difference is small or not existing...

..on the other part, PR is telling us that the fire implementation on Farcry2 was the work of a small dude. True or not... we love and remenber games often by tiny details. A game can be good because a redeming feature. If a programmers is bad, maybe that cog will not work, and a resulting, a big cog will not work and a feature will not work.

Programmaers have already demoed that a programmer can drive the art of videogamme developping, now is time for other people (designers, writters,..) to take the role, and show what other type of people can do.

Methinks the "reign" of programmers as been glorious, with all these early 80's crazy games. No-Programmers people have still to beat that "score".

Originally posted by Dana

Ah, zones. Long ago in a virtual world, user initiated feedback told the developers of a particular game that they hated, HATED Zone X. They constantly mocked this zone – “the golf course.” It was, admittedly, not the finest zone in the game. It was large and green without a lot of features. It has been hastily assembled at the last second. The new zone was pretty neat on paper, lots of obstacles and control elements, requiring tons of strategy.

And yet removing the golf course was, shall we say, not popular. What? Whyever so? The usage data on the old zone showed that despite the mockery, the much maligned zone was where everyone went to play if they had nothing else to do. It was the default, where a group could always be found, and you didn’t need a minimum number of people to do anything, and you could always find someone to kill.
 

 In business, as in pretty much every aspect of life, something called the "80/20" rule applies: eighty percent of your expense comes from 20% of your customers. The trick to managing a business is identifying and then focusing on the part where you get the most bang for your buck.

I've seen a lot of very energetic and well-meaning owners go out of business because they gave in to their own egos, and focused on making their already-pleased customers even more slavishly devoted.  The problem is that fanatic customers always give the worst possible advice. First of all, no matter how much they whine; they are not going to leave.  They love your product a lot more than it costs them.  What you need to do is figure out new ways to suck more money out of them -- and that is the LAST thing they'll ever suggest to you.

In order to grow your business, the people you need to listen to are your marginal customers.  The quiet souls who (God bless them) sit around and pay their fees on time every month with no drama.  But a lot of those players have already left the game, and (for all you know) a bunch of other ones may be sitting on the fence right now.  The problem is that listening to them is hard to do.  You have to be motivated, take a proactive and disciplined approach to get this data. 

I suspect that in MMORPG's, for every ADD kid who hates wasting a whole week blowing past those first 50 "transition" levels to get to the "uuber-leet endgame"; and for every compulsive-psychotic who wants an even more fiendishly convoluted raid encounter that will require a second pair of adult diapers to complete in one sitting; there are a bunch of other players who really just want a giant sandbox to quietly hunt and fight in...  People who won't say a word if the class they're playing (let alone another class) is "unbalanced" - because they couldn't possibly care less.  People who don't scream like stuck pigs about every little change because they LIKE new aspects to explore.  People who will deal with a dungeon that doesn't perfectly fit their needs - because they've been doing their best to try and squeeze some enjoyment out of areas specifically designed to exclude them. 

I say I "suspect" all that because there's really no way to know for sure.  Those folks aren't the type who are likely to jump up and scream their heads off -- they'll just kind of quietly leave once they've finally gotten sick of infinitely re-playing the tiny fraction of the game's content they can reach. 

But when I see that a game designer ACCIDENTALLY made a zone these players loved more than any other, it sort of makes me wonder what would happen if somebody actually went out and TRIED to give these people the kind of experience they're looking for.

Interesting article.

Sanya regarding player feedback do you think adding polls at the game login screen would work?

This way devs would have not only the feedback from the forum users but also from all the others that never go on the forums. 

Originally posted by Deewe

Interesting article.

 

Sanya regarding player feedback do you think adding polls at the game login screen would work?

 

This way devs would have not only the feedback from the forum users but also from all the others that never go on the forums. 

Great suggestion!  We do something very much like this with one of our business application programs at work; and its been very successful.

This was a very good article but it told me a lot about why devs are sometimes clueless with regards to what the majority of the loyal player base of any game is thinking. There is one thing that you need to understand about all of these forms of feedback mentioned in the article. A lot of the people who regularly give feedback though these methods lie though their teeth about content that is not popular with them. Why? because they don't want you to listen to those who don't agree with them. This is true with rule sets, content, and probably most of all in existing games class balance.

I've been playing MMORPG from early 02 with pre Luclin EQ, beta tested a huge list of games [still do] spent years in WoW and EQ2 going back and forth between them. I'm prolly an odd person but I talk to people in game about what they like. I don't work for anyone, I just do it to figure out if my thinking about some of the game content is as way off in left field as forum behavior would seem to indicate. Turns out most of the time it isn't and the forum intell is often skewed way off from what a lot of actual players feel. I think what set me off on this kind of investigating was what I call the great horse nerf of early EQ2. There was a big stink with regards to the cheap crusader horse that had a moderate run speed [less than SoW]. I recall one individual who was very vicious about it and she insisted that ALL the paladins in her guild wanted it nerfed because it would prevent newbies from rolling paladins, then had her friends [and maybe other accounts] back her up, and even reposted the same demand every time the thread died down. She also resorted to threats and a lot of profanity when anyone had another view. Yes it sounds stupid now but she was fairly successful in her efforts. In the end the run speed of the noble steed got nerfed so bad you could get run down and killed while on it but the dev team [I suspect in resentment of the issue] also nerfed every other horse in the game as well. I didnt really have a position on the nerf but it was rather interesting to watch. Beta test forums can sometimes be even worse as some of the people there never pay to play but tend to just camp beta test and play free games. Forums get very political , yes they do.

At any rate that was what set me off wondering and I have been asking in game questions of people and making comparisons ever since. People are sometimes not honest on forums and in study groups. They defend the status quo or what the mentality of their group is. This is how words like "care bear", or "the grind" came about. Its just like real life politics , you take what your trying to undermine and then assign it a name you can call it then use it in negative ways. Never mind that thousands of people logged in daily and dutifully paid monthly fees to participate in "the grind" when in the early days of EQ the company didn't even expect them to. At some point end gamers made their way in later down the line and began to pay others to do "the grind" for them so they could just skip to the end and then of course set about establishing that nobody else wanted it ether. One of the trade marks of this particular type is that they like to speak as though everyone else had the same views as they do.
The main point is when she said silent majority, she hit the nail right on the head. Personally I think the only way to really get a chance at regular player opinion is to have polls in the game it self. Honestly most people I talk to in games don't even read the forums and are often shocked if I ask them about a discussion that is leaning in one direction or another. The great search for player opinion. This could be an epic quest.

I agree with that the official personnel should pay more attention to what the players have said on their official forum, after all that’s the most important source on the basis of which the developers could improve their production. But I do feel that the game developers are also not such cool blooded animals who never care what the players have said. In fact, I did have this kind of experience: when the developers really followed the players suggestion and launch a new version of game, players will very likely find more Bugs in that game. The conclusion is improving games needs time and it takes risk to produce more Bugs and more players complaint. Besides, game developing costs Capital. Take Funcom’s “the Age of Conan” for example, Funcom spends a great deal of money to produce high quality game, but it turns out that it will eat our hard drive severely and people complaint about it; Besides, Funcom also flaunt “the real combat system” as this game’s main feature, but not long ago I have read a review complaining that this system makes the game burdensome. So presently players are very easy to be weary of any new version of game, this is why the game developers don’t want to take the risk to alter the game program any more.

Nice read and a much needed insight to an aspect we hardly hear about, sadly.

There is a twofold issue here as we have one group conplaining about the issue at hand. Then there's the people who want their return on investment squeezing as much as they can from the Devs. So I can imagine how the Devs and teams holding them all at bay.

So how come we end up feeling ripped and fan the flames for us to leave?

I'm sick of getting the soft appologies from the feedback team about an issue and there's only so much they can actually do, even though they themselves see the problem.

Now behind the scenes must be the same, just like a theatre performance, things may look good and as long as the show goes on, us the audience do care.

Which brings me to your article, why if there are way to get the problems, do we all leave these games because nothing is getting done? Maybe things are being done, but on the wrong things if you ask me.

I'm yet to have a dev or feedback person contact me a month later and ask if things are better. Just like the players, maybe the Devs etc. focus too much on the bad too. So maybe it's a matter of changing "how" it's done and the procedure to fix it.

Napolian even understood that you cannot get people to do what you want, but offer them a ribbon and they'll put their life in your hands. So just maybe by asking how the problem could be solved could be another option.

Holy walls of text!   j/k  Some very insightful posts for a very young science (social science)

I agree that people need to realize that they are not that important (see Twitter heads) and that ultimately it is the decision of the service provider to determine what service they will provide.

However....I think it is important for game companies to open the doors a bit and provide feedback TO THE CUSTOMER. 

There is nothing more fustrating than to see a "blue" post from the game company on a forum to some stupid random comment but absolute silence following a well thought out post on a serious game issue.  For instance, I'm cruising the forums the other day and found a post relating to a controversial game mechanic.  The OP makes a very orderly and logical argument, that at the very least deserves some critical thought, and page after page of responses there was no "official" reply.  Some troll makes an asshat comment regarding something unrelated in the SAME thread and the Community Manager decideds to reply to the 2 lined troll bait of a post.

In the absence of good information (from the game company) bad information will fill the void.  People left to their own "theory crafting" will come up with every conspiricy and left field reasoning to explain why something was changed when there is no official word from the horses mouth.

BUT, the flip side of being "open and transparent" is that the customers will hold an official post from the company to its word....even years down the line.  If a Community Service Manager makes takes a stance on a particular issue that is in development and a year later things turn out different.....the forum warriors will quote that CSM from a year ago and want to know why the game company lied!!!???!!!

The reason so many game companies are secretive with their information?  Maybe because they get screwed to the wall everytime there is any kind of inconsistancy in what was said on the forums and what was done in game.  I say game companies need to grow some thicker skin and tell it like it is.  I'd prefer honest open transparency and get told "We changed our mind" than to be left in the dark and not be told anything.  But thats just me.

Thank you Sanya for a very good read.

I have posted many times that the majority of players do not read or join forum.  Forum regulars really do not understand how that impacts the wieght of thier opinion to the devs.  I am very pleased to see here how feedback is reguarded in that light.

This is a good read but I have a fundamental disagreement with what the Author purports in her usefulness of feed back. The most of the time before player feed back is requested a decision has been reached on what they are seeking feed back on. The time in which it takes to get valid feed back (Take the feed back and analyze it) the time period between this is too long. The design team has reached its conclusion by the time community has come back to them weeks later saying hey the users love this feature. Mean while the designer is often saying to themselves "So what you want me to undo 3 weeks worth of work drafting out this system?"
There are two major problems with feedback systems one an appropriate tool could resolve the second it cannot.
1. The time it takes to get feed back is often too long critical decisions are made and often they are onto the next issue. A tool could help with this process but there are issues with this as well. You have to ask the right questions so shape the specific type of feed back you need. If your questions are too general you have more work to do on the distilling of the feed back. Too specific and the community wont get their creative "juices" flowing.
2. You have to ask yourselves what kind of people are behind the scenes. Are they humble people who can take feed back or are they multiple? The reality is these people are VERY smart they understand aspects of game mechanics that the average player never even takes into consideration. When the basic premise of the game comes to light there a specific feel or idea that the Senior Design people are trying to get into the game, and once they are target locked they honestly don’t have the mental time to spend on feed back. This is even when that feed back is immediate.
So the part that I disagree with "The most common misconception in the MMO universe is that MMO studios do not care about feedback.". While the studio may desire feed back. The studio is made up of many types of people. The people who the feed back should make a difference too tend to really be oblivious to it. They would if they had 100 years to make a game implement every feature that was suggested. They'd make a individual game for each of us. But the reality they are there to make the game that was on the original design documents that was approved and funded. The more they deviate from that the more the cost of the production time the less money the Game makes.

Any ways thats my two cents.

Merxion
 

Sanya,

I appreciate your well thought out comments about how development teams use feedback and where that feedback comes from.  I will grant that the development companies must listen, but I have doubts about how well they understand.  

One of the most enjoyable MMO's that I have played was SWG.  I started with the opening day and played until changes made by the development company made the game virtually unplayable.   The changes were made over time and despite massive player resistance.  Did SOE hear or see the input?  They must have, but nonetheless, SOE chose to destroy an outstanding game.  Actions like those of SOE make players wonder if their input has any effect.  The changes in SWG must have been for some reason in the minds of SOE, but those reasons clearly had little to do with the player base. 

I would agree that game developers and companies should be more tansparent in their reasons for changing a game (or not changing), but it will take a long time to creat understanding let alone trust given past experience.

Robet DeWalt

veteren of EQ, DAoC, EQ2, SWG, CoH/CoV and LOTRO

I enjoyed your article this time Sanya.  Your experience with these topics lends obvious credibility to your point of view and it's nice to have a bit on an inside look at how all this works.  In my opinion the most important point you made was the comment on answered prayers.  If we could all be honest I think we'd admit that we most often want, nay demand that we get a "yes" answer.  Heh, that tends to frustrate me some.  Oh and just for the record, my Foozle is plenty powerful.

lol, Hibernia's original PVP Zone is the golf course she was talking about. I miss those days.

Oh I believe that developers listen to feedback... I have always believed that.

Sometimes its good, sometimes its bad and sometimes its taken somewhere that wasn't asked for.

Issues in general:

1) The loudest voices are usually not the majority of players.

2) As seen with many MMO's (SWG a good reference) the "forum game" is in general a very small percentage of the "game".  Thus people who "play the game" and "don't play the fourms" log in one day to a totally screwed up experience.

3) If a lot of negative feedback is given do you actually see a drop off in subscriptions to tie these things together (aka is it just some unhappy vocal people but most people are happyily playing).

They are all kind of the same thing and obviously tied together.

On the other hand I also know that developers sometimes seem:

1) To lie

or

2) To be mis-informed

or

3) have very potent drugs in their systems.

Tho I use SWG as a reference to some of this..

Dark Age of Camelot and Mythic Entertainment are THE emboidement of ALL of these things.  Which somehow seems ironic...  and that goes back many years when a certain someone was still doing the community updates on the Camelot Herald.

My favorite two arguements from developers when their views do not mesh with feedback:

1) You have rose colored glasses.  (yet developer cannot explain why their awesome idea have resulted in one server and the loss of some 55,000 to 60,000 concurrent online players).

2) We were bleeding subscribers (yet developer cannot explain why they had 250,000 subscribers when they were bleeding but only 50,000 when everything is "ok").

Is it really that hard to see why the people who give feedback but don't post on forums think they are not heard?

I think the entire subject is to wordy to get into and the article was a good read and illustrates much of what I think personally.  Oddly most people who played a game and left almost always have a common time frame that the game was "no longer fun" even tho they may not agree on much else.

That "time frame" when things were no longer fun is 99% tied to a change that was made which no one seems to have wanted or asked for.

Now how all that ties together is... changes are made based on feedback.

That feedback doesn't represent the majority of the player base.

Or was most likely given by someone who just rage quit and isn't coming back anyway.  (which is why the people left behind don't know why the game was changed the way it was... which just moves the cycle onward).

Obviously Devs have access to data we don't.. but the main issue is lack of communication.  If there was more open and constructive communication then more people would be happy.  Happy players = subscribers = money = happy devs with jobs.

I think a bit part of this is simply the design and implementation process at most MMo studios as well. If I have learned one thing from all my many years on "test" servers, submitting feedback and talking to the occasional dev it's that by the time nearly anyone in JQ Public category gets to see a "change" or "New feature", it's too late. The train has already left the station so to speak. You may be able to slightly, very slightly tweak some minor details with good feedback, but the change/expansion is basically happening anyways because the dev team has already spent XX days of paid time on it and no managing exec is just going to willingly write if off and say "oops, mabe that was a bad idea after all".

I loved the article, and I so miss the golf course. 

I think the problem is that developers are champing at the bit with ideas they want to implement and when they see player feedback say: " We want the Frontiers to be bigger and the keep setups to change, running in a cirlce up to the lord room is silly" , developers attach their ideas to what the player has said instead of just giving us what we asked for.

Example:

Players": We want a bigger frontiers with more travel paths to run around in.

Developer:  Oh sweet!  Let's add triple the size of the frontiers and add in places that are "Too Steep" to run across. While we're at it,  let's add in boat travel routes instead of portals. But we can add in a system of teleports too at every keep, that way people will only need to run from keep to keep if the portal system is broken. Oh and we can add bridges at every keep.

Players: We want keeps design to be updated. Currenty they are all just an open box with one room and one entrance to the lord.

Developers:  Oh nice, we can add in destructible walls. That will take forever to destroy giving it the feel of a real world assault on a keep. Let's also add in towers that are shaped like a box and have on entrance to the keep lord.  Oh we can put swamps around the keeps so people will run 1/5th their normal speed, further slowing down RvR and travel.  Don't forget to put a lot of water around keeps so people dont lose health while sneaking up on the castle.  It makes it near impossible to escape the keep if they are failing too!!  

It was simple. We wanted bigger keeps that had better design. We didn't ask for standstill combat and blanket additions to turn RvR from dynamic warfare to keep sieging.   We wanted bigger frontiers to run around in with multiple access to them so when the milegates were zerged or camped by the stealthers we had an opportunity to port into a second location via port!

Part of the problems with feedback in my opinion are :

Developers looking for a moment to exploit feedback to implement designs they want. Then blaming the players saying "You asked for this!""    No.. that's not really what we asked for...

Developers really not caring about feedback. " When we created Dark Age of Camelot we always had it in mind that siege warfarce would be a core concept of game design and the focus or our RvR experience where large battles of over 200 players would crash together."  ---- Except that's not what was bringng enjoyment to the game for everyone. The players loved many aspects of RvR not just the large battles and keep sieges.

Developers not asking the right questions to narrow down exactly what that feedback is looking for.   - So when you guys say you are unhappy with keep design what exactly are you asking for?   Is it the difficulty, the shape, the lord room's tower in particular?

Here's hoping some miracle worker with a good listening skills can put together a game that could be dubbed DaoC 2: The Realm Wars Created Right.

Originally posted by ericbelser

...but the change/expansion is basically happening anyways because the dev team has already spent XX days of paid time on it and no managing exec is just going to willingly write if off and say "oops, mabe that was a bad idea after all".

I think this is where the best companies (like Blizzard) really differ from the rest of the pack. 

Nobody always has perfect ideas, and throwing ten times as much money in the wrong direction just results in a pile of crap that's ten times bigger.

The difference is that good designers aren't afraid to pull the plug on bad ideas, eat their losses and get it over with.   Most companies have a feedback disconnect on the INSIDE.  No amount of "customer community dialogue facilitators" are ever going to fix that.

 To the OP:

Great article. 

Thanks!

Good article, and one that can be expounded on somewhat.

The utility of feedback isn't just sabotaged from the customer end.  While customers do tend to give crappy feedback ("I like this class, so make them capable of doing everything and I'll be happier"), we also have to consider the flaws on the other side of the screen.

What flaws?  Let's say a lead developer has a certain notion of how raids should work.  They put some folks on a design they want, then go out to get feedback on it.  But if they aren't objective enough, they start skewing their viewpoint on the feedback.  If feedback is negative to the new design, they start looking for reasons for the negativity other than poor design on their part.  If the feedback is positive, they will frequently refuse to put the design through rigorous testing to make sure nothing really bad will come from it.  They sabotage the customer feedback this way by not remaining objective themselves

Or let's say that a certain developer believes a class is imbalanced and wants feedback on it.  But the developer never bothered to play the class themselves; they only take second-hand accounts of what the class is doing in the game to base a decision on.  No matter whether the reaction is positive or negative, they make a poor decision because they aren't getting any actual face-time with the class, just using other players to gauge the extent of the change.  They sabotage the customer feedback here by being less informed on the issue than the customer is, and that's just plain not acceptable (after all, we're paying them to be professionals so we don't have to be).

Blizzard and SOE have both been infamous for being somewhat sloppy about checking their data in a timely fashion, and often seem astonished when the data supports something the community has been trying to tell them about for some time.  The fact is that you often see better math skills posted on the official forums at times than you see in the patch notes when some change is made, and that's just shameful.  I don't expect devs to be perfect, but some of the seesawing being done in many MMOs these days is getting ridiculous.  If they don't have the supposed math skills needed for programming computations of this nature, why were they hired in the first place?

Perhaps that is what we need in Quality Assurance departments in these companies: a division between customer feedback, and hard number crunching.  Get some sociologists working on the customer end, do some questionaires, see if they can pick up the signal out of the noise on forums and such.  Then get some folks, ideally programmers and mathemeticians, to look at the numbers being debated, see how things add up, and if not find out why.

One last thing that might help is if some of us customers stopped acting like $15 a month meant that these people had to do what we told them to do.  We should also be a bit more objective.  Is a class overpowered because they beat another one?  Or is there a defect in the loosing class that needs to be addressed, rather than nerfing the other class?  Is something "challenging", or is it really just too difficult to the point of stupidity?  We're paying to play a game with other people, not above them.

I think if we were more realistic about what was going on in a game, and the companies were more interested in being more careful about analyzing potential issues, we'd get amazingly better results.

Another good article from Sanya. I remember the grab bags and Sanya's CM days for DAoC, although I played in Europe, not the US. The Herald Grab Bags were us Europeans' way of getting most of the info available and I remember back in the day that Sanya raised the bar for CM feedback.

Semi-related to this thread we have an event occuring in Warhammer Europe on Monday 27th July which is related to a lot of the issues raised in this article. Rather than waffling on, I'll just link you to the thread here on mmorpg.com,

http://www.mmorpg.com/discussion2.cfm/thread/244858/Warhammer-Europe-27th-July-2009-Europe-Stand-Together.html

For further info, feedback or if you wish to support the event, drop by my blog at warofalts.wordpress.com

In my opinion two very important points are missing.  I accept your expertise in the area of community relations as a given and perhaps you didn't make your blog article exhaustive, however, since you've posted your thoughts and allow for replies I'll put in my two coppers.

First I read nothing about attempting to define the community.  You seem to define "community" as if it's something static and monolithic and that it should be accepted without some attempt to understand it's component parts.  It stands to reason that knowing just who you are dealing with is an important element in determining what to do with community feed back.  You kind of approached this idea when you were writing about direct feed back, but it was done in a way that made it sound like the poster/written feedback was almost inherently untrustworthy.  Focus groups have some of that same "unreliability" built in as what one focus group absolutely hates another might absolutely love.  Defining the demographics of community gives context to the feed back received.  It also helps you anticipate what kind of response is expected on your end.

Second, while feed back can be relied on to point to potential problems in any product I don't think it can be expected to point the developers to the reason for the potential problem.  To use your own example if no one is complaining about a class of character then it is probably hideously overpowered.  However it does nothing to tell you exactly why it's overpowered.  Usually the combination of several factors makes something overpowered.  Feed back usually points to the result (i.e. XYZ class is overpowered because it always pwns me and that's just not possible because I'm an uber player with mad skillz) instead of the root cause of the problem.  I think that feed back should be used to bring attention to potential problem areas, but it then becomes incumbent on the Devs to study the potential problem and make a determination on exactly what the fix to that potential problem is or even if there is one necessary.  You touched on this with your thinly veiled poke at Dark Age of Camelot but again stopped short of making the point and instead satisfied yourself with saying something along the lines of, if everyone is playing it and no one is complaining then it's broken.

As a community relations person your bias is always going to be toward the "open" end of the communication spectrum where devs slug it out on public message boards with the actual players.  I understand that one of the best ways to get people vested in and playing a game habitually is to listen to their imput and respond to it.  However I think that it is important for the Devs to maintain some dettachment from the community to avoid stampede type changes that the mercurial nature of communities dictates.  It is a given that no matter what people have some of them won't like it, will scream for change, trumpet the change when it happens and then a week later sit down and pine for the old way.  That doesn't completely devalue community feed back, but it does put it in it's proper perspective.

In conclusion it is simply this.  The developers are the keepers of the original vision and direction of the game.  The community is probably mostly ignorant of this original vision and direction.  Therefore without context and careful consideration feed back is not a complete or reliable tool for the developers to use.

And shame on you for playing a Fire Controller. :p