Champions Online: How Developers Handle Beta Feedback

A huge part of any beta test is wading through the massive amount of feedback you get from your testers. This input can range from the ridiculous to the sublime, and we're very fortunate with Champions Online to have not just a very vocal community, but also a passionate one. Our players really care not just about the state of the game, but about how they will be interacting in a superhero-based environment and experience.

 advertisement 

A great deal of what we have been doing over the past few weeks is in direct response to the interests and bugs reported by our testers. An excellent example of this is the creation of The Powerhouse. This is a building that exists in every major zone where heroes can try out powers before they lock them into their power framework. A "buy before you try" concept was brought up in a forum thread about training powers and the difficulties of selecting from the vast array available. It was a great concept, and we worked hard to get it into the game as quickly as possible.

Much of what we glean from a beta test comes in the form of bug reports. We have a pretty amazing bug reporting system within the game that not just allows players tell us what's wrong, but lets them track the open issues. It provides a list of missions, including all the ones completed by the player reporting it, which can be selected from a drop-down box. It also provides a browse and search function for reported issues and a search feature to see if others have reported the same bugs. If you find a bug has already been reported, you can press a button that adds you to the list, or you can add more info to the issue. You also have a separate dialog box where you can keep track of your open issues, see their status, and update them or even close them. Tools like this allow us to gain the maximum we can from our testers.

The most difficult thing we have to do is deal with testers that are passionate about the game, but communicate that passion in ways that aren't constructive or definitive. Their angst or concerns are evident, but there needs to be the ability to have a rational and measured discourse about the issues. Also, it's important to listen to what is being said as opposed to transposing fears or unfounded beliefs onto what any developer is saying.

On our end, we have to work to look past when a player is fuming to see the core issue at hand and try to address it. The recent change in the game's experience curve during open beta is a great example of this. A group of very vocal and passionate players became enflamed over the changes. The experience shift was indeed dramatic, but we quickly worked to inform players that this was not the last change, and that we were going to be moving things back the other direction. The passionate discourse began, and while some people were willing to see what was going to happen with the next course correction, others would not be swayed from their belief that we had utterly destroyed their game and wouldn't be doing anything to make it better. Ultimately it comes down to realizing that no developer wants to make a bad game or do something to hurt their players, even when you're upset. And on our end as developers, we have to keep cool and keep talking to our players.

And after all, both in beta and afterwards, communication is the key. Staying connected to your player base is essential to the longevity of the game. Understanding their needs and concerns and making sure they know what you're doing to address them is vital. We've worked very hard throughout the beta to do this, and that dedication to communication is paramount to our live service. So keep talking, people - we're listening!