Richard Vogel (BioWare): MMO Betas

We’ve all beta’d a game. It’s that special golden age, where actions don’t matter, money isn’t an issue, and players are actually encouraged to find bugs. It’s that period of time where communities are small and selective (closed betas), or overflowing with penniless teens, unable to purchase a subscription to a live game (open betas). They are our golden years, our memories of the beginning.

 advertisement 

Rich himself has run about 4 separate betas, each of which were unique. The beta experience, he said, has changed over the years; players participating in betas used to be hardcore gamers who wanted to test the game, give feedback, and help discover bugs. Today’s beta testers have more selfish goals: we seek to gain advantage by learning the layout of the game and the UI and to try the game before we buy it.

Rich has noticed that players “don’t play beta like they play live.” Indeed, the knowledge of an impending wipe tends to create different play patterns among players – for example, while participating in the Lineage II closed beta, I witnessed a great many “raids” on the Light Elf town by the Dark Elves, a roleplay-related slaughter-fest that would never happen in the real game, where dropped gear can mean the loss of hundreds of hours.

So, back to our topic. “Who”, Richard asked his audience rhetorically, “runs betas?” The answer: a politician: someone who can talk to and handle an excitable and infant community. Community managers need to have thick skins and good attitudes, said Vogel, especially for games based off of IPs (i.e. Star Wars Galaxies) that have rabid, eager fans.

These community managers have tough jobs, and must perform many tasks to keep both development and customers happy. The community manager is one of the most important communication lines between the aforementioned parties, and needs to make sure that everyone is as happy as possible. Vogel mentioned one cardinal rule of this: Never, ever promise something that is not there. “Don’t talk to your audience like a PR guy; they’re not stupid.”

Developers and community managers also have to deal with controlling the community’s flow of information. Vogel’s key method to obtaining this goal is the use of open and closed areas on a website – areas that everyone can access and areas that logged in, confirmed beta users can access. Separate forums where beta users can report bugs, talk about game play, etc. are crucial, or else that chatter will go elsewhere. Any forums on the website must, of course, be monitored – “I’ve always hired 4 or 5 moderators,” said Vogel. These moderators serve to remove troublemakers, while the community manager can interact with and be helpful to members on the board.

Vogel suggests that developers take suggestions from the boards, but not exclusively and with more than a grain of salt. The people on the boards are the vocal minority – only 13% of players are really active on the official boards. The most important variable to consider when making changes in a game are the metrics and statistics – people are complaining that rogues are overbalanced, but how many people really play rogues? What skill do they use most often? What do they specialize in?

Another important point was the actual purpose of beta testing – believe it or not, it’s not there for you to try out a new game! Beta testing is important to work on balance and to find exploits – with hundreds of people playing and discovering bugs, exploits, strategies, and more, the developers are able to see how their game works on a large scale in action, not just in theory. Beta testing is also extremely important for stress testing: if developers are expecting three thousand concurrent on launch, the game needs to be able to handle that number.

Beta testing is also important for gathering statistics, which in turn can help with balancing and other issues. If a game has four class choices, and 70% of people are playing rogues, then there is an obvious unbalance in game play and the developers need to fix that before launch. Beta testing is also useful for marketing, and to predict how a game will be received. One poll question Vogel recommended was “Would you recommend this game to a friend?” If less than 75% of your users say no, you are not ready to launch!

Which moves us to one of the hardest parts of beta testing: when to stop, and launch your game. Launching a game early can be its demise; as Vogel said, “you only get one opening night.” Along with the above recommended poll, Vogel suggested that developers should look at the number of registered bugs (it should be in the 100s, not the 1000s, before you think about launch, and none of them should be showstoppers), server crashes, and the numbers of users. Also do your best to avoid negative buzz: deliver what you promise, make visible changes if things are wrong, and never get defensive or negative.

You can comment on this article here.