In this week's edition of The MMO Underbelly, Sanya Weathers tells us the truth about how Betas work. Find out how people are selected and some other behind the scenes anecdotes.

I stuffed all three hundred mailers, took them down to the mail drop, and stuck them in the bin that says absolutely no tampering or you are doing hard time with roommates bearing poorly spelled tattoos. I got back to my desk, and started looking for that Eddie Izzard disc. You know, I’d done my work, I was ready for a reward.

I popped the disc into the computer, but instead of a cross dressing surrealist, I saw the install window for the beta I was working on.

I briefly contemplated breaking into the mailbox. Instead, I trudged to my boss’s office, confessed, apologized, waited for him to pull up Eddie Izzard on Google, apologized again, and went back to my desk. Then I held my breath for two weeks, waiting for someone on the beta list to call me in a complete tizzy.

Read it all here.

ROFL!!! Sounds exactly like a recent discussion I've heard over lunch with a coworker. I'm starting to wonder if I may indeed have worked with Sanya in the past, and didn't realize it!

Hah, that misplaced CD story made me cry a little.

Stories like that is why I love 'dev' posts.

The truth about Betas in recent years is that they are early marketing gimmicks and not much more.  Having been in several myself over the years, I know from experience that many of the games GLARING problems which are discovered in Beta are never fixed before release.  How many games have released in recent years where the Beta players themselves were screaming from the hill tops and saying the game is no where near ready to go gold? Conan is the most recently famous example.  I was seeing warnings about that game from Beta players months and weeks before release, all the way up to the last day.

I've given up on trying to Beta new games.  Most of the time they are released in an unfinshed form anyway, so why bother getting in early just to experience the same bugs I will continue to experience later?  If I sound cynical, it's because I am.  This genre is stale and in a sad state of affairs.  Our only hope will be the indy game devs who are willing to take chances on new ideas.

I was a long time DAOC tester  (6 yrs.) Sanya and as always I enjoy reading what you have to say. The day you left Mythic was a sad day indeed for us players and  the Herald never was the same. Although I have moved on it's nice to see you still in the industry and working. I hope to that one day we will meet on the battlefield again. Long live Hel's Black Brigade!

MindTrigger, you should have said "Our only hope will be the indy game

who are willing to take chances on new ideas."  I've yet to meet a dev that wasn't passionate about providing the best game experience possible, but guess who's calling the shots when the dev team says to the boss "Our beta testers have pointing out issues X, Y, and Z and we need another 8 months to finish the game before we're ready for release"?  I'll give you three guesses and the first two don't count.

Back on topic, this was a good article.  One thing that frequently frustrates me when beta testing a new game (and I mean real beta testing, not beta previewing) is when they make submitting bugs either a real chore or their bug reporting mechanism doesn't let you give a complete report.  Reproduction steps can sometimes be quite lengthy and if you stick a size limit on my bug report then I'm not going to be able to give you a good bug report.  Either one will make me stop beta testing.

I was one of the lucky ones to beta test the end all expansions for DAoC, The Trials of Atlantis, I have to admit I was having fun with it, but I was on Gaheris at the time so it didn't matter that I would have to grind out that expansion to do good in pvp. I am an avid beta tester (beta testing 3 major games atm) and I am sooo careful when I fill out beta apps. Good read :)

Originally posted by Daerkannon

MindTrigger, you should have said "Our only hope will be the indy game

publishers

who are willing to take chances on new ideas."  I've yet to meet a dev that wasn't passionate about providing the best game experience possible, but guess who's calling the shots when the dev team says to the boss "Our beta testers have pointing out issues X, Y, and Z and we need another 8 months to finish the game before we're ready for release"?  I'll give you three guesses and the first two don't count.

Back on topic, this was a good article.  One thing that frequently frustrates me when beta testing a new game (and I mean real beta testing, not beta previewing) is when they make submitting bugs either a real chore or their bug reporting mechanism doesn't let you give a complete report.  Reproduction steps can sometimes be quite lengthy and if you stick a size limit on my bug report then I'm not going to be able to give you a good bug report.  Either one will make me stop beta testing.

Well, that's their choice.  Had EVE not been self-published, it would have never happened.  A mainstream publisher will always go for the safe (and often wrong) bet, so as a dev house, you are making the decision, knowingly, to go for the big payoff rather than making a game you are truly in control of and passionate about.  Sure it's a harder road to travel, but it's still a decision that is made by the dev company. CCP made the right decision, and they can take their game in any direction they want without publisher interference.

Beta are some funny things, and thos ppl that want in betas are often also.

I have been in alphas, open/close betas, stresstest and other funny testing fases, and the is always some players that think their are smarter then the whole dev team, even if their havnt tryied the game yet. Always funny to read about how a few testers want to redesigne the whole core part of the game to their wises, when the rest of the testers are happy with the currrent form.

I sign up for beta on games I think may be fun to play, but games I really look forward to, I say away from as much as possible, dont want to spoild the fun, but cant wait for it to come out.

but being dev must not be easy, on one side you have your testers saying this game needs more work, on the other side you have your futere players, that keeps complaing about delays and ask why game is not out yet.

Originally posted by MindTrigger

Originally posted by Daerkannon

MindTrigger, you should have said "Our only hope will be the indy game

publishers

who are willing to take chances on new ideas."  I've yet to meet a dev that wasn't passionate about providing the best game experience possible, but guess who's calling the shots when the dev team says to the boss "Our beta testers have pointing out issues X, Y, and Z and we need another 8 months to finish the game before we're ready for release"?  I'll give you three guesses and the first two don't count.

Back on topic, this was a good article.  One thing that frequently frustrates me when beta testing a new game (and I mean real beta testing, not beta previewing) is when they make submitting bugs either a real chore or their bug reporting mechanism doesn't let you give a complete report.  Reproduction steps can sometimes be quite lengthy and if you stick a size limit on my bug report then I'm not going to be able to give you a good bug report.  Either one will make me stop beta testing.

 

Well, that's their choice.  Had EVE not been self-published, it would have never happened.  A mainstream publisher will always go for the safe (and often wrong) bet, so as a dev house, you are making the decision, knowingly, to go for the big payoff rather than making a game you are truly in control of and passionate about.  Sure it's a harder road to travel, but it's still a decision that is made by the dev company. CCP made the right decision, and they can take their game in any direction they want without publisher interference.

Eve was not self-published, their hade a pulisher, but their bought back their rights to the game after a short time out.

I've been in more than 20 MMO betas, and this article is a great piece of advice for how to get in.

• I got into the UO beta by complete random chance.
• I got into the EQ alpha/friends and family by being positively active on the forums, and they made me found the Guide Program because of it..
• I got into the CoH alpha by being so positively active on the forums they made me moderate them.

Getting into an early beta is all about your presence...filling out the form won't get you in, especially these days with millions of people signing up.  Getting the early slot is all about establishing an identity to the devs.  How do you do this?  Allow me to expand aupon the OP's suggestions:

• As the OP said, avoid drama like the plague.  Don't start it, don't join in, don't try to stop it.  Avoid it.  I don't care *WHAT* your opinion is relative to the topic on hand, everyone in that thread has a black spot on their record.  When threads like this get 4-5 pages long, the devs don't even bother reading it, they just notice who's participating.
• Write FAQ's, helping threads, guides, suggestions, ideas.
• Be helpful in the community.  Don't say "RTFM" to new users, welcome them...a welcome person stays and likely buys the game, an alienated one doesn't.  My honest suggestion is act like you want the community to survive and have everyone in it, not like it's exclusive and new folks are shunned.
• Someone who's an ass on the forums will be an ass in the beta, and in the early beta lots of things are broken.  Someone who is bitching on the forums about something that doesn't exist yet will do so 10x more when they get in the game.  That kind of person may make it to the open beta, but they'll never be hand-picked to be invited in early...why court that kind of misery?
• Don't kiss up.  Be nice, not obsequious.  Be supportive, but not a catamite. 
• Have opinions and express them clearly and concisely.  No one reads 14 page dissertations on what you think PvP should look like, nor do they read your 12th post on how you feel the economy should be like.  Express your opinions clearly and concisely and if the devs want more info, they'll ask.  I've gotten PM's from devs of 9 MMO's, asking me for more info on something I'd posted.  1 of those ideas was actually used in some form.  The devs want early beta testers who will jump into a bottomless pit 3x just to validate the coordinates vs one that complains the first time they encounter it.
• Don't jump on people whose ideas are different than yours.  Don't treat everyone who disagrees with you as if they're attacking you, personally, even if they actually are.  You're not a dev, they're not a dev, neither of you has a clue, neither of you is right, so the argument is not resolvable.  Be the "bigger man" and show how reasonable you are.  You obviously don't have to agree, just don't bicker, it's petty and annoying.

This brings me to what should you do once you're in an early beta:

• Report everything.  Not on the forums, but in /bug, in as much detail as possible.  So what if someone else has seen it?  So what if someone says they reported it.  The number of times a bug is reported is indicative of it's exposure.  A bug that's reported once is less likely to be visible than one that's reported by 50 people.
• If you find an exploit, *DO NOT POST IT IN THE FORUMS!!!*  Do not allude to it on the forums.  Don't discuss it, don't mention it to anyone else.  /bug the heck out of it, and perhaps send a PM to the community manager about it, privately, but for pete's sake don't make it public.  People who believe "the more public it is, the quicker it will be fixed" hurt the game.  And don't keep exploiting it either.  After you've validated it a few times, move on to something else, there's no value in continuing to do it.
• Don't bitch.  Not in game, not on forums, not on your private blog that you jot your dark soul into.  It serves no purpose and only creates negativity.  It's beta, heck it's early beta, stuff is broke.  If you can't test what you want to do due to a bug, go test something else.  Roll up a new toon.  I can't tell you how many times I've gotten a toon stuck, tried everything I could think of, then logged and made a new toon to test something else.  It was only when all 8 of my CoH toons were in some manner of bugged state that I asked the devs for help.
• Don't act like you're entitled to anything, because you're not...early testing is it's own reward.  No, you can't keep your toon.  No, you can't get an exp or loot boost because of all your amazing effort.  No, you can't get some kind of after-release item to display your massive ePeen.  What you can do is get to know the game such that, after release, you have a "head start" play experience-wise, and that's enough of a reward.

Good article, thanks to the OP/Correspondant for putting this out there.

An excellent read and very true. 

I got into the friends and spouse's test and later alpha testing Horizons as one of the devs was a friend of mine.  Aplha tested a few more and closed beta tested some others.

Beta tests these days do appear to be previews. I was in Aion this last weekend.  Still a preview with some needed suggestions is all.  I will not pass judgement as it is not released yet and I really have not seen the rest of the game.  I did have fun though and the world was breath taking for what I played..

I can rermember those long bug report forms and feedback forms for every encounter and quest.

I do like to beta testing and hope to help bring a dev's dream to market. 

One poster put it best and it happenend to Vanguard and Tabula Rasa is investors want to see a return on their money and will override Dev''s when they feel the game is ready or taking too long. 

Both games had great concepts but were in development too long.  They were released before they were close to being finished.

Originally posted by megagame

Originally posted by MindTrigger

Well, that's their choice.  Had EVE not been self-published, it would have never happened.  A mainstream publisher will always go for the safe (and often wrong) bet, so as a dev house, you are making the decision, knowingly, to go for the big payoff rather than making a game you are truly in control of and passionate about.  Sure it's a harder road to travel, but it's still a decision that is made by the dev company. CCP made the right decision, and they can take their game in any direction they want without publisher interference.

Eve was not self-published, their hade a pulisher, but their bought back their rights to the game after a short time out.

And are now free to do whatever they want with the game.  They are self published at this point.

Great article and so true. Question: Is the star-f*cker's reference from the NIN song? If so, great song and they were in NY recently. Sadly, I couldn't get a ticket. Same analogy with getting into an exclusive beta, you need to be an active fan and know when and who to ask. I fell out of the loop on certain beta's and concerts because I just wasn't "There" with it. So I am marching my butt over to the Champ-O's forums and start...being me.

That's gonna help.

lol, good stuff sanya, 

Ink

This article points out everything that is wrong with betas, and it is just another reason why games release in a less than desireable state.

Nice article!

Flag.

 Great read. I also tested a few alphas and betas and even some stress tests in my years of gaming. Good to see some insight from a dev. It seems like alot of betas are pre play for pre orders these days. Last alpha I was in was Rome Gods and Heros. The game had some good play to it. Shame there were so few testers to be found, when I was on. Shame it all folded up. I did Aion thx to mmorpg last weekend and I was impressed. thanks for the discussion!

I would like to reiterate the obvious, which is find a bug and report it to the appropriate place.  But I also want to mention, should you find a work-around, post that as well, with the number assigned to the bug.

Finding a bug...

1. Play the game, try to have fun... but always be on the lookout.
2. Try doing things when there are server shutdowns.
3. Try doing things when the server comes back up.
4. Try doing things when you are logging in or out.
5. Look up information, and see what others have done.
6. If you see something weird in the game, investigate it.
7. If somethig weird happens, try to duplicate it.
8. If you notice an area that has a server slow down (lag) find out why. 
9. Don't confuse server lag with screen lag (zoom all the way in, and stare at the ground... if lag stops it is screen lag).
10. There are new bugs introduced every patch, read the patch notes on what changed, mess with the changes.
11. Never assume that someone else has reported a bug, if you find one.
12. If the server or landblock shuts down when you are doing something, investigate to see if it was from your doing.
13. Keep notes when you find something.
14. Report bugs.
15. Preview your post, at least 2-3 times and edit as much as possible to make the post as detailed as possible, before you submit it.

While you might think that you can use the bug later if it makes it to gold, if you don't post it... post it anyways.  The less bugs a game has, the more likely it will become a successful and fun game.

I have been in numerous beta tests from MMOs to RPGs, and the information above can help you not only make it from a preview to become a regular beta tester, but a more successful beta tester as well.

By the way, Aion ships in September according to my key supplier, with pre-sales happening now.  Last weekend there were 1 million players trying the game out... but likely this is spread across all continents, as my friends in China were mentioning it as well.  So consider that probably about 150k-250k US players?

Originally posted by Dana

 

The main way in is to be an active member of the message board community around your target game. There’s a lot more to being active than posting psychotic twenty paragraphs screeds on arcane design points that may not even be in the live game.

 

Damn, I guess I'm S.O.L. then... (;-_-)

And probably about half of the Star Trek Online fans (who just won't STFU about ship interiors).