Sanya Weathers's MMO Underbelly: The Truth About Betas
How do developers really decide who gets in and who doesn't? That and other behind the scenes stories of how Betas really work.
I got to play in the Aion beta last weekend, and you didn’t.
;)
It was this or a photo of
Sanya giving everyone not
in Beta the finger.
“What the hell,” you say. Why’d SHE get to be in the beta and not me? You correctly point out that you’re more experienced, a better player, more skilled at PVP, and probably better looking, too.
This time, the answer was “press account.” But most of the time, it’s… luck.
That’s it. The game studio throws you into a pool with all the other fish, and when they need five thousand testers, you just have to hope you were swimming near the floodgate.
Actually, there are two things you can do, and that’s “read the minimum requirements” and “fill out the form completely.” Beta builds often have higher system requirements than do the final games, because optimization is a lower priority than stability. But studios know that no one reads the system requirements, which is why the forms often ask you what sort of video card you have. If you leave that form blank in any of its particulars, you are low hanging fruit to be whacked off the branch before anyone harvests beta users. Jeremy Dalberg, community manager for Webwars, says “The number of people who don't bother to tell us what video card they have, even though we ask, astounds me. I just throw those out.”
Wait, you’re saying you want to be a real beta tester, not one of the swarming masses allowed in to load test the server and help the studio build buzz for an impending release? Oh. You mean a *real* beta in the original sense of the word. Sorry, my bad. The term “beta” has been watered down by every chimp with a website claiming their three year old product is really in beta so you can’t hold them responsible when the server eats your lunch money. That has in turn trained the market to expect a beta product to be finished and to charge you money.
But you are special, and you understand that betas products are incomplete, lack any semblance of balance, may be prone to crashing, and are short on content until key systems are integrated.
;)
Also used to pick Beta testers?
Go ahead, make the joke about whatever MMO you hate the most right now while I wait. It’ll be funnier and cruder than any joke my evil editor will let me make.
Hey, speaking of funny, crude, and MMO betas: Years ago, I was sending out beta CDs (for a Camelot expansion, if you’re wondering). It was one of those horrible in between beta phases. We were far enough along that having people download the beta would have crippled the ability of our paying customers to get the regular game (and before you explain how that’s not possible, it was, it sucked, I’m still bitter, and I cannot talk about it). But it wasn’t one of the big massive phases in beta where it would have been cost effective to farm out the CD-mailing job to a contractor. So the task of stuffing mailers with CDs and labeling them fell to me. My little desk was buried in an avalanche of discs, mailers, labels, and lists.
A friend of mine came by and said, “Hey, I burned that Eddie Izzard show to disc for you, you can watch it on the computer!” He’d been telling me for weeks that I would love Eddie Izzard, a cross dressing surrealist comic, and I kept saying, sure, I’ll go out and buy it and watch it, in all my free time, jerk. Anyway, he burned the show for me, because he was a sweet guy. I thanked him, and tossed the disc on my desk.
You see where this is going.
I stuffed all three hundred mailers, took them down to the mail drop, and stuck them in the bin that says absolutely no tampering or you are doing hard time with roommates bearing poorly spelled tattoos. I got back to my desk, and started looking for that Eddie Izzard disc. You know, I’d done my work, I was ready for a reward.
I popped the disc into the computer, but instead of a cross dressing surrealist, I saw the install window for the beta I was working on.
More Developer Perspectives Features:
More Columns:
More Features:
ROFL!!! Sounds exactly like a recent discussion I've heard over lunch with a coworker. I'm starting to wonder if I may indeed have worked with Sanya in the past, and didn't realize it!
Hah, that misplaced CD story made me cry a little.
Stories like that is why I love 'dev' posts.
The truth about Betas in recent years is that they are early marketing gimmicks and not much more. Having been in several myself over the years, I know from experience that many of the games GLARING problems which are discovered in Beta are never fixed before release. How many games have released in recent years where the Beta players themselves were screaming from the hill tops and saying the game is no where near ready to go gold? Conan is the most recently famous example. I was seeing warnings about that game from Beta players months and weeks before release, all the way up to the last day.
I've given up on trying to Beta new games. Most of the time they are released in an unfinshed form anyway, so why bother getting in early just to experience the same bugs I will continue to experience later? If I sound cynical, it's because I am. This genre is stale and in a sad state of affairs. Our only hope will be the indy game devs who are willing to take chances on new ideas.
I was a long time DAOC tester (6 yrs.) Sanya and as always I enjoy reading what you have to say. The day you left Mythic was a sad day indeed for us players and the Herald never was the same. Although I have moved on it's nice to see you still in the industry and working. I hope to that one day we will meet on the battlefield again. Long live Hel's Black Brigade!
MindTrigger, you should have said "Our only hope will be the indy game
publisherswho are willing to take chances on new ideas." I've yet to meet a dev that wasn't passionate about providing the best game experience possible, but guess who's calling the shots when the dev team says to the boss "Our beta testers have pointing out issues X, Y, and Z and we need another 8 months to finish the game before we're ready for release"? I'll give you three guesses and the first two don't count.
Back on topic, this was a good article. One thing that frequently frustrates me when beta testing a new game (and I mean real beta testing, not beta previewing) is when they make submitting bugs either a real chore or their bug reporting mechanism doesn't let you give a complete report. Reproduction steps can sometimes be quite lengthy and if you stick a size limit on my bug report then I'm not going to be able to give you a good bug report. Either one will make me stop beta testing.
I was one of the lucky ones to beta test the end all expansions for DAoC, The Trials of Atlantis, I have to admit I was having fun with it, but I was on Gaheris at the time so it didn't matter that I would have to grind out that expansion to do good in pvp. I am an avid beta tester (beta testing 3 major games atm) and I am sooo careful when I fill out beta apps. Good read :)
Well, that's their choice. Had EVE not been self-published, it would have never happened. A mainstream publisher will always go for the safe (and often wrong) bet, so as a dev house, you are making the decision, knowingly, to go for the big payoff rather than making a game you are truly in control of and passionate about. Sure it's a harder road to travel, but it's still a decision that is made by the dev company. CCP made the right decision, and they can take their game in any direction they want without publisher interference.
Beta are some funny things, and thos ppl that want in betas are often also.
I have been in alphas, open/close betas, stresstest and other funny testing fases, and the is always some players that think their are smarter then the whole dev team, even if their havnt tryied the game yet. Always funny to read about how a few testers want to redesigne the whole core part of the game to their wises, when the rest of the testers are happy with the currrent form.
I sign up for beta on games I think may be fun to play, but games I really look forward to, I say away from as much as possible, dont want to spoild the fun, but cant wait for it to come out.
but being dev must not be easy, on one side you have your testers saying this game needs more work, on the other side you have your futere players, that keeps complaing about delays and ask why game is not out yet.
Well, that's their choice. Had EVE not been self-published, it would have never happened. A mainstream publisher will always go for the safe (and often wrong) bet, so as a dev house, you are making the decision, knowingly, to go for the big payoff rather than making a game you are truly in control of and passionate about. Sure it's a harder road to travel, but it's still a decision that is made by the dev company. CCP made the right decision, and they can take their game in any direction they want without publisher interference.
Eve was not self-published, their hade a pulisher, but their bought back their rights to the game after a short time out.
I've been in more than 20 MMO betas, and this article is a great piece of advice for how to get in.
Getting into an early beta is all about your presence...filling out the form won't get you in, especially these days with millions of people signing up. Getting the early slot is all about establishing an identity to the devs. How do you do this? Allow me to expand aupon the OP's suggestions:
This brings me to what should you do once you're in an early beta:
Good article, thanks to the OP/Correspondant for putting this out there.
An excellent read and very true.
I got into the friends and spouse's test and later alpha testing Horizons as one of the devs was a friend of mine. Aplha tested a few more and closed beta tested some others.
Beta tests these days do appear to be previews. I was in Aion this last weekend. Still a preview with some needed suggestions is all. I will not pass judgement as it is not released yet and I really have not seen the rest of the game. I did have fun though and the world was breath taking for what I played..
I can rermember those long bug report forms and feedback forms for every encounter and quest.
I do like to beta testing and hope to help bring a dev's dream to market.
One poster put it best and it happenend to Vanguard and Tabula Rasa is investors want to see a return on their money and will override Dev''s when they feel the game is ready or taking too long.
Both games had great concepts but were in development too long. They were released before they were close to being finished.
Eve was not self-published, their hade a pulisher, but their bought back their rights to the game after a short time out.
And are now free to do whatever they want with the game. They are self published at this point.
Great article and so true. Question: Is the star-f*cker's reference from the NIN song? If so, great song and they were in NY recently. Sadly, I couldn't get a ticket. Same analogy with getting into an exclusive beta, you need to be an active fan and know when and who to ask. I fell out of the loop on certain beta's and concerts because I just wasn't "There" with it. So I am marching my butt over to the Champ-O's forums and start...being me.
That's gonna help.
lol, good stuff sanya,
Ink
This article points out everything that is wrong with betas, and it is just another reason why games release in a less than desireable state.
Nice article!
Flag.
Great read. I also tested a few alphas and betas and even some stress tests in my years of gaming. Good to see some insight from a dev. It seems like alot of betas are pre play for pre orders these days. Last alpha I was in was Rome Gods and Heros. The game had some good play to it. Shame there were so few testers to be found, when I was on. Shame it all folded up. I did Aion thx to mmorpg last weekend and I was impressed. thanks for the discussion!
I would like to reiterate the obvious, which is find a bug and report it to the appropriate place. But I also want to mention, should you find a work-around, post that as well, with the number assigned to the bug.
Finding a bug...
While you might think that you can use the bug later if it makes it to gold, if you don't post it... post it anyways. The less bugs a game has, the more likely it will become a successful and fun game.
I have been in numerous beta tests from MMOs to RPGs, and the information above can help you not only make it from a preview to become a regular beta tester, but a more successful beta tester as well.
By the way, Aion ships in September according to my key supplier, with pre-sales happening now. Last weekend there were 1 million players trying the game out... but likely this is spread across all continents, as my friends in China were mentioning it as well. So consider that probably about 150k-250k US players?
Damn, I guess I'm S.O.L. then... (;-_-)
And probably about half of the Star Trek Online fans (who just won't STFU about ship interiors).
LOL... The reciever must have been baffled but also been happy to see free movie copy!
or worst stil is that he being a noob thought that the cd is actully a beta phase tried to decipher how to play it!
and any way Gr8 post and GAME ON ppls
Nice article. Its basically saying be a friendly person, and don't act like a moron. Not only will that help everyone in life, but it also will help you get into beta. Funny that.
Personally, I beta tested one game. I hated it, and won't do it again. I found myself never logging in, and probably was taking someone's spot who would have done a much better job. I just don't like the surprise of a new game to be worn out when it releases.
As always, great article Sanya. I admire your writing talents.
ehhh, if only this was true at least for 10% of the games involved... seems like these days no matter how well you present your view on some problematic area, its always a purely subjective decision whether that feedback actually goes anywhere beyond the beta forums. just go say 'this is stupid, you shouldve done it in that way, because bla bla, etc etc' and if your lucky you get infraction, if not ban ) too bad companies and especially the people involved with talking to/getting feedback from the community cannot realize the simple truth - no matter how skilled your dev team is, a crowd of milion will always think of something you havent
My last beta test was Age Of Conan, I swore I would never pareticipate in another beta after that.
A good read, as always.
My first beta was Anarchy Online. I still have the CD in the original airmail envelope. Unfortunately, although I eventually managed to get it to install (only took about three days and large chunks of my scalp), I could never get it to run. But then, it barely ran when I got the launch version, so no surprise there.
For a few years after that I applied for many MMO betas and got into most of them. Big ones like EQ2 and small ones like Rubies of Eventide (both of which I still play). All I ever did was fill in the application and forget about it, although occasionally I might have also been active on the forums. Mostly I'd find my invite coming in the later stages of closed beta, somewhere from two to four months before open beta rolled around.
For a long time i found the whole process fascinating, but eventually I realised that beta-testing a game you actually plan on playing isn't such a great idea. Several games I betaed were clearly more entertaining and enjoyable for me in beta than they were when they finally went live. There were late changes made in two that I can think of that, in my opinion, were materially significant in the problems those games had at launch, and which were reversed later. I'd actually have enjoyed both of those games more at launch had I not known how much more fun they'd been just a few weeks earlier.
Nowadays I generally only apply for betas of games I don't intend playing when they go live. I'd rather come to the ones I am really interested fresh.
Uhhh... what about all the 14 year old mentally disabled people who have multiple beta accounts in every beta or earlier stage of development I have ever ever played in or heard of? The way many people get into beta is simple, and it's identical to the method many people use to reach "success" in real life, but it's even easier to implement this strategy over the internets:
1) LIE.
2) Kiss ass.
It's funny to read the dev comment about not kissing ass. I got a huge laugh about that. Then some master beta dude in this thread writes a whole long post about how to kiss ass with level 9 mastery and how it got him into 32 betas, but then tacks on the caveate, "but don't kiss ass". LOL. Every development community I have ever seen was quickly staffed by the #1 ass kissers on the official forums/irc [they get nominated to be moderators usually, and even admins in indy projects like Darkfall for example], and these are usually among the first of the wider community vanguard brought into "beta" after the friends and relatives stage.
I was the one who recieved the Eddie Izzard CD. Thanks, btw. It destroyed my marriage.
HI Sanya
well after retirement from five years of Daoc gaming and attending 2 orlando fla road shows anf a five year trophy i am free floating around looking for a better game hoping i can get a key for Aion cross your fingers for me nice to see i crossed paths with you once again in my gaming life hope you are enjoying this much more than where u were not that it was a bad thing lol happy hunting catch you on the flip sirbillyboy / carlie
Wow, pretty demanding... people who are sincere, friendly, no drama... gawd then we all here are doomed. lol
Besides, it cant be true. According to enough people here, I am the most visicious, trolling, evil, illiterate drama queen that exists, and I constantly get into closed betas. hrhr
Ok I am honest. I always tell straight of the bat what I think. Does that balance the bad character I have? Ok then maybe you get me into that SWTOR beta pls. XD
Same here. I won't alpha or beta another game unless I know for certain the devs will put the test back in.
And interesting read. I cant help but be an asshat on internet forums most of the time, just make it so much more entertaining. best off avoiding posting if I want a slot tbh.
I like your writing style and story telling. I just wish you caricature didn't seem so much like Jerry Seinfeld.
Nice article Sanya. I am already seeing little kiddies writing a love letters to the devs of the game they want to play early.
"Deer developr. I luv ur game so much, that I be good beta task master. Please please please let me in. U will like mah betah testring skillz. I promize. I testid many games. I testid "mmorpg.com", I testid little doggis on my little consil I press som button and doggis make funny sounds. I testid all games that those people talk on mah TV. My TV has little doggis on it. They make sound if U touch it. So yeas. Pleees, let me in Ur game. U will luv mah beta testrink."
QFT. I can't beta games anymore as well. I've tried many times, I've tried being a "positive" community member and encouraging to the devs to fix the issues that I myself have found as well as major issues that a game has and to see that stuff go live when it's being reported for several months was enough to kill my desire to beta test.
P.S. I loved the story regarding the sending of the wrong disk to a potential beta tester. Reminds me that I am the king of "mistells" in games. I cannot count the times that I have sent the "I hate that ****ing jerk" tell to a friend only to realize that I sent it to the person I was talking about, or sending random crazy comments about guildies (not mean) and then realizing that I just typed it in guild chat. I once made a comment to a friend that the wife of our guild leader had a very sexy voice in vent and that I'd like to show her a good time only to realize I actually put that in guild chat. Thankfully he was very good natured about the whole thing.
Same here. Gave up on Beta testing. I mean real bug exploration. Now if I get in Beta I just see what game is like. Most of the games in Beta stages in most cases are the same
as they go into Open beta and then get released.IN any way people supposed to sorta "work" in Betas, helping devs to catch the bugs that were potentially missed. A lot of people just play the game and argue about its features and just being morons on general chats.
That was a great article. It was well-written and interesting. However, you need to remove the comment under the picture of the tranny that says "Definitely not a DAoC Beta client!." That's unbelieveable that you would put that there. How fucking bigoted can you get? Why ruin a good article by showing your social ignorance? Seriously.
Have you smoked yourself retarded? Is there any chance that the picture this caption is attached to is the person who does the performance she references and that she sent a CD of to a beta tester? Way to show your ass without pulling your pants down by making your social commentary without first comprehending what you read. /golfclap
one thing that i think has helped get me in a hand ful of betas is agregae experience. My first beta 5 years ago was City of Heroe but i was active in the forum 2 years before that when they still had the "open power types "as their promo on the site (sorta like what Champions Online does now)
Since then whenever i applied for a abeta i always include my past experience. My current resume would looke like this.
Previous Betas?
City of Heroes
City of Villians
Matrix Online
Champions Online
Tabula Risa
Okay, I LOL'd at this part... we've all been there!
And @Sanya, I hope you eventually got around to watching Eddie Izzard. I've seen a few of his stand-up acts, and without question (IMHO) the best one is "Dress to Kill". He has also appeared in quite a few movies and TV shows lately... very funny guy.
Have you smoked yourself retarded? Is there any chance that the picture this caption is attached to is the person who does the performance she references and that she sent a CD of to a beta tester? Way to show your ass without pulling your pants down by making your social commentary without first comprehending what you read. /golfclap
Who jerked your chain? My comment was not directed at you, now was it? It was to the author of this article and we have already conversed about this. Frankly, it's none of your effing business.
Regardless your OPINION, there are people of different persuasions (AND opinions) that are gamers and some would be offended. Not everyone lives in the UK and knows who that comedian even is, for that matter. I'd never heard of him. A lot of people on these forums probably haven't. And since the scenario you described above was not detailed in the article....you're making an assumption too. Now piss off and mind your own business.
I have periods that I search the net furiously for The One Game, that I know don't exist. But I suppose Searching The One Game is another mmo for me, one that is played with Firefox :-)
In this search I join a million +1 closed betas, and I provide as details as the dev ask.
From these million closed betas, I receive (months later) some invitations, so I have some luck, and play some of these games. Some are just marketing plots, so don't really need, other are real efforts to make the game better, and I am then abduced on the cycle to report bugs and try to really help the game, make some suggestions (flawed, since I am a gamer, not a game dev).
But after some of these events, that after released where not all that good games, I am somewhat sad. You can try to help these games, but by the time a game is in closed beta, most games are mostly "done". Even in alfa state most games are designed by people that don't really need your help, other than beind a drone that play the game.
I feel like betas are a "lacking" way to help game dev's. So of course, I got, and buy his games on release. Thats probably a better help than betaing his games.
In real life I am programer. So I secretly like flawed betas. I was really happy to play Vanguard, and Tabula Rasa betas. These games where much fun on the naive first runs, as a beta tester. I think most games in the process to be Ok for everyone, get mediocre, and is sad. Most games in closed beta have parts where are Great, but these parts are removed because are overpowered, unbalance the game, corrupt the database, or some other problem.
Thanks to all the dev's that have let me the privilege to beta test his games :-)