Todays MMO industry trend of handing out beta keys like a pedophile hands out candy I think is a major down fall
to the quality of game that these companies are creating. In this blog I will be posting a few opinions based on
facts from my personal experience, and the experiences of close friends I have talked to.
Recently it seems that video game companies use beta testing in order to promote thier games rather then to
actually test them. It used to be that a person had to fill out an application and actually be able to input thier
experience in games, testing, RPGs, computers, etc... in order to have a chance of getting selected.
Now days you really just need to provide an email address, and sign up for thier forums, in some cases, all you
need to do is purchase the right magazine, or the pre-order for the game.
It has reduced beta testing down to a pre-view of an upcomming title, a demo if you will. This is advertising, not testing.
I can only imagine how a developer feels when he has to sort through hundreds of stupid bug reports to get to that one legit
report. I can only imagine how much revenue a company loses when thier bug riddled beta gets ripped apart by those on the
various forums that are not concerned with an NDA. The bad publicity alone would cost a company a lot of money.
Here are some beta testing ideas I have seen recently implemented by various companies and would have worked
miracles in the bug hunting tests if it would have been attempted by a more skilled bug hunting crew.
Dungeons and Dragons Online: While the game wasn't what I hoped, the game was pretty complete on release compared to beta.
The person in charge of the beta testing were amazing, occasional events to kill the tedious boredom, rewards if you were
caught in the act of actually testing what you were asked to test. Developer interaction with the testers via an mIrc channel.
All of these things, and some more really made us feel as though we were a part of the team, and actually making some
changes to help the game. The in game voice chat was actually suggested by a part of the testing crew I hooked up with a
lot during that test, because we used ventrillo during the testing to better coordinate are groups. It was one of the best
betas I have ever taken part in.
Vanguard: Someone over in the Vanguard team actually went to the forums and asked for people in specific level ranges,
of specific classes to sign up for a focus group, such as This thursday we will need 6 Rangers levels 10 to 19 to join me
at 4pm to discuss pros and cons, bugs, and such of this class. Fridays I will need 6 Rogues between levels 30 and 39 to
meet up at 4pm for the same.
It was much more detailed then that, but the first 6 people to sign up for the focus group would meet with a member
of the Vanguard staff and discuss it all. This would have been a huge boon to the game...fi only they didn't invite
everyone and thier mother, and people who bought a magazine into the beta, due to the poor selection of beta testers,
You got a lot of stupid conversations going on during a focus test.
I am going to end this blog today with a few suggestions that I hope will get to the current game companies:
1: most importantly....screen your beta applications, if I can take the time to fill out an application to
not only test your game, but do it for free. The least you can do is make sure we have some qualifications,
and a computer that will work.
2: Use some of the ideas above, and create more ways of making beta work better for you. The information
we provide is very important make sure you are getting what you need, and don't be afraid to ask us to test
something specific if a person complains about playing a class or race he doesn't like, boot them from beta,
They are not there to help.
3: Keep records of people who have beta tested for you and provided good bug reports. I was contacted by a
developer working for a diffrent company a couple years ago. He asked me if I was interested in testing a new game
he was working on, I was like sure, but how the heck did you get my e-mail? He told me he used to work for another
company and he ended up with several of my bug reports and they were exactly what he needed to track down soemthing
in the code. He said I found a couple of bugs that nobody else in the test had noticed, yet they all had to have
crossed by it. I have also tested two other games for him since then.
4: Don't use beta to test your server load, or to invite the fanbase to appease them, don't use beta as
an advertisment. This tends to lead to more bad press then good press, and but too much of a load on your
development teams trying to sort through all the garbage to get to the key problems. Server load test,
and advertisment is the final phase and should be done when beta is done and you feel you have a complete game.
5: Don't just release the game. Check with all of your development teams before you go gold, I can't tell you
how many times I here a developer tell me, yeah, if we had two more weeks before they went gold we could have tied
up that problem. I know, it's always a couple more weeks, but a couple more weeks to fix a small graphic glitch
may not be worth it, but a couple more weeks to fix that bug that crashes on 15% of the computers out there
will save you a lot of time and money.
6: Reward you testers. Yes, a lot of people claim it is unfair. But if you reward your beta testers with something
like you tend to put into pre-order packages it would be cool for those that tested, and not be something to hurt the game.
If your pre-order package offers a special orange armor dye if you buy it from best buy, and a special green armor dye
if you order from ebgames, well then add a special blue armor dye for the beta testers. Something that will not give an
advantage, but will set them apart from everyone else in game, something that tells other players this person was a tester.
A club of bug squashing, an in game t-shirt that actually says <I tested this game and all I got was this lousy t-shirt>
It may seem stupid, but it gives a tester a sense of pride and will insure that you will be able to use him again if called upon.
If any game company, developer, or tester has anything to add please post. If a developer or game company is seriously looking
for some beta testers that can squash some bug arse drop me a pm, or e-mail and I will give you a list of names and e-mail address to try.
I am not sure that all of them still test, but I am sure those that do, do it well.
Thank you for taking the time to read this, I hope it actually one day makes a difference,
Brian "Twisted"
Original leader of the "Vampyric Knights" a beta testing guild.
Previous leader of the "Twisted Knights"
User Comments
don't want to sound offensive, but I didn't read much after the 1st phrase where you call beta keys: "our beta keys".
They're certainly not mine, otherwise I would of gotten into some betas I've subscribed to :/
Most games I know do betas gradually. Small crowd, keeps getting bigger and will end up in a server stress test. Betas at a later stage are a perfect advertisement tool. I won't buy a MMORPG without having tested it first.
Made if you actually read the sentence it would occur to you it was a simple typo, it is handing out, not our...
i couldnt honestly read that....wall of text
Sorry next time I will add a pictorial for you... but this time it doesn't matter, it wasn't for you.
Im with you, especially with the last 2 point. Some games like Lotro gave beta keys to everyone and their grandmother instead of having a smaller useful beta. Its better giving away free 14 days account after launch.
you know there is a difference between open and closed betas right?
I'm sorry but I can't really agree with most of what you said Though, it's also my opinion that the decline in game quality is more related to the gaming market becoming more trendy and 'corporate' in the last few years. Gaming markets have exploded now that gaming doesn't require 2years PC know how and corprate big-wigs are dipping in to grab what they can get a quick buck from regardless of quality of the game. An MMO developer may have every intention of releasing a flawless game next month, but Mr. Funding the Publisher says you've got a week.
This has all happened before, in the 1980s when home gaming first started catching on. For reference: http://en.wikipedia.org/wiki/Video_game_crash_of_1983
Shoddy game quality will likely increase until gamers get fed up and quit buying their crap, and history will repeat.
1. I feel that developers mass invite people to test their games BECAUSE of all the redundant reports they get, and unique reports, and useless reports, because regardless of when beta ends, those reports can still be reviewed and implemented through the life of the game.
2. No comment, this one just makes no sense to me.
3. I know SOE and Mythic do/did this, and it seemed to work for them. But if you were to draw strictly from this team, why not just hire a professional QA team and cut out the reliance on outside sources totally?
4. This is usually done at the end of testing for the exact reasons you're saying not to do it. To test server load capacities, aquire mass backlogs of bug reports for future reference, and even aquire press.
5. Again, if the Publisher cuts funding to your development and Payroll, and tells you to ship, you can either A) ask developers to work for free with nothing, or B) ship and patch. Crap system but thats the way now.
6. Reward volunteer work? If you're honestly there to help make the game better for the sake of making it better, no need for reward, the warm fuzzies inside should be enough of a pat on the back.
Sorry my comment is about a slong as the blog but another viewpoint I guess. It sounds almost like you want to make some sort of Beta Testers Guild/Union, which could be cool and I'd suggest giving it a shot if you're adamant enough. :)
Good cause i want bears and tigers in my pictures please make sure you get on this right away....oh i tried really reading that mess again was hoping you had kinda fixed it....but jimmy cricket dreams dont come true all the time......
Popori excellent counter points. I would like to try and push my points on a couple of them. I only meant to blame poor beta testing standards as a very small part of the reason we continue to get such poor quality games these days, after reading my blog again, I can see how it sounds like I am blaming everything on it.
1: I understand the reason behind mass invites, I just disagree with them on a whole. On a game the size of say Vanguard, you could have had just as much, if not better success with a more controled testing, with half the people. You would have eliminated through screening all of the people who are just completely computer illiterate, inexperienced, and those who have inadequate computers. The results? You would have eliminated 90% of those post on the forums the day after invites go out claiming that the game sucks because it won't even load/run/play/looks like crap/etc...
2: on this my point was, and obviously I should have mentioned it again rather then assumeing you guys would memorize my "wall of text" on a couple of previous beta test I experienced some nice features from a couple of companyies earlier in the blog and I think they should become company standards.
3: pretty much cost is the only reason you would reward good work with giving them future work. Hiring a QA team is very costly, I know I used to work on a QA team for a local game publisher.
4: My attempt here was to suggest an open beta for server load testing etc.. rather then beta, beta should be strictly bug squashing and game tweaking.
5: understood, and without going into great detail I can not really explain this further, and from the previous comments on this blog you can see most people quit reading after the first sentence.
6: there should, and never is a need for rewards, if there was it wouldn't be a reward, it would be a payment. They give rewards to a lot of people for things they do in everyday life, while the fuzzy warm feeling should be enough, why shouldn't you work be recgonized?
Thank you Popori for actually reading my blog and making constructive post, rather then the previous comments. I used to run a beta testing guild, but it is hard to get an entire guild into a beta then it is to get myself in. I will say I still have contact info for a dozen or so people that still do testing and have been as long as I have. I would be more then happy to pull them back together for some testing if a game company needed them.
xxxariochxxx some how I knew by your reading level this is what you were into bears and tigers [IMG]http://i64.photobucket.com/albums/h169/checha_2006/POOH%20BEAR/tigandpoodt.jpg[/IMG]
I am sorry though, even at my age, which is pretty high up there, my spelling, grammer, and punctuation is very poor....I did physical work and played games all my life....sorry
Login or Register to post a comment