This is a great article. As technology advances will we see larger populations on single server that the player selects from?  Most servers today seem to hold only a maximum of 3k players. Will there be a day when we will see 10k-30k players on one server that the player will select from? Eve online has been able to handle larger numbers with their current cluster.

The MMO techs have learned a ton in the last 3 years.

That dev knows what he's talking about. This makes me excited because now i know that the Bioware project is in good hands. Even though we might not know much about his game, at least we now know his philosophy is on par with a great ability for execution. Good article.

---

Today’s beta testers have more selfish goals: we seek to gain advantage by learning the layout of the game and the UI and to try the game before we buy it.

Or make sure we aren't ploping down $49.95 for a selfishly over-hyped piece of crap game, with gameplay only good for newbies, so reviewers can give the game high scores after playing 1 hour.

With single player games or FPSs, we can get a demo, but MMO companies only let people try a game for free either A) during open beta or B) after everyone finds out the game is crap and subsciption numbers stop rising (14 day trials)

If companies want better beta testers, they should stop making open betas the only way someone would get to try a game before buying it.

I think every MMO should have an open beta for about a week just before launch.  The devs can use it as a stress test, as a test launch, and a preview for people planning to buy the game and those undecided.

You get the real testing done and players still get to try it before they buy, plus devs get to stress the hell out of their servers.

!!

"One poll question Vogel recommended was “Would you recommend this game to a friend?” If less than 75% of your users say no, you are not ready to launch"

I think he meant it the other way around...

Anyway, we get the idea, the concept of player recommendations is so fundamental and easy to grasp and yet so many companies seem to totally ignore it.

---

Originally posted by Gor'Gar
"One poll question Vogel recommended was “Would you recommend this game to a friend?” If less than 75% of your users say no, you are not ready to launch"

I think he meant it the other way around...

Anyway, we get the idea, the concept of player recommendations is so fundamental and easy to grasp and yet so many companies seem to totally ignore it.

---

Ya I caught that right away too lol...

What really erks me is when you are serious about beta testing some game and are totally honest with the applications and don't get selected yet at the same time you know someone who did get into beta and doesn't give a care in the world about bugs and only playing it...

Maybe it's just me but every once in a while this happens and it just drives me nuts cause I know I would love to find bugs and exploits while the other guy is just powerleveling and starting a beta only uber guild anticipating the release of the actuall game and learning all he can to out uber the other live playerrs when they come...

You get what you pay for.

There was a time when beta testing was considered important and testers were hired employees. For some reason that changed with the introduction of MMOGs.

He mentions the forums and uses smedley's words: "Vocal Minority"

I would think that by now that some of these developers would have learned that though it is a vocal minority they often times, not always, but quite often speak for the majority of the players.  With an attitude like this will we see another swg sized flop?  They thought only a Vocal Minority were against the NGE and as those year long prepaid subs run out swg gets smaller and smaller.

It's an interesting article but this little nugget jumped out at me:

---

Another important point was the actual purpose of beta testing – believe it or not, it’s not there for you to try out a new game! Beta testing is important to work on balance and to find exploits

---

Here's the thing. As he says in the article, yes, I have been in betas. Developers keep shooting themselves in the foot. Frequently, they do not listen to beta testers at all. By this I'm not talking about them not listening to people complaining that rogues are overbalanced, but rather, I'm talking about the fact that major, game-breaker kinds of bugs are found and reported in beta, and whether acknowledged or not (often NOT), the game goes live with the bug that the intrepid beta tester found, tested, logged, documented, and sent on to the authorities as he was supposed to.

This has caused a lot of players, myself included frankly, to become jaded about testing for and reporting bugs. If there was some credible reason to believe that the developers would actually fix or at least work on the bug you found for them, I'd be more enthusiastic about finding bugs. But a lot of the ones I've found have been ignored, and for instance, a couple of real zingers are still in existence years later that I and others found in beta. This makes you not want to bother testing.

The other thing MMO devs have been doing a lot, which is related to my first point, is that, because they aren't actually listening to the testers on bugs and exploits, and are letting their games go live without fixing them, a lot of recent launches have been a mess -- games full of bugs and holes, lacking content, and basically not worthy of charging people a dime. A lot of gamers have been hosed, now, by purchasing a product that is unacceptable, and feeling like they were shafted by the company into buying and paying for an extended beta. This shoddy workmanship has led a lot of players to want to get into beta to see if the game is going to be worth paying for or not. Beta lets you get a look at the main features, at what the content is like, and at how buggy (or not) the game is, without paying for it. Now the quote I have above says "that's not what beta's for" (free trials), but it's what a lot of gamers use it for, specifically because so much drek is being produced right now that players are tired of having their money sucked out of them for what amounts to a pile of unworking code.

So, if developers actually listened to their beta testers and fixed the problems we detect, we might have more respect for the process. And if they did that and as a result launched nice, solid games that people could expect would be worth their money, less of us would feel the need to "try it free" to see if it's any good, because we'd know ahead of time that (since they listen to beta testers and fix problems) the game is likely to be solid.

This starts and ends with the developers not being willing to listen to beta testers. Once that happens (and it happens a lot), the process is destroyed. It's not the fault of the players who want to try the game free. There are tons of those no matter what, sure... but by destroying the process, it means that even people who'd beta test properly if the devs would listen, are not going to want to bother.

C

Great comments Chessack!

This article and the one by Rob Pardo from WoW give me very little hope about the MMO's of the future.  These folks underlying views of their customers couldn't be more contemptuous.  Its like a throwback to company/customer relations from 30 or 40 years ago.  The next big breakthrough will be a game that actually is driven by the customer base, not the development house.  Unfortunately, it may be years before any company has the sense and guts to try that.

My recommendation to Bioware: Fire Vogel and hire Chessack.

Buck

LOL. They would not want me. I'd stop them doing all the same old crap that everyone else is doing and force them to do something truly original, which is risky, and carries with it no guarantee of at least a minimal enough subscriber base that they'd make back their investment.

Mind you, I think that a truly original game (well beta tested of course!) would blow the tired old stale model of games we currently have out of the water in terms of popularity... but I have no evidence for that other than my gut feeling and my interactions with other gamers, most of whom are sick and tired of all the old stuff.


C

Quote:

Today’s beta testers have more selfish goals: we seek to gain advantage by learning the layout of the game and the UI and to try the game before we buy it.

Don't blame the beta testers (alone at any rate).  The companies have blatently fostered this behavior.  If the experience were about test users would not always be so expected to 'play' the game.  They would be granted abilities/capabilities that they wish to test.  Now granted at some point that includes 'playing', but in my experience (having beta'd more then a dozen games -- easily more than that) is that very, very little of this occurs.  Instead players are told to just go out and play or go out and 'test'.  Like that means much.  You want less selfish beta testers, then start by treating them as testers first and players second. 

What's worse is the favortism that is bread into beta's i have participated in lately.  There is always a core group (usually a guild or two if implemented) that the GMs and Devs continuously give advantage to or turn to for opinions.  These folks are Rarely the run of the mill testers, thus the game is astonished to find that those folks getting all those perks did not to share the information / experience / opinions that do no directly benefit them.

You wan't it about testing?  You singlehandidly have the power to make that happen.  Do it.  You want opinions of the non-vocal majority?  Implement in-game polls.  Ask.  Randomily send tells and request feedback.  Search folks out in certain areas and play with them. 

Devs and certainly in game customer support appears far too lazy and far too into enjoying their position then they are in actually gathering feedback from the masses they hope to sell to.

One beta tester's 'feedback'.

---

Originally posted by Jorev

You get what you pay for.

There was a time when beta testing was considered important and testers were hired employees. For some reason that changed with the introduction of MMOGs.

---

Exactly what I was thinking. For some reason, a majority of companies seem to think the privledge of being a beta tester is incentive enough for you to slave away on testing their game. EHHHHN, sorry. If I'm not at least enjoying the testing process, I am not going to keep plunking time in to it.

Vogel's battle cry seems to be "I'm not NOT paying you to try the game out or gain an advantage, I'm NOT paying you to work!"

---

Originally posted by Lateris

This is a great article. As technology advances will we see larger populations on single server that the player selects from?  Most servers today seem to hold only a maximum of 3k players. Will there be a day when we will see 10k-30k players on one server that the player will select from? Eve online has been able to handle larger numbers with their current cluster.

The MMO techs have learned a ton in the last 3 years.

---

EVE-Online has over 30,000 on a single server.

This makes me have doubts about biowares mmo.

Bad enough they hired "talent" that was from other games that I and many others are NOT playing becuase of ALOT of decisions that "talent" was involved in making,but to hear him give such an incorrect view is scary.

The beta process USED to be handled much better.Game companies used to hand pick the beta testers, give them goals,arrange specific feature tests,kick out people who didnt contribute,etc...

NOW the damn companies use beta as a marketing ploy.They seek out and listen to those same lame "uber" guilds that helped screw up the past games.They totally ignore  info IF its negative (even when its honest, correct, and constructive) becuase they decide its to late to fix the problems.And they go out of their way to open the flood gates to allow massive amounts of people in who will serve no purpose other than stress testing.

The people havent changed so much,and no where near as much  as the companies.To try and blame the people is arrogant as hell.The good testers are still out there,though many have stopped even bothering due to the companies handling of betas.

But the companies dont seek the good testers out, and they dont listen too them when they get in either.

You want to point at someone for making beta near pointless as a testing tool, POINT SQUARELY AT THE COMPANIES, becuase as usual they cause the most harm to themselves ,the product,and the process of making the  product .

Even the alpha stage is becoming pointless, as the DDO alpha clearly showed..

---

Originally posted by Vertex1980

---

Originally posted by Lateris

This is a great article. As technology advances will we see larger populations on single server that the player selects from?  Most servers today seem to hold only a maximum of 3k players. Will there be a day when we will see 10k-30k players on one server that the player will select from? Eve online has been able to handle larger numbers with their current cluster.

The MMO techs have learned a ton in the last 3 years.

---

EVE-Online has over 30,000 on a single server.

---

Er... isn't that what he just said?

I'm sorry I did not see the part where he told DEVs to do their job.

I was in the UO Alpha and have been invited to many many tests since then. And I met Rich when he came to Ausitn and joined Origin Systems (ok EA Austin for noobs)

First can DEVS use a spreadsheet? When testers compain about balance, there usually is a bias for it.  It only takes a DPS calculation on a spreadsheet with a single timed test to know if it is happening.  But we routinely see unbalanced launches.

The MMO Project Managers say they want feedback, but when any BETA tester can point to half dozen KNOWN bugs on launch, do you actually expect people to respect the "beta testing" concept? PMs have themselves eroded the beta phase.

Marketing with their exclusive beta passes and coop advertising, for a game that is not yet finished.  Need I say more?


NOTE: To those who talk about paid in house testers.  Sorry it does not work anymore in anything but console games at best.  And it never worked in MMOs.  There is too many skill/item/environment/etc. combinations  and too much human ingenuity to be properly represented by a dozen jaded testers in the back corner of the office.

Personal note: I now Car Wars since it was created by friends of mine.  So when BETAing Auto Assault, I honestly tried to too see,test, and report on a game I hoped would change to a fun car based MMO.  The DEVs NEVER listened, had no clue about grouping, and I swear were brainwashed by the Raphael school of crafting.
They gave me a free acount on launch, they reactivated it after I never played post launch.  But they never listened to what we said.

Amateur Game Playtester since '82
Austin, TX

VultureTX,

I don't understand why the paid testers would be jaded? If you pay them a fair wage and respect their input as you should, they will be happy campers.

Obviously you hire people who are detail oriented and explorative. You certainly won't get positive results by relying on a free labor force whose motivations are mostly selfish. The only time you should open the game up to the mobs is when it is complete and you want to stress test.

What is your solution for proper testing?