Each week, the entire Star Trek Online team gets together for an internal play test. It’s an incredibly important part of the development cycle, and I think our fans should know more about how we use our experience to shape the game and make it better. We play together every Tuesday for about an hour. Obviously, certain teams play the game quite a bit more than that, but Tuesdays are a special day where everyone on the team – Art, SW, Deign, Audio, Concept Art, Production, QA, OCR – is required to play our game. We get to really focus on key elements we’re adding to the game, and to sit down and have real discussions about what we’re working on.
We assemble a build of the game late Monday, and on Tuesday afternoon we drop what we’re doing and get into a conference room to go over what we’re testing that week. We go over what the major changes and additions over the last week have been to the game, what content we’re going to play and what is the particular focus during the play session.
We spend about an hour playing, and then everybody gets back into the same room. We spend another 15 or 30 minutes gathering as much feedback from the team as possible. What did we like? What didn’t we like? Most importantly - did we have fun? It’s important to talk about the good stuff in addition to the bad, so we can stress the “likes” in future tests.
Running the play tests achieves a number of things:
- Improves game stability. It’s so easy when you have a team of 30-plus people adding features and content to a game simultaneously, the stability of a product can drop off a cliff. You come back a couple weeks later and nothing is playable. None of the bugs are really huge, but the more you have, the more they add up to a huge problem. The frequency with which you force people to take a look at a game really helps with its stability. It lets people say, “Oh yeah, I’m making a game. When I put this feature in, I need to make sure it works. QA’s not going to give me a bug, everyone on the team is going to see this crappy bug I put in without testing it.”
- Increases focus. It’s really important to see your progress on a constant basis. Anybody can sit in a room and design systems to their heart’s content, design the most complicated game possible. But really, until you get what you design in the game, until you play it and other people see it, you really don’t know what works and what doesn’t. For example, in space combat, there’s a huge list of features that we really want in space combat, and some of the features we imagined were the most important have turned out to be secondary and even tertiary systems. It’s getting stuff out there that keeps you from getting off the rails. It’s right in your face.
- Gets the team in sync. If the project is going well, if we’re making good progress, if we’ve got a lot of good content coming down the pipe, the team feels it and morale is up. If we have crappy play tests and they’re unstable and they break and the content isn’t quite that good, the team might get depressed about it. But they realize, “OK, this is the final product, so I’m going to step up and take a little ownership and I’m gonna make this thing better.”
This Week's Focus
These play tests are often the first time a lot of people are getting feedback on whatever they worked on that week. That’s really exciting; you can see people step up and ask questions of the people who had comments about their quest or system. It’s really personal; they’re getting direct feedback.
We focused on a lot of different issues this week. It was a number of little things; we didn’t focus on the content as much as a bunch of tiny changes we’ve made over the past week.
We made some pretty substantial changes to the weapons on the ground, and we added a few new ones. We changed some of the special items on the ground; these are the powers that you end up having that are your career powers, your specialization powers. So if you’re a Tactical officer, you have Aimed Shot and stealth powers. If you’re an Engineer, you have shield buffing powers and some other things like that.
And then there was a brand new ship configuration that some people tried this week.
The gameplay continues to get better and better, and the content that goes in goes in right the first time a lot more often than it doesn’t. As we add new ship configurations and new bridge officers for you to play with, the depth of space and ground combat just continues to astound everybody. We come up with new strategies daily.
Having these weekly play tests have allowed us to have a much more stable game, help us iterate on the features, and really generate a much more fun game on weekly basis. Everyone here at Cryptic Studios loves making games. We all really wanna be in the games industry, and on a weekly basis, these tests get people into the product they're making.