At Paragon Studios we sent design candidates a test to judge them on a variety of factors. The test was a smattering of topics that covered all aspects of game design, so we could get an idea of how a designer thought and their level of creativity. The test was timed, but not rigidly so.
A couple people asked in the comments about the time restriction. I’ve seen design tests at other companies that are rigidly timed, with websites that track the minute you download the test to the minute you submit it. We didn’t do that at Paragon. We simply asked the candidate when they felt they would have enough time to devote to the test, and scheduled email delivery of the test to that time period. They would then complete the test and return it. When the HR department delivered the test for “grading” (not a real thing), we were told how long the applicant had the test for.
)
We could use that time to judge the quality of the test. If the applicant had the test for three days and returned something that looks like it took three hours, we knew something was amiss. Usually anything longer than 24 hours and we would be looking for what aspect of the test took so long. Many tests came back with fully sketched “small towns”, complete with flow markings and encounters mapped out. We liked those, not only because it showed us more of what the designer was capable of, but what the designer felt was truly an example of what she could deliver to the company should she be hired.
Anyway, back to the questions.
There was a great question on the content designer’s test that tasked the designer with creating 20 missions/quests to take a character from level 1 to 20. There was a variety of enemies that could be used in the test, but all they were was a name and a level range. The designer was also required to “spice things up.” They had four different “objective types”, Click object, Collect Drop, Escort NPC, Kill NPC. They were required to have no two quests out of the 20 have the same objective or combination of objectives. If you had a mission to escort an NPC and then kill a boss at the end, you couldn’t have another mission using Escort NPC and Kill NPC as the only objectives. You could add Collect Drop and it would make it different enough, but then you cross that combination off the list of usable combos as well.
What I loved most about this question was not the brain exercise of puzzling together different combos of objectives, but how the designer interpreted the names and level ranges of the enemies available to them. We didn’t give them a specific game world to work in, all we gave them was names along the lines of Scorpions, Sand Giants, Nomads, Mummies, Bandits, and Sciroccos. Definitely a middle-eastern feel to the names, but we didn’t give them a time period or genre of the type of game.
While most designers took the Indiana Jones route, and set their game in the early 20th century, we had others use the names as groups of super-villains for a superhero game, and still others created pure Fantasy style interpretations. I really looked at how the names were being used, were they inspiring to the designer? In the end we really liked this question and had just as many applicants tell us this was their favorite question on the test as told us that it was their least favorite.
Over the years at Paragon we added a couple questions to the test; one of which was to fully flesh out the dialog for one of the quests in their 20 Mission question. Write the intro dialog, accept text, and return text that the player would encounter during the mission. I can think of one designer off the top of my head who did amazing on this single question. He immediately gave me the impression that he knew how to write for MMOs, giving characters memorable words and voices.
There was one question that I know many of my Powers designers hated, and the hatred came from the format in which I gave the question. It was an “MMO Math” question, which, if you have the luxury of time, should be pretty simple, but I would give the question during the phone interview. I would inform the candidate they could take as much time as they needed and use any tool at their disposal (calculator, Excel, etc) but many times the stress of the situation would get the better of them and they would blow the question.
)
Getting this question wrong was never a deal breaker (unless you weren’t even in the right ballpark). If the answer given back was wrong, I would walk through the steps of the question, asking them how they arrived at given steps, and usually the Powers designer would see their mistake before I could point it out to them. (In all honesty, when I was pitched that question by one of the programmers, I initially got it wrong, but my methodology for arriving at the answer was 90% correct, so it was a simple correction for me to get the correct answer.) I know by now you are all wondering what the question was. It was a three-part question:
- If a power does 50 Points of damage, followed by 5 “ticks” of 10 points of damage, what is the average damage the power deals?
- If a power does 50 Points of damage, followed by 5 “ticks” of 10 points of damage, but each tick only has a 50% chance of applying damage, what is the average damage the power deals?
- If a power does 50 Points of damage, followed by 5 “ticks” of 10 points of damage, but each tick only has a 50% chance of applying damage, and if it fails to apply damage no further ticks occur, what is the average damage the power deals?
You can see how a designer might initially think “I got this” and then get to the last question and find they need to now “show the work.” Then realize they are in a phone interview with at least one person on the other end waiting for their answer. It’s stressful, but no more stressful than getting the math right on a power when you are trying to fix something happening on the Live server and you have Leads, Producers, Ops, and Executives breathing down your neck for the patch to fix something.
Feel free to put your answers to the MMO Math question into the comments below. I also enjoy reading your interpretations of the questions. I am only disappointed now that if I start hiring designers again, I’m going to have to come up with an entirely new test!
Matt Miller / Matt Miller is a 22 year veteran of the computer game industry and columnist for MMORPG.com. He was Lead Designer for City of Heroes over five years, and has "seen it all" when it comes to MMOs (but still learns something new every day). You can always reach him on twitter @MMODesigner
Read more of Matt's columns:
