Mark Jacobs Explains the Beta Delay
Today, City State Entertainment is announcing the delay of the beta for Camelot Unchained. In light of this, we talk to CSE's Mark Jacobs about the game, it's status, and what to expect going forward.
MMORPG: Thanks for taking the time to chat with us. Let’s get to the most important stuff straight away. How are you and the team feeling after several weeks of “Beta Crunch” (ducks MJ’s thrown shoe)?
Mark: [ The shoe crits MMORPG for an additional 25 points of crushing damage! ]
I think that sums it up nicely, don’t you? :)
In all seriousness, we are recovering from the hard crunch, allergies, a surgery, and assorted other stuff. We made a lot of progress during crunch, so we are very happy with that. On the other hand, we didn’t get everything done during crunch that we wanted to, so we have to figure out why that happened, and make sure it doesn’t happen again.
Andrew: There’s a lot of “deferred maintenance” on people’s lives that’s catching up with us hard right now. When you crunch, you end up putting off things like medical appointments, car repairs, or fixing that leaking pipe under the sink. You’re not getting extra time; you’re borrowing time from your future self. That debt always gets collected eventually, and sometimes it breaks your kneecaps.
MMORPG: The prior delays have largely been due to a shortage of senior programmers; this one sounds like it was due to the ability system. Can this be taken as good news that you have enough programmers going forward, or at least, that everything else was on pace until the ability system stun locked it?
Mark: While the ability system is a major reason we are delaying the opening of Beta 1, it isn’t the only reason. As far as having enough programmers, nope, we certainly don’t. Fortunately, we have two recruiting companies, one internal recruiter, and a friend, all looking for candidates for us. We have an offer letter out to one programmer, and I have three follow-up interviews with good candidates this week. I want to add a total of four programmers to the team before the end of the summer. If we can do that, we will have enough programmers to accomplish everything we want to accomplish in the following year.
Andrew: We came to a certain checkpoint, and looking at the path to where we wanted to be, it became really apparent that there were two general categories of work. For most things, there was a pretty clear sense of “OK, this is going mostly as planned, and there’s some amount of bugfixes or known work, but it’s clear where we’ll end up and how we’ll get there.”
But there was this second set of things where the progress hadn’t gone as hoped, and the path to delivering what we wanted still seemed rocky. What those things all had in common was the server-side ability code. So we had to look at that and make a choice: Do we push forward with what we have and end up with something sort of duct-taped together to make a date? Or do we take a step back and look for a different path to reach the same goal in a better way?
MMORPG: With so many unknown unknowns that only become known after you create and test them to find out if they work or not (if you think that sounds confusing imagine what it’s like CREATING them), what sort of process do you use when setting dates?
Mark: We try our best to put out a date that we think we can attain. However, given the frequency of delays that seem to plague most creative endeavors (Tesla Motors, Oculus Rift, and just about all MMORPGs), I don’t think anybody has figured out how to adequately predict the timing of a system which is by definition unpredictable.
Andrew: You make a list of end-goals, you try to think of every system needed to reach those end goals, you try to think of every subsystem of those, and eventually once you think you’ve divided things down far enough you say, “OK, how long will it take someone to write each of these things?”
But those are always estimates. The thing with software is, if you plan things things down far enough and detailed enough that you absolutely 100% know every step and how long it will take…you have almost literally already written the software. That’s all software is: An extremely detailed list of steps to follow.
So you have to keep things at a higher level and accept that you’re always estimating. Because the alternative is to re-label all your programmers as project managers. You’ll have perfect estimates, but you’ll have spent as long coming up with your estimates as it would to just write the software.
MMORPG: Considering how difficult predicting the unknown is, might we suggest a blind fold and a dart board?
Mark: I think we did a little better than that this time, but for some things, it might be more accurate. Probability is just a PITA, and with a team this small, even an otherwise minor loss of productivity really hurts. Having been through three MMORPG development cycles before this one, I’ve seen how bad it can be, even for big teams. Many MMORPGs that were developed at large publishers had the luxury of pulling in talent from other internal studios to get their work done, and even after doing that, they were still very late. I wish I could say that we were the only studio that “didn’t get it right” because then I would have a path to follow to fix the problems. Unfortunately, this problem is endemic to the industry, and is especially true for MMORPGs.
Andrew: We actually did pretty well on most of our systems. But you can’t pitch a tent with 19 out of 20 poles if the 20th is meant to be one of the tallest poles supporting the whole tent. So now it’s about getting that thing right.
MMORPG: Enough with the light hearted stuff, let’s get serious. Is curiosity killing the cat? Is CU too innovative for its own good or is this sort of issue common in game development?
Mark: We won’t know if we are trying to be too innovative until a year after the game launches. Yes, we are trying to be innovative in a genre than hasn’t seen a lot of innovation until recently. Too innovative? That will be up to the players to decide. It is certainly possible that players could be saying that they want more innovative games and gameplay, but when presented with these things, find that they prefer something less innovative. As I always say, time will tell.
Andrew: On the tech side, I think we’ve already solved the big risks. Stuff like server-side physics, putting 1000 players into a big melee, firing 1000 catapults to damage 1000 buildings at once, those kinds are things are the “hard” problems, in terms of programming. Those are the places that have taken a lot of technical innovation, and if you’ve got alpha access you’ve seen us making good progress on them. If things were the other way around, with abilities being fun but no more than 50 people supported on a server at once, that would be a lot scarier to me as a programmer than where we are right now.
MMORPG: Do backers deserve to be upset with crowdfunded game delays or are these sorts of delays simply part of the industry, ergo should the fans have come in expecting delays to be part of the package when they made their purchase and not be upset?
Mark: Fans/Backers have every right to be upset and disappointed; I know I would be. The question is, how should we all react when that happens? Calling for the digital equivalent of tar and feathers isn’t helpful. I would hope that members of our Community, who have seen how hard we’ve been working and the progress we’ve made in the last few months, will keep that in mind.
We are trying to do something very rare in the MMORPG space: create an innovative MMORPG while at the same time building out almost all of the tech ourselves. If we were using off the shelf tech and we were still always late, that would be one thing. However, we are trying to bring something special to the genre, and well, that’s hard. And, as I have said before, I did underestimate how difficult it was going to be to hire top programmers, thanks to mobile/VR studios staffing up. OTOH, VR wasn’t even a big thing in 2013, so…
Andrew: It’s almost circular. A lot of the reason people will get upset is that we’ll show them something that upsets them, because we’re being transparent and because we feel like we owe that to people who backed us.
The contrast to the way we run our project would be something like Fallout. Fallout 4 didn’t officially exist until June of last year, and then it shipped five months later in November. But the actual development took years, not months. And I know, from having worked at Bethesda and having seen how they experiment and iterate, that during those years there would have been things that went back to the drawing board, or that got cut entirely, because they weren’t working out. That part took place in secrecy. People only know the parts that were good enough to make it to the finished product. The development looks “cleaner” on the outside.
So it’s kind of a self-reinforcing problem. Crowdfunding means you set expectations much earlier and then show your progress. But I genuinely love our community; it’s been a lot more fun than spending this time in a silo, and I wouldn’t do this any other way.
MMORPG: The development stage known as “Beta” and what that entails has no exact definition. Why not just call what you currently have “Beta” and send it out to the unknowing masses?
Mark: I would be less than honest if I didn’t say that was something we had to think about doing. We could easily just call Beta 1 next week and use the “old school” tag to excuse it. However, while I’ve never said I’m perfect, I’ve always said that I won’t lie, ever, to our Backers. Calling Beta 1 with the ability system in its current state would be a lie. I’d rather have people say that we suck, rather than that we lie. At least in the former, I know we can fix that problem. Once you lie, it’s there forever.
Andrew: If it had been a less-central system like building plot ownership, I would have grabbed the duct tape and pushed forward. Getting products shipped isn’t about writing textbook-perfect code all the time. But the whole team felt that with something as important as the abilities that define each of the unique classes…well, that’s where we have to come out strong with a beta where we can plant a flag and say “This is Camelot Unchained”.
Sticking the beta tag on it means a lot of people will try the game for the first time. This part needs to be good.