loading
loading

Dark or Light
logo
Logo

A Year In, How MMO Developers Maintained Their Game Worlds During The Global Pandemic

And how it's changed their studios

Joseph Bradford Updated: Posted:
Category:
Interviews 0

2020 was hot garbage. I think, collectively, we can all agree on that, right? With a global pandemic that saw multiple countries close borders, millions get sick from the COVID-19 virus, and cities in lockdown, 2020 was a year many of us would just a soon forget.

As a result of many of us being stuck at home, many of us also had more time to play games. Games became a more literal way to escape the dour feeling of 2020. They provided social links to friends we would normally go out to eat with, as well as a way to host virtual events like birthday parties and work meetings.

As such, 2020 saw huge growth in the gaming industry, with the NPD Group (via VentureBeat) reporting that game sales were up 34% in March 2020 versus 2019, and hardware sales were up an even more impressive 63%. The NPD Group also notes that within 6 months of 2020, industry sales were around $6.6 billion USD in the US alone, something not seen since 2010.

However, while game sales soared and the pandemic deepened, the pressure to keep these games online only increased as the year went on. While the working conditions across a multitude of industries meant instead of sitting in a cubicle many of us were sitting in our kitchens or home offices, juggling homeschooling kids and our day jobs, game companies were also challenged with ensuring that the games people were escaping to were still there as they too transitioned to working from home.

“Luckily, we already started looking at some policies around flexibility working arrangements prior to lockdown,” Bergur Finnbogason, creative director on CCP Games’ EVE Online told us via an email interview for this story. “[S]o that really helped all our studios carry out a very quick shift to working from home. We doubled up on our tech set up by sending computer hardware and other necessary equipment to everyone’s homes so we could replicate our studio workstations.”

This is a sentiment echoed by the four MMO studios that were able to respond in time to interview requests for this story. Being able to quickly adapt to the changing landscape was huge, and it meant a mix of ensuring employees could connect easily to the office, or just ensuring that employees were taken care of emotionally and mentally during the long periods of isolation. The pandemic hit studios around the world differently, with teams in the United States facing challenges unique to those in Korea, and so on.

Maintaining an MMO is hard work even without all the external factors the pandemic brought on, and ensuring that service remained uninterrupted when fans were desperate for an escape from the new normal was something MMO devs had to overcome. Studios such as ZeniMax Online Studios, developers of The Elder Scrolls Online didn’t seem to skip a beat, releasing their planned DLC dungeon and story updates, as well as their large, yearly chapter update in The Elder Scrolls Online: Greymoor. While Greymoor itself was a week delayed, the team at ZOS was able to launch the DLC on all platforms, including adding Google’s Stadia to the roster of places to play the MMO.

“It was definitely a lot of learning,” ZeniMax’s Rich Lambert, the creative director for ESO told me over Discord last week. “We’ve been live for almost six years when the pandemic hit. And so we already had a lot of pipelines and processes in place, because we’re always on 24/7. So you can get a call at two o’clock in the morning saying something’s broken, login, do some work, right? So we had a lot of those kind of processes in place already. But what we didn’t have in place was just the scale, to be able to have everybody in the office all log into VPN and work through things.  And so we had to work out how to get all this to work. And it was a huge exercise from the beginning.”

Pearl Abyss, the MMO publishing giant based in South Korea, had a different experience than many of the US and European studios tasked with keeping the lights on in the game worlds millions flock to daily. Thanks to how effectively and efficiently the Korean government handled the pandemic, Pearl Abyss’ employees were able to keep working from the office, albeit with new procedures in place to ensure the safety of their employees.

“Like all companies, we had to make adjustments for our employees’ safety and well-being,” Pearl Abyss America CEO Jeonghee Jin told MMORPG over email. “Pearl Abyss undertook an internal campaign to maintain the flow of development and the quality of everyday life in our workplace. It’s a bit different in Korea than it is here in the US - our employees still came into the office, so we had to put safeguards in place. For example, we put a guard in the cafeteria and a whole-body sanitizer and thermal sensor camera in the lobby. As a result of these actions, there have been no confirmed cases of COVID-19 from Pearl Abyss.”

Getting employees settled and working on their projects was a massive hurdle to overcome for every industry, but it was only part of the story for many of the developers we spoke to. Keeping up with development schedules, ensuring that games and updates released with the same level of polish as would be expected in optimal circumstances and more was on the shoulders of every gaming developer out there, moreso on MMO devs.

“I think that entire year of updates were some of the toughest ones we’ve ever done,” ESO’s Lambert told us. Star Trek Online’s executive Andre Emerson echoed this sentiment, stating that, especially due to the nature of moving from in-office workstations to at home rigs, there were obvious technical hurdles to overcome.

“There were technical challenges early on attempting to achieve workstation parity between office and home and it’s still not always 1-to-1 due to varying ISP performance, etc.”

Part of the challenge facing developers who maintain an always online, persistent world game was the speed with which many had to adapt. For many states there was the feeling there would be a general lock down before it officially happened, but being able to react as quickly as possible to ensure games didn’t go down was something each major MMO developer had to adjust to as fast as they could.

“Because this happened so quickly, we were initially streaming our office desktops to the home office,” STO’s Emerson told us via email. Emerson says that the results initially were spotty, with the team now working with more performant solutions.

The STO team also talked a bit about their updates through the year, with the team thinking that they would need to either push dates, reduce the content in those updates, or even do both to cope with working from home initially. Thankfully, the team found their footing quickly, resulting in only one delayed update.

“Our updates are critical to keeping players engaged and running a healthy business. At the start of the pandemic, we anticipated needing to either push dates, reduce content or potentially both. Our team however rose to the challenge and we were clicking on all (most) cylinders relatively quickly. As a result, we briefly delayed one update, but didn’t cut any of the content we had planned for.”

The Elder Scrolls Online: Greymoor faced challenges after its initial launch. Delayed about a week from its initial launch date, the ESO team faced issues with connections for players, downtime and a host of bugs, resulting in less than polished feel compared to its previous Chapter releases.

“[I]n terms of the number of issues we had to hotfix, the number of times the servers went up and down. Some of those were hardware failures, but jus the overall quality wasn’t good, or wasn’t as good as what we have done in the past.”

As such, Lambert says the team has adapted to the new normal, as well as changed their approach of not trying to do too much, instead giving the team more time to polish and review before shipping an update.

“So for this year, and these releases this year, we focused a lot more on trying to finish earlier so that we can have more time to review and polish,” Lambert says. He continued, “Another thing that we’re doing is we’re just trying to do less. We announced that at the end of last year, we’re just going to do less this year so we have more time to polish things and do them better. And I think we did a great job of showing that with Update 29 in the Flames of Ambition DLC.”

By and large too, this has worked. The recent release of Flames of Ambition saw two, really well made dungeons release leading up to this year's Blackwood. The mechanics for the bosses were fun, the environments striking - it was a great start to the year for the ESO team with this new philosophy leading the charge.

Before the pandemic started, CCP had just announced a more agile update release cadence, called Quadrants, at EVE Vegas 2019. Bergur tells MMORPG that, in hindsight, moving the release cadence to the new Quadrant cadence actually helped the team set up for success.

“Working from home definitely did change everything when it comes to how we work in terms of delivering new content and features to our players. Looking at the year in hindsight, I think moving our release cadence to Quadrants certainly helped set us up for success. This has allowed us to move faster, try new and varied things, scale features quickly and quite a lot more. Of course, we are still learning and fine tuning how we do Quadrants and that’s something we will continue to do, going forward – we are always learning!”

Pearl Abyss’ team faced some challenges, but thanks to the nature of their studio with multiple teams across the globe, the Black Desert Online creator had some experience with working in remote conditions.

“We also switched to virtual meetings, restructuring our strategies and schedules, all while staying dedicated to development and our community,” PA’s Jeonghee Jin says. “We are fortunate to have offices in multiple countries so there was some familiarity with virtual calls. But facing a global pandemic is difficult on people’s mental well-being and their families - we had to make sure to take care of our team members as well.”

The well-being of not just the game, but more importantly those making it was something at the fore of every team’s mind, as Jin mentions. The pandemic has been hard on a lot of people throughout the world across industries. Dealing with the mental strains of your home becoming your work place, as well as the isolation of a global lockdown can have a terrible effect on people.

Teams like Cryptic were cognizant of this, with Emerson telling us how they approached scheduling meetings and even regular work deadlines, ensuring that the developer could be flexible to make it work for families now forced into a new normal.

“Until we got a clear picture of what we could accomplish remotely, we kept a keen eye on release scope and timing. We also only held our virtual meetings during normal business hours. We also recognized that people had young children at home, kids in virtual classrooms, etc., so we remain very flexible as to when the team does their work. Some folks like to work straight business hours while others enjoy the flexibility of spacing their work out to accommodate other aspects of life.”

Pearl Abyss Black Desert Online NovaThe Nova, one of the classes released during the pandemic by Pearl Abyss in Black Desert Online

Pearl Abyss’ Jeonghee Jin says the pandemic has allowed the publishing giant to revamp some of their policies, especially as it pertains to benefits for families with children or other dependents.

“Having some parts of the teams working from home for over a year now made us better communicators internally and gave all of us greater compassion for one another. Those lessons will never leave us. We’ve proven our team can stick together through some tough times and still deliver quality gaming.”

Pearl Abyss had a year, though. Though the pandemic did affect things, the studio still put out multiple updates, including releasing a few different classes for Black Desert Online, cross-play for its console ports, as well as development on Black Desert Mobile continuing at a good clip.

“Of course, as with any team during a crisis, there were definitely adjustments, not just to how we work, but what we’re working on,” Pearl Abyss’s Jin tells us. “During the last year, we were also able to take over publishing of Black Desert Online in North America and Europe as well as release a brand-new trailer for Crimson Desert at The Game Awards. I think we provided a pleasant surprise to our players and gaming community with the appropriate response to the pandemic.”

One common theme was how the pandemic shaped the creative process, as well as pointing out some of the meetings and moments during the regular workday that are just taken for granted. “Watercooler talk” as it’s called around many offices can often provide many creative outlets for developers to talk about ideas, solutions and more in an environment and setting that just feels more relaxed. Being able to walk over to an artist’s desk and bounce ideas off them or sit down with a colleague at lunch and talk about a potential solution to a problem is taken for granted when it can no longer happen.

“I think the biggest thing is don’t take the watercooler chats, those one off, ad hoc-type things for granted,” Lambert told us when asked about one of the major lessons ZeniMax’s team has learned now that we’re a year in on the pandemic. “They are supremely important and you don’t know just how much you use them and how much you need them until you can’t really do them anymore. So that for me was the biggest eye opener. I’ve been making games for a long time now and I totally didn’t realize just how important that was and how much we relied on that kind of stuff.”

As such, ZeniMax hosts water cooler meetings, scheduled for 30 minutes every day where developers can jump in and chat about work, or really whatever else they want to in the end. Rich also mentions that the team’s daily stand-up meetings have changed, stating that there much more of a focus on the problems rather than just the normal status check ups that would have been the focus before. As a result, they have really reinforced how important those meetings are to the team according to Lambert.

Blackreach Greymoor Elder Scrolls Online

CCP’s Bergur Finnbogason echoed these thoughts, talking about how these impromptu chats are hard to replicate via a Skype or Zoom call.

“Communication and isolation were probably the biggest challenges that we faced,” Bergur notes. “It’s very easy to fall out of touch with your team or company when you are sitting at home doing your thing in a silo. Impromptu chats are really difficult on Zoom or Teams, and you don’t really realize the importance of, for example, a VFX artist having a discussion over lunch with a systems designer until it’s impossible to have those watercooler moments anymore.”

CCP’s Brand Manager, Sæmundur Hermannsson, added to this, by talking about some of the ways the Icelandic developer is helping to try to spur these types of discussions and moments again in the remote workplace.

“To add to Bergur’s point, plushing certain things – that is to say, finding little extra things or ways to help improve your or your colleagues work – takes extra effort in a digital environment. We’ve gone some way to get around this, for example organizing cross-team show-and-tells, early morning casual hangout meetings and after-work social video calls.”

Bergur mentions too that he, like many developers, needed to find that escape, which led him to start exploring Wormholes in EVE Online as his way of “getting out and socializing.”

“My social needs are not fulfilled by Zoom calls, and the truth is that being stuck in the house all day looking after my kids and their education needs while also needing to have those Zoom calls is hardly an optimal work/life environment. So, wormholes became my way of getting out and ‘socializing’.”

As we’re now a year into this new normal, with the pandemic having been a thing for the last twelve months we’ve all had to cope with, some changes have come about with every studio we spoke to for this story. While the pandemic was not a good thing, and no one would suggest otherwise, it has forced companies to think differently about how they approach working in the office.

Studios like Cryptic have now instituted policies allowing employees to choose whether they want to work from home or in the studio once it becomes safe again, with Magic: Legends Steven Ricossa telling me over a call in late March that the hiring pool was widened thanks to this move as well, meaning the studio can bring in talent from across the country and not just Southern California. STO’s Emerson also echoed this, mentioning how some employees of Cryptic have been able to take advantage of this flexibility and move out of California entirely.

CCP’s creative director has also talked about this, as well as some of the changes to the studio’s operations, such as an internal “talk show” where the developers talk about the MMO, its development and more. And while CCP can have almost every developer in the office physically now thanks to the response to the pandemic by the Icelandic government – as well as being “on an island – with a single point of international entry – in the middle of the Atlantic Ocean” according to Sæmi, Bergur says that CCP has a hybrid-culture, with many choosing to work from home versus full-time at the office itself.

“I have an internal talkshow every Friday where we talk about upcoming things, different playstyle, vision, tech, etc. and shine a spotlight on the work being conducted by various different teams across CCP. It’s a really great way to showcase what we’re all up to for one another. We also have live, weekly AMAs with our senior leadership team, Hilmar in particular, where anything goes and no topic is of limits. We also have the company-wide show-and-tells that Sæmi mentioned earlier. All in all, the global pandemic created the condition where we needed to build improvements into our frameworks and procedures rapidly, which will pay dividends long into the future. It has also highlighted any missing links in how we work and highlighted key positions that will allow us to work faster and more efficiently moving forward.”

Each studio and game is different, and how they adapt to the pandemic a year in, especially as vaccinations start to become more widely available is up to them. While some will still prefer that the majority of the work happens in the building, others, like Cryptic and CCP will adopt the hybrid model to give developers more flexibility in the long run.

All of the developers we spoke to for this story felt optimistic that the changes will end up being for the better. More diverse hiring pools thanks to being more open to remote work, more flexibility for those working hard to bring players the experiences that we all want at the end of the day – being able to adapt to the changes being forced upon many studios thanks to the new normal will have long-standing effects on the industry as a whole. As far as moving forward from here, each team is hard at work on their next major updates to keep delivering players what they want: high quality content in their favorite game worlds.


lotrlore

Joseph Bradford

Joseph has been writing or podcasting about games in some form since about 2012. Having written for multiple major outlets such as IGN, Playboy, and more, Joseph started writing for MMORPG in 2015. When he's not writing or talking about games, you can typically find him hanging out with his 10-year old or playing Magic: The Gathering with his family. Also, don't get him started on why Balrogs *don't* have wings. You can find him on Twitter @LotrLore