“Yet even in his deathless state, the Emperor continues his eternal vigilance. Mighty battlefleets cross the daemon-infested miasma of the Warp, the only route between distant stars, their way lit by the Astronomican, the psychic manifestation of the Emperor's will. Vast armies give battle in his name on uncounted worlds. Greatest amongst his soldiers are the Adeptus Astartes, the Space Marines, bio-engineered super-warriors. Their comrades in arms are legion: the Imperial Guard and countless planetary defense forces, the ever vigilant Inquisition and the tech-priests of the Adeptus Mechanicus to name only a few. But for all their multitudes, they are barely enough to hold off the ever-present threat from aliens, heretics, mutants - and worse. “ ~Rogue Trader, Games Workshop, 1987
Sounds awesome, right? Vast armies clashing on a planetary scale, fighting for the very survival of humanity… or its overthrow. Sadly, we have never been quite able to replicate this on our computers, the closest we have come is the eight vs eight of Space Marine by THQ and Relic Entertainment. Not really reminiscent of vast armies, more like skirmishing scout forces, really. Until now; until Eternal Crusade, with the Pikko Server. This new-ish technology makes huge battles with thousands of participants possible; which, in turn, will allow the devs at Behaviour to create Eternal Crusade.
How does it work? I have no idea. Magic, maybe? Let’s take a look at Pikko server technology and see if we can figure it out.
Developed in 2011 by MuchDifferent, a software development company based in Uppsala, Sweden, Pikko technology was used in early 2012 to break the world record for concurrent user in a FPS battle. Pikko Server is written in Erlang, a programming language built specifically to enable parallel computing, allowing very efficient use of multicore processors. It turns any single-core game server into a cluster of cell servers, called masts.
)
Essentially the system consists of a Pikko Server, a bunch of masts and the players. The Pikko Server handles load balancing between the masts, and seamlessly switches the players from mast to mast as players move around the game world, keeping the player on the most efficient mast or masts, as players generally are tracked by the mast they are assigned to, and the other masts in the vicinity. Load in one area getting too high? The Pikko Server also allows the masts to move in relation to each other, balancing the load again. More masts can be added as the overall load rises, ensuring smooth game conditions as the battle rages on.
Pikko Server also contains redundancy features designed to maintain the seamless flow of the game even through mast failure or crashes. Pikko server technology can be compared to a cell phone network. The players are the cell phones and the game servers are the towers. At the heart of all of this is the Pikko server that routs the calls (player actions) to the appropriate tower (mast). However, if the action gets called into the same area more towers will be activated to share the load. Towers can even move closer to each other to ease the strain, something real-world cell towers cannot do. This, of course, is an over-simplification of the actual Pikko Server technology, but it serves well enough as an example.
What does all of this mean to us, the future scions of the 41st millennium? Well, it means that Behaviour will be able to put thousands of warriors onto the battlefields of Eternal Crusade without putting all that stress onto a single server or server cluster. As the battlefield population shifts, so will the Pikko Server dynamically allocate system resources to maintain a top-notch playing experience, keeping frame rates high and movement smooth and responsive. It’s really the beginning of a paradigm shift in massively multiplayer games, more developers will be moving to this platform in the future.
So, now we a bit about the technology involved, the next question is: will it work? Well, as mentioned earlier, a world record for concurrent players in a first-person-shooter game was attempted and set back in January of 2012 (you can see it here: https://www.youtube.com/watch?v=JNN_J0g5fmQ ). The player commentary I was able to dig up mostly indicated complaints about the gameplay being unbalanced, I found no complaints about performance, so it works at least that well. In the two years since that attempt I am sure that the tech has been further optimized, so it looks like Eternal Crusade won't fall prey to that particular bottle-neck.
)
There may be other bottle-necks ahead, rendering limitations, network limitations, database or memory issues, but, back in November, Patrick Balthazar, the Lead Programmer on Eternal Crusade, had this to say in one of their forum posts:
- network bottleneck: this is actually the biggest problem, not because of latency, but because of the amount of traffic that needs to be handled. imagine you get only the position data of every player you see 15 times a second, in simple terms a 3d position, 3 x 4 byte (4 byte = 32 bit = integer) that would be 15 x 4 x 1000 (for 1000 players) ~ 60 kb/s. Similar to this you have a lot more other information that needs to be transferred, like shots, effects, grenades, etc. so you can guess, normally, that is very heavy on your network. to solve that problem we have the so called 'area of interest' algorithm. basically, the closer something is to your location in the game, the more updates you get from there, the further something is away, the less updates. but in any case, this is something we are constantly monitoring and where we will optimize a lot.
- server load: we don't expect this to be a problem, since we are distributing the load onto multiple servers (the cell servers)
- server network: as of now all the servers will be in one network, specially designed to have very low latency, so that handovers of objects between them are very fast, we don't expect this to be a problem either.
- rendering: there is a multitude of things that can be done here, starting with loading, culling, mesh optimization, texture optimizations, reduction of draw calls, mesh combination, shaders, area of interest shading etc. while here we have the most tools to address the problems, it is definitely also one of the bottlenecks, because we will have to be able to render a lot of characters on screen.
- database: because we have persistent elements in the game, we will need to be able to save and load data on the fly, having a distributed database helps us to keep up with this task and make sure that this is also not a real bottleneck.
- memory: we are planning on running both client and server on 64bit so that we can address enough memory when needed. this will only be a bottleneck if the players don't have enough memory in their pcs. “
So it looks like Behaviour has anticipated all sorts of potential issues and is preparing solutions for each of them, in order to deliver an unparalleled Warhammer experience, and I, for one, can't wait to see what they come up with next.
