Face of Mankind - Beta Secrets
Although Beta 1 officially started on Monday, April 5th of this year, we had already sent out all invitation emails on previous Sunday. The very beginning, as was somewhat expected, was a total disaster. The game account interface of the old "Beta Portal" caused a few problems and the majority of our testers couldn't log in due to their game account containing invalid data, so we had to invite them manually one by one, fixing the database entries by hand.
Early on the morning of the 5th of April, all accounts were finally ready to go. At this point the excitement was such that even the thought of sleep was impossible. Before this point in time we had never seen more than five accounts logged in simultaneously, so watching the first logins was our only interest, and sleep could wait. It was awfully stirring to watch all those testers logging in and see them moving through our vision of New York City in the 24th century, but unfortunately it didn't last long. The server applications crashed after only about five minutes, and it crashed again a few minutes after each automatic restart. "Great start..." I said to myself, but I didn't really expect immediate success with a project of such complexity.
We spent the first day hunting that bug. Even if I had found the time for sleep, I wouldn't have closed an eye - the fact that our testers couldn't play longer than a few minutes was unbearable. I did a lot of testing throughout the day and into the night, but after 40 hours I had to follow the cues my body was giving me and get some sleep, although it wasn't very satisfying.
The next day the bug was found rather quickly in a new feature that hadn't been tested much before. After removing it, everything worked like a charm.
The testers and developers met in-game at Moe's Pub in NYC Manhattan. The mood was great. Some players still had no clothes, so it was rather fun to watch them jumping around. From Moe's Pub we travelled as a group to the Shibuya district of Tokyo at night. I was expecting a few more problems here, but everything continued without a hitch. Later the same day, the first patch was released to test the new patching system and to fix a few graphical and movement bugs.
I couldn't believe that the server was still running and stable after 24+ hours. Our testers had a lot of time to find all the countless world glitches and bugs. They also experienced a few client-side crashes, mostly caused by terminal functionality. When you started a mining process or wanted to refresh your process view, it crashed if you were doing it quickly one after the other. It was also determined that the client-side networking pipeline needed a big revision.
We had to tweak a few compiler problems where the release and debug builds were acting differently, and these bugs are always very hard to find.
Most of the server and client stability issues were to be fixed within the first week. In this week we all experienced a whole new world of game developing. It was our first beta for a massive multiplayer game. None of us had experience with this type of thing before. The mood in the IRC chat room and all the hype has been extremely exciting and will forever remain in our memories.
We wanted to catch the opinions of our testers and so we asked them to write short reports about their personal experiences in this first week of beta-testing Face of Mankind. (You can read it here: Early Beta Reports)
During the following weeks a lot of work waited for us. Our level designers had to continue their work on unfinished areas and also had to polish and expand existing parts of the game. We also released new worlds to keep the testers motivated. :)
The latter wasn't very easy as we concentrated our programming efforts on the economic system and the interface in general. That means hours of repetitive mouse clicking and moving to verify if a bug is still there or whether a new one appeared after a fix. At that point combat hadn't been activated yet, which was a purposeful tactical decision, as we felt that nobody would have tested the boring - but very important - interface issues if they could just play and have a little combat fun. Yeah I know, that was a bit nasty. :)
Besides fixing interface bugs, many features of the economic system had to be renewed. If you test something very often you get to know the procedure very well and only then do you come to learn how the process can be improved to be more usable and user-friendly.
After about 2 more weeks the interface felt as good as new and we were able to release our first major performance improvement patch. Many parts of the engine had been rewritten and we finally revealed the combat system to our testers, who had been anxiously waiting for it. We had a lot of fun in our first big fire fights. Although it still was very basic, it went smooth and was rather lag-free, which was nice to watch.
The first month had passed, with over 40 patches released, and we again asked our testers for reports about their personal experiences with the game. (You can read those here: Early Beta Reports: part 2)
During the next two weeks we spent most of our time improving combat and implementing implant functionality. Also, the real-time update mechanism had to be rewritten for better timing to reduce lag. Client-side movement interpolation and extrapolation needed a lot of new code as well.
In the meantime tons of new character skins, music tracks, environment objects and new worlds were added, keeping the entire team extremely busy.
On May 17th the time for Beta 2 had come. With now about 200 testers, many new bugs had been found and a lot of opinions were being offered for discussion. Over the next few weeks we concentrated on bug fixing and improvement of existing features, rather than adding missing features.
Now you might ask, "Why does all that take so long?" Let me be straightforward and to the point - Our team is small, and the workload severe. Had we more financial resources available to expand the team, it might have gone faster, we were already doing more than our bodies could stand, and doing things even faster without a larger team just wasn't possible.
The next big thing - the toughest feature of all - was the mission system. Damn, that was evil. The first version was ready by June 20th. We initially went with the plan of having structured ranks, in which missions were distributed from the top ranks down to the lowest via mission splitting. It took us a few weeks to get it all up and running in basic form, only to discover that it wasn't going to work exactly as we had planned.