Daily life is filled with a cacophony of noises and melodies. We are constantly bombarded with invisible waves that tickle our ears. Sometimes those waves create or amplify powerful emotions within us...
Obviously I can't make a game without sound right?
So of course I needed to deal with all manor of sound issues. My entire week was consumed by the quest for sound. The sound system is rather complicated, having internal designations and special structures for different types of sound, and also having a number of transforms and effects processing (reverb, doppler, etc). The main complication, however, was the nearly completely missing pipeline tools to build the sound assets into a useable package for the client.
Putting the wind in the pipes...
Before sound assets could be put through the nearly non-existent pipeline system, I had to deal with a more fundamental issue. I needed sound assets. My brother, being a musician and having his own studio would be a logical source, but he is busy with various things and what I really needed was a way to thoroughly test whatever pipeline systems I fixed up. The perfect test would be to have all of the original sound assets that I could then compile and then compare with the precompiled sound pack, at the byte level and also in game. If I can take a bunch of .wav files and stuff em through whatever pipeline i managed to put together, and the end result was functionally equivalent to the precompiled sound pack, I would know my pipeline is working and could then proceed to fill it with original sounds. So my first job was to reverse engineer the precompiled sound pack and reconstruct the original .wav files from the packed data.
Reconstructing a ton of little .wav files meant absorbing all of the code that read and used the stuff, and also researching all of the metadata (.sound and .sound_anim files). All of the sounds used in the game are first organized into sample banks, which are basically files that contain serialized sound buffers containing raw data in 2 formats ( 16bit mono PCM and ADPCM ). Probably the most difficult thing about extracting and recreating the individual .wav files was that there was no standard sample rate. A large portion of the sounds were recorded at 22,050Hz but many were at 16KHz and there were a whole range of things at every odd frequency you would never think of using normally, like 13KHz or 5KHz. I suppose for pure efficiency, low frequency sounds do not really need high frequency sample rates, so the original sound designers figured out the lowest most efficient sample rate for each sound that still kept as much fidelity as possible. In total, there were 11 different sample rates in use.
After a lot of trial and error (which resulted in very loud static noises), I had 1043 .wav files organized into 71 sample bank folders. Yay. My eardrums were quite happy to be done with all of that.
When, where, and how many times?
The sounds are not going to play themselves, so it was obviously required that I either create the metadata files from scratch or reconstruct them from the packed data. I wanted the closest I could get to the original stuff, so I chose to reverse engineer the metadata too. This was an entirely different challenge. Turning serialized binary data back into the original XML it all came from is kind of insane. But I like insane stuff so I dove right in.
There are five different fundamental sound types that the sound engine uses for various purposes. All of them share a base data format for things like gain, transposition, 3d sound projection, and distance. Beyond those basics, everything changes depending on the sound type. Most of the sounds were "simple" sounds... one shot sounds played in connection with animations such as sword swings, animal grunts, combat impacts, etc. "Music" sounds were pretty easy also, the main feature on these sounds are crossfading, minimum play time and minimum delay before replaying. The oddest sound type was the "background" sound type... each was an array of individual sounds, each sound having a bank of 32 possible filters attached to it.
After another round of trial and error, I had 2599 .sound files. Apparently a lot of the .wav files were reused for a number of purposes, most likely using the transpose and possibly also filters to make them sound different. It took eons ( days ) to get these files right. Luckily, I did not need to recreate the .sound_anim files... these were conveniently left as intact XML files inside the sound asset pack.
Now I had the prerequisite assets... I just needed the pipeline tools and scripts to compile it all again.
Digging the trench, laying the pipe...
As I mentioned at the beginning, the asset build pipeline had nearly no concept of audio. There was a single folder with a sound process script, but it was never called by any other script, the directories and configuration variables were not set anywhere, and the binary tool itself was missing. There was definitely not a lot to work with, but this little bit was way better than nothing.
Digging through the cmake project for the entire game engine ( client / server / drivers / tools ), I had hoped to find the sound compiler tools, but there were no sound tool projects at all. There were some sample projects that used compiled sound assets, so there had to be something somewhere to compile the sounds. Searching the source tree, I found three source files for the tools, but there were no project files. It was seriously just three .cpp files, not even any headers, and each .cpp file was in its own folder with a readme file with the command line to use the tool. Wow. So I fumbled around trying to build project files for these things, and after a ridiculous amount of time (hours) I finally managed to get the dumb things to compile. A trip to the command prompt confirmed the resulting tool actually operated (at least enough to tell me I'm doing it wrong, and by the way, here is the proper command line sequence....)
Ok I have the audio converter tool, I have the assets, I just need to figure out some python scripting for the pipeline. Oh yeah, I love python. Except I've only messed with it in passing ( adjusting the pipeline scripts ). I'm pretty sure I never wrote a python script from scratch. Well there is a first for everything.
Ok I hate python. Or it hates me. Or I'm retarded. It seemed to take me longer to make a working python script (which looks like it should be stupidly easy) than it did to figure out and compile the binary tool. Anyway after what seemed like a battle to the death (to my poor fingers at least), I got the new script working with the rest of the pipeline. Directly after the most basic test, I set the pipeline to rebuild all those assets back into the compiled package I extracted them from. I crossed my fingers and waited.
I must be hearing things... or is it selective?
So I took the resulting sound pack and added it to my test client, and started it up. My loading background music was working, but that wasn't really part of the challenge. Once I got in the game world, I heard sound. I was like woohoo and then I was like wait.... I can hear the background sounds like wind and bird chirps and odd frog noises and such (totally out of place on a nearly barren moon of course) but I did not hear any noises coming from the little test animals. I tried a few actions that would normally produce a sound effect but none of them made sound. It seemed that the sounds that were not working were all the ones designated "simple" sounds. With probably the deepest sigh I have ever produced to date, I turned to my trusty hex editor for another session of retina-burning research.
It did not take me long ( and my retinas were most thankful ) to find the issue, or at least the right direction. About 1/3 the way through any one of the "simple" sounds compiled by the binary tool I found a great big gaping hole of zeros. I got the notion to take my recompiled assets and put them through the decompiler I made, and it showed me quite clearly what was missing... every single compiled audio asset was missing the data designating the number of 16 bit samples each sound was made of. Without knowing how many samples to make a buffer for, the game engine was simply rejecting those sounds as unusable. The problem of course came from the audio converter tool I had earlier compiled.
Debugging the audio converter tool did not take terribly long, but the issue was probably one of the most ridiculous ones I have seen in the whole codebase. Right there in the source of the sound system, the code was seriously resetting the variable that held the number of samples, just before calling the next function that needed that very information. Really. So I added a little bit of code and a few snarky comments, and recompiled the tool.
After moving the newly recompiled tool to its proper location and running the pipeline process again, and copying the resulting asset pack to my test client... I held my breath and started up my client again. I wasn't able to hold my breath terribly long but I made it at least until the loading background music started playing... but anyway, when I got in game and I heard the weird animal barks and such I knew that I had pretty much conquered the sound system. Once again I was a happy Zombri.
The end of the beginning is on the horizon...
With the sound system fixed up, I am ever closer to being able to concentrate all of my time and talent (assuming I have any, I think I do!) on world building and game design. Of course I can't just use all of these audio assets I have decompiled and recompiled, I will definitely be replacing it all with stuff I'm legally licensed to use. The experiment is over, my tools work and I have all of the formats and information I need to compile a completely original audio asset pack. But before I start doing all of that, and all of the other game design stuff I have put on hold, I need to finish my build pipeline and completely free my client from all precompiled assets.
I only have 6 more files to deal with. I am hoping I can finish off these last few files this next week. After that, the game will begin to really take shape!
Wish me luck, and look forward to next week's developments...
the Undead Dev