Music!

Hi everybody,

After a long silence, I'm going to talk today about music in the game, and more specifically how I make it. I recently tried to re-record the intro music because I had made it quite quickly in the first place.

I - First audio tests: Ardour

For the first music recording, I used the Linux software "Ardour", which is the "equivalent" of Protools, a famous recording software in the audio/musical industry. It may not have all of its features, but it was enough for me. I quickly described this setup at the beginning of this post.

The sound softwares are really great (and free) under linux, and they have a very different philosophy than the ones of Windows or Mac: they all rely on a "sound server" software (called JACK) which is designed to treat the audio information with a minimum delay to have a real-time response, but, above all, ALL audio softwares are compatible between them because they use the same Input/Output interface (relying on JACK, the sound server). So, under linux, each software has a speciality (audio recording, post processing, effects, synthetizers, drum pattern creation, score editing, midi recording) and you can mix them as you want, whereas on other OS, it is generally difficult to make softwares from concurrent companies communicate between themselves.

The incredibly powerful and intimidating interface of Ardour, the reference of audio recording softwares on Linux.

However, in my case, the problem is, Ardour is above all an audio recording software, which means it is designed to record REAL instruments, if you have the good microphones, and hardware setup. It is very powerful, and I recorded the intro music with my simple Yamaha Keyboard, but I'm not a perfect musician, and I could hear a lot of mistakes (on the rhythm especially). In this case, there is no other solution than to re-record again and again to correct the mistakes. Moreover, I am limited to the (poor) sound quality of my little keyboard, which sounds a bit cheap.It's not a big problem in the kind of video game I plan to do (because, except in big productions, it's quite rare to have a symphonic orchestra recording the music), nevertheless, this poor sound quality annoyed me.

Here is the first try of the intro theme recorded directly in audio from my keyboard:



If you can't see the audio player above, try this link
 
So after testing "real" instrument recording, I switched to another pipeline: MIDI recording.

II - The promise of MIDI tools

As a lot of compositors, I finally chose to use a MIDI sequencer in order to create the musics. I use LMMS, a simple but powerful software from the Linux world (again) but which works also under Windows.

[By the way, at the moment, I use it with Windows because my full pipeline production is a Windows-based (for development, graphics, audio...) but all the softwares I use also work under Linux... except Unity, my software development tool. But when they'll decide to port it to Linux, I can switch my full pipeline production under Linux for better stability while keeping the exact same softwares.]

Anyways, what is a MIDI sequencer?

A MIDI (Musical Instrument Digital Interface) sequencer is a software that enables you to write/record musical scores, but in a simplified version (not the real classical notation, even if it is sometimes also possible): when I "record"/"write", the computer doesn't actually record the SOUND coming out of my Yamaha keyboard, it records the NOTE, its PITCH and DURATION (with a few more informations), and it creates a simplified musical score with these notes. The 3 big advantages of this method are:

• You can make many change AFTER having recorded the musical part: change the sound of the instrument by using what we call "soundfonts", change the tempo, ...
• You can correct errors with a simple click on the score (move a note, change a duration, change the velocity...)
• You can "quantize" the score, that is to say that every note is played with a perfect timing according to the tempo of the song (thus, there is no rhythm approximations).

Of course, not everything is perfect, but this new process simplifies a bit the musical creation.

For each recorded instrument part, you have to select a "soundfont". This is pretty much the same thing than a "font" for a text except it doesn't define how characters will look like, it defines how notes will sound. This is some kind of instrument bank. And there are lots of them on the internet, with different qualities, sizes, and instrument types. In the end, I've got many more sounds available than the simple sounds from my Yamaha keyboard, with a better quality too.

The LMMS MIDI sequencer. In the left window, the blue segments represent different pattern/melody parts. Each line is a different instrument with different settings. For each blue segment, you can modify it note by note on the right window (the "piano roll") where you can add notes, delete them, move them, change their duration or velocity... As you can see, it's not a real musical score; the visual representation is quite simplified and intuitive.

III - Installing LMMS

[This part is a bit technical, so go to the next one if you want]

So this seems quite cool, but setting up the software correctly took me some time, because I have a basic soundcard. Indeed, on Windows, the sound drivers always introduce some "lag" when recording from audio devices. It seems to be a known issue. Thus it is impossible to record from my keyboard because each note is systematically played 1/4 beat late. This drived me crazy.
Fortunately, the ASIO company has some kind of "generic no lag driver" which seems to do the trick. But this new driver seemed not to be supported by LMMS natively. I had to find a precompiled DLL driver library with the ASIO driver support and replace the native LMMS DLL driver library.
I wouldn't have had those problems under Linux, but this finally saved me a lot of reboots when working on audio composition.

However the version of LMMS I got also had a MIDI recording problem with soundfonts tracks (Haaa, free software ;) ). But the community is great, and a new version correcting the problem was released a few days after.

The last problem was using soundfonts. I found a huge soundfont with great quality but it needs more than 1.5 GB in memory (yes, you read well), and the 32-bit version of LMMS can't load it. So I finally had to re-setup my whole pipeline in 64 bits (LMMS + real-time DLL drivers). To this point, I still have a few audio output problems from time to time, but it's globally OK.

Achieving to have a "stable" setup is sometimes long and difficult, but once it's done, you can finally work! Because yes, until now, I haven't produced anything.

But after trying various instruments, learning how to use the software (I read the documentation and tutorials for a few hours), I finally could re-create the intro theme! And here is the same song than before, but recorded in MIDI with a better sound quality (I hope):



If you can't see the audio player above, try this link
 
All this work for this 1 mn of the same music... Depressing isn't it? But now I know I can easily create other music, and that's the good part. This first part took me several days to achieve, but after that, I created another test track in a few hours only. Meanwhile, I keep listening to the original soundtracks of my favorite games on youtube for hours to understand how they are built. Very enlightening :)

IV - Integrating the music in the game

Now I have musics of better quality, but how are they played in the game?
In the game, I created a few different "musical zones". When the character enters a zone, it starts the corresponding music (with a little fading-in for a smooth start). An when the character leaves, it stops the music.
But this is not as simple as that, because sometimes, I want a different music for a little cinematic taking place somewhere where there is already a musical zone. So I added some kind of priority information for nested musical zones. Moreover, I also have to deal with memory managerment (musics have a huge impact on memory use) and I must clean them when the character leaves a zone, especially now that the world is streamed into memory...

Anyways, that's the basic process of creating a music for a video game. That was a bit complicated to setup (especially using LMMS under Windows with my poor sound card), but this should be easier now to create additional musics.

Thanks a lot for reading until here, and see you next time, maybe for an updated demo!

Peace.

PS: The final news which made me depressed: the last version of Ardour (the soft I first used) now handles MIDI recording/editing... But I didn't see that before. Anyways, my work in MIDI is not lost, since I should be able to import it in the new Ardour version. I might give it a try one day, but I'll have to update all my linux installs. I guess I'm not finished with all that musical stuff...