Luca Fusi Sound Design | Implementation

24Apr/112

Game Audio – Movement Soundscape

The third term of my time at VFS has finished, which means we're 6 months from reality. It's incredible to think about how much I've learned since starting here last October; moreso to try to imagine all the things I've still got to learn. There's enough for several lifetimes, and seeing the work my classmates, our teachers and the wide world of sound on the Internet puts out every day is super inspiring and challenging. It's time to start looking back through my most recent batch of work...

Here's an end-of-term project for our Game Audio II course:

Game Audio II

Reconstructing Movement in Game Audio:

Environmental Sound

The aim of this project is to record source material, and then to re-construct a simulated environment for the user/listener to establish more than one ambient game audio context. The project should take the user on an auditory journey transitioning the user through more than one ambient ‘zone’, as if the character is moving through the game world on a “spline”.

After hearing all the examples of past students' work we were shown in class, I was pretty impressed by some of the ambient shifts these former all-stars had come up with. Some really cool stuff, like underwater to above-water, nice examples of occlusion through shutting a car door in the face of a horde of zombies, etc.

But while they all had lots of imagination and detail, none of them really felt like they were moving to me so much as the environment was changing over time.

I wanted to do better, here; to get movement across at all costs, even if it meant I had to keep my concept a little more grounded in reality. From jump, I thought that this would be a great chance to learn a bit about binaural recording (the project was going to be played back/evaluated in headphones) and to try some in-the-field source recording techniques, stay away from the very close-mic'd dry recordings we'd normally do in our Foley rooms.

The goal was to keep the environments simple and just let the movement speak for itself. And in that respect, I think I did pretty well.

What is Binaural Recording?

Brief aside on what all this "binaural" stuff means:

Binaural recording is a method of recording sound that uses a special microphone arrangement and is intended for replay using headphones.
-- Wikipedia, "Binaural recording"

Sound waves don't just dive straight into our brains via some kind of "line in" jack: they have to move through air to get there. And as they do that, these waves bounce and reflect off your shoulders and chest, wrap around your skull, move through your hair, filter through your clothing, etc., all the while undergoing subtle changes in frequency, volume, and arrival time (e.g. hit your left ear just before your right ear), to name a few.

Obviously, our hearing receptors don't sit in an open, microphone-shaped capsule in front of us, they're buried deep in our ear canals. So by recording via two tiny, omnidirectional microphones positioned roughly where your ears are, you are going to record the sound not "as it sounds" in some pure, idealized sense, but as it sounds to you.

When played back, all of those subtle frequency/delay/arrival time changes you captured by recording this are interpreted by your brain in a way that makes you instinctively feel as though those objects are coming from where they were when you originally heard them. We don't hear "slight loss of frequencies around the 8,000Hz range because those tiny waveforms were absorbed by my haircut," we hear "behind." As an incorrect but illustrative example.

By recording this way we can capture much more detail than just "left" or "right" - we can get above or below, in front or behind, etc. And you can turn two channels of simple stereo into sounding like a whole 360-degree sphere of surround.

And yes, people build lifelike human mannequin heads, complete with density-matched materials to simulate the hard reflections of our skull vs. the soft, absorptive tissue of our outer ears, embed microphones in them and use these devices to do these sorts of recordings. At the fanatical level.

Raaaaaaaawr

This is a popular binaural recording example that might blow your mind - wear headphones.

There's a lot more to say on this than I can teach, so if you're interested, look around. Our ears are amazing.

Deciding on a Scenario

First, I brainstormed a few scenarios - I wanted something simple and real, with the chance to show off a lot of movement and cool perspective changes. The most interesting to me was that concept of a Metal Gear Solid-style infiltration, with some close spaces, hearing people filtered through a vent shaft, and eventual running/scuffling.

I "prototyped" some of my recording techniques by squirming around in a broken vent we have in our props room and adding a little bit of reverb to the signal, and got some really nice, claustrophobic results. I also affixed two omnidirectional clip mics to a pair of classes, hooked myself into a portable recorder and walked around our campus a bit to see if it'd work as a poor man's binaural setup. The results weren't 100% interesting all the time, but a couple of cool moments convinced me that this was an avenue to chase.

Planning and Recording

Vent decided on, I sketched out the rest of the soundscape and the environments it'd take place on - again, sticking to stuff that was close to campus so that I could use a lot of my own recordings. I had to figure out roughly how much time I wanted the character to spend in each environment, plus making sure that I was hitting the assignment requirements in terms of "signals" (in-your-face, attention-calling sounds like guard footsteps and key jingles), "keynotes" (middle-ground emitting sounds like vending machines, vents, a radio) and "ambiences" (persistent background sounds that give some subtle information about where you are).

With only 90 seconds to work with, I wanted to make sure you knew where you were as fast as possible whenever the environment changes. If you were in a vent, you needed to know within a couple of seconds. So I put a lot of emphasis on making those binaural recordings the "bed" of the piece, cranking them in the final mix. Normally, your ears tune out "room tone" as you hang around an environment, but I thought that if I were too subtle with these recordings, the cool perspectivey effects wouldn't come through.

Some things I recorded this way were all of the different room tones I wanted, the character's footsteps, and the stairwell door opens and closes - this let me put all of that easily within "his" perspective without having to do any extra mixing. I also spent a bunch of time tilting myself around like an idiot outside in our school's courtyard to get some really dizzying panning on the nearby traffic for the fight up on the rooftop. That, with some of the low- and high-end rolled off to give a "rooftop" sense of distance, became the ambience for the final fight on the rooftop.

For the main character's breaths, I used a single lav mic and just panned it directly in the center (when little else was) to give it a bunch of presence. Simple but effective!

All other effects were recorded in mono with the phenomenal Sennheiser 416. I love love love this microphone for what it's good for, and it feels great to finally be discovering favorites. It's extremely directional, which means I was able to walk around campus and point it at the tiniest little emitting spots on machines, coolers and various electronics and pick up their distinctive buzz without capturing too much of the rest of the environmental noise. That plus our Foley rooms' new preamps allowed me to grab some extremely clean recordings of all the other stuff I needed: guard boots, cloth punches, key jingles, mug and paper tosses, etc.

These recordings were also almost totally free of our floor's soul-crushing, inescapable 120Hz hum. As a result, I barely needed to treat these recordings with EQ etc. before I started cutting the final piece. Saved me a ton of time.

Finally, the brilliant guard dialogue was written by yours truly in a lame attempt to explain why these two guards just wouldn't shoot the guy on sight, plus give a little extra depth and a sense of threat to the whole piece.

I wanted to experiment a bit more with having my classmates (the two guards) read their lines a bit farther back from the mic and even "off-axis" (not facing the direction that the mic picks up sound in) to try to have that sense of people moving around and not talking directly towards the main character's ear, but stuck to just recording it normally and positioning it in the mix later on. The one "hey!" in the stairwell was recorded a few floors up and in the stairwell, though.

Editing and Mixing

My major discovery for this part of the project was WaveArt's Panorama plug-in, which I managed to convince our IT department to install a trial of on my machine. It's a 3D panner that uses some of those same binaural recording principles to subtly adjust certain frequencies, volumes and delay times in a recording to let you position objects wherever you want within a stereo sound field. And as an RTAS plug-in, everything in it was totally automatable.. which basically let me "draw" the path of all the objects in my soundscape according to where I thought my main user should be. Tooooooo freaking cool.

The Panorama interface.

The only shortcoming I found with it was that it doesn't respond well to really quick pans/head turns, as that's just too much math too fast for the plug-in to crunch. You end up getting a lot of phasey, washy sounds when you do it, so I had to keep the 3D panning stuff slow and gradual. Small price to pay! Being able to place the guard's voices below and increasingly behind the character as he moved forward was amazing, and I'm especially happy with how the quick front-to-back pans on the office objects (mug of pens, paper) worked in the middle of the chase. That plus the whirl-around as the guards open the door on the roof.

The next big victory in positioning everything in this environment came from messing around with multiple reverbs. When I started, I was using a standard low-pass filter (that "muted" effect) to try to make the guards' voices sound as if they were being blocked by the walls of the vent, but it felt forced. Sending their voices into the same claustrophobic, metallic-sounding reverb I was using for the main character's vent squirming, while keeping their dry voice signal down, worked way better. Favorite examples of verb usage in the soundscape are as the main character pans his head back up and one of the guard's voices seems to "move" into the vent, as well as the way I treated the alarm as the character bursts into the stairwell.

Final Result

Separated into stems because we had to turn them in that way, and it might give you a better look at what's happening in each part of the whole.

Ambiences Only:

Keynotes Only:

Signals Only:

Final Mix:

What Went Right / What I Learned

  • You shouldn't be scared of recording *in* a location when you want something to sound like it happened there. It seems obvious, I know, but up until this project, I really felt like the safest way to do things was to record sounds clean and close in a soundproof room, then control the way everything was positioned/echoing in a mix later on. For an assignment like this, I saved a ton of time by *not* doing this that way, and even managed to create some more realistic results by doing it. I couldn't have made that stairwell running happen as realistically in any other way but just recording myself jogging up them with two tiny mics clamped to my glasses.
  • A little noise isn't a big deal. Seriously. When recording room tones etc. with our tiny omnidirectional microphones, I had to turn the gain way up in order to get anything to record at all. Doing tihs added a ton of hiss and white noise into the recordings - the mics weren't meant to really be turned that high - but in the final mix, once all the ambiences were in and with everything else going on, you couldn't hear that shit at all. If I had spent any time cleaning those recordings up with Xnoise etc. before mixing them, it would have been totally wasted.
  • The Sennheiser 416T is awesome. So is Panorama.

What Went Wrong

  • In retrospect, the whole project took me way longer than it should have due to my own meticulousness - if I had gotten my shit straightened out with recording planning and done that all in one day, things could have happened faster, but because of this term's schedule, all my recording sessions were happening on and off through the end of the term. As usual, "organization will set you free."
  • I wouldn't have done the fall from the vent/ring-out as dramatically, if I could go back and do it again. It's a bit over-the-top and Gears of War-y, but it helps cover up the fact that there's not much going in that section of the piece besides some footstep scuffles. A straight ring with a less dramatic low-pass might've done it.
  • Not totally totally sold on the fall from the roof as well - I wish I had some closer-sounded traffic sources to gradually fade in to give a better sense of proximity. Struggled a lot with this last section!

Please leave any questions/thoughts/criticism in the comments!