Luca Fusi Sound Design | Implementation

24Apr/112

Game Audio – Movement Soundscape

The third term of my time at VFS has finished, which means we're 6 months from reality. It's incredible to think about how much I've learned since starting here last October; moreso to try to imagine all the things I've still got to learn. There's enough for several lifetimes, and seeing the work my classmates, our teachers and the wide world of sound on the Internet puts out every day is super inspiring and challenging. It's time to start looking back through my most recent batch of work...

Here's an end-of-term project for our Game Audio II course:

Game Audio II

Reconstructing Movement in Game Audio:

Environmental Sound

The aim of this project is to record source material, and then to re-construct a simulated environment for the user/listener to establish more than one ambient game audio context. The project should take the user on an auditory journey transitioning the user through more than one ambient ‘zone’, as if the character is moving through the game world on a “spline”.

After hearing all the examples of past students' work we were shown in class, I was pretty impressed by some of the ambient shifts these former all-stars had come up with. Some really cool stuff, like underwater to above-water, nice examples of occlusion through shutting a car door in the face of a horde of zombies, etc.

But while they all had lots of imagination and detail, none of them really felt like they were moving to me so much as the environment was changing over time.

I wanted to do better, here; to get movement across at all costs, even if it meant I had to keep my concept a little more grounded in reality. From jump, I thought that this would be a great chance to learn a bit about binaural recording (the project was going to be played back/evaluated in headphones) and to try some in-the-field source recording techniques, stay away from the very close-mic'd dry recordings we'd normally do in our Foley rooms.

The goal was to keep the environments simple and just let the movement speak for itself. And in that respect, I think I did pretty well.

What is Binaural Recording?

Brief aside on what all this "binaural" stuff means:

Binaural recording is a method of recording sound that uses a special microphone arrangement and is intended for replay using headphones.
-- Wikipedia, "Binaural recording"

Sound waves don't just dive straight into our brains via some kind of "line in" jack: they have to move through air to get there. And as they do that, these waves bounce and reflect off your shoulders and chest, wrap around your skull, move through your hair, filter through your clothing, etc., all the while undergoing subtle changes in frequency, volume, and arrival time (e.g. hit your left ear just before your right ear), to name a few.

Obviously, our hearing receptors don't sit in an open, microphone-shaped capsule in front of us, they're buried deep in our ear canals. So by recording via two tiny, omnidirectional microphones positioned roughly where your ears are, you are going to record the sound not "as it sounds" in some pure, idealized sense, but as it sounds to you.

When played back, all of those subtle frequency/delay/arrival time changes you captured by recording this are interpreted by your brain in a way that makes you instinctively feel as though those objects are coming from where they were when you originally heard them. We don't hear "slight loss of frequencies around the 8,000Hz range because those tiny waveforms were absorbed by my haircut," we hear "behind." As an incorrect but illustrative example.

By recording this way we can capture much more detail than just "left" or "right" - we can get above or below, in front or behind, etc. And you can turn two channels of simple stereo into sounding like a whole 360-degree sphere of surround.

And yes, people build lifelike human mannequin heads, complete with density-matched materials to simulate the hard reflections of our skull vs. the soft, absorptive tissue of our outer ears, embed microphones in them and use these devices to do these sorts of recordings. At the fanatical level.

Raaaaaaaawr

This is a popular binaural recording example that might blow your mind - wear headphones.

There's a lot more to say on this than I can teach, so if you're interested, look around. Our ears are amazing.

Deciding on a Scenario

First, I brainstormed a few scenarios - I wanted something simple and real, with the chance to show off a lot of movement and cool perspective changes. The most interesting to me was that concept of a Metal Gear Solid-style infiltration, with some close spaces, hearing people filtered through a vent shaft, and eventual running/scuffling.

I "prototyped" some of my recording techniques by squirming around in a broken vent we have in our props room and adding a little bit of reverb to the signal, and got some really nice, claustrophobic results. I also affixed two omnidirectional clip mics to a pair of classes, hooked myself into a portable recorder and walked around our campus a bit to see if it'd work as a poor man's binaural setup. The results weren't 100% interesting all the time, but a couple of cool moments convinced me that this was an avenue to chase.

Planning and Recording

Vent decided on, I sketched out the rest of the soundscape and the environments it'd take place on - again, sticking to stuff that was close to campus so that I could use a lot of my own recordings. I had to figure out roughly how much time I wanted the character to spend in each environment, plus making sure that I was hitting the assignment requirements in terms of "signals" (in-your-face, attention-calling sounds like guard footsteps and key jingles), "keynotes" (middle-ground emitting sounds like vending machines, vents, a radio) and "ambiences" (persistent background sounds that give some subtle information about where you are).

With only 90 seconds to work with, I wanted to make sure you knew where you were as fast as possible whenever the environment changes. If you were in a vent, you needed to know within a couple of seconds. So I put a lot of emphasis on making those binaural recordings the "bed" of the piece, cranking them in the final mix. Normally, your ears tune out "room tone" as you hang around an environment, but I thought that if I were too subtle with these recordings, the cool perspectivey effects wouldn't come through.

Some things I recorded this way were all of the different room tones I wanted, the character's footsteps, and the stairwell door opens and closes - this let me put all of that easily within "his" perspective without having to do any extra mixing. I also spent a bunch of time tilting myself around like an idiot outside in our school's courtyard to get some really dizzying panning on the nearby traffic for the fight up on the rooftop. That, with some of the low- and high-end rolled off to give a "rooftop" sense of distance, became the ambience for the final fight on the rooftop.

For the main character's breaths, I used a single lav mic and just panned it directly in the center (when little else was) to give it a bunch of presence. Simple but effective!

All other effects were recorded in mono with the phenomenal Sennheiser 416. I love love love this microphone for what it's good for, and it feels great to finally be discovering favorites. It's extremely directional, which means I was able to walk around campus and point it at the tiniest little emitting spots on machines, coolers and various electronics and pick up their distinctive buzz without capturing too much of the rest of the environmental noise. That plus our Foley rooms' new preamps allowed me to grab some extremely clean recordings of all the other stuff I needed: guard boots, cloth punches, key jingles, mug and paper tosses, etc.

These recordings were also almost totally free of our floor's soul-crushing, inescapable 120Hz hum. As a result, I barely needed to treat these recordings with EQ etc. before I started cutting the final piece. Saved me a ton of time.

Finally, the brilliant guard dialogue was written by yours truly in a lame attempt to explain why these two guards just wouldn't shoot the guy on sight, plus give a little extra depth and a sense of threat to the whole piece.

I wanted to experiment a bit more with having my classmates (the two guards) read their lines a bit farther back from the mic and even "off-axis" (not facing the direction that the mic picks up sound in) to try to have that sense of people moving around and not talking directly towards the main character's ear, but stuck to just recording it normally and positioning it in the mix later on. The one "hey!" in the stairwell was recorded a few floors up and in the stairwell, though.

Editing and Mixing

My major discovery for this part of the project was WaveArt's Panorama plug-in, which I managed to convince our IT department to install a trial of on my machine. It's a 3D panner that uses some of those same binaural recording principles to subtly adjust certain frequencies, volumes and delay times in a recording to let you position objects wherever you want within a stereo sound field. And as an RTAS plug-in, everything in it was totally automatable.. which basically let me "draw" the path of all the objects in my soundscape according to where I thought my main user should be. Tooooooo freaking cool.

The Panorama interface.

The only shortcoming I found with it was that it doesn't respond well to really quick pans/head turns, as that's just too much math too fast for the plug-in to crunch. You end up getting a lot of phasey, washy sounds when you do it, so I had to keep the 3D panning stuff slow and gradual. Small price to pay! Being able to place the guard's voices below and increasingly behind the character as he moved forward was amazing, and I'm especially happy with how the quick front-to-back pans on the office objects (mug of pens, paper) worked in the middle of the chase. That plus the whirl-around as the guards open the door on the roof.

The next big victory in positioning everything in this environment came from messing around with multiple reverbs. When I started, I was using a standard low-pass filter (that "muted" effect) to try to make the guards' voices sound as if they were being blocked by the walls of the vent, but it felt forced. Sending their voices into the same claustrophobic, metallic-sounding reverb I was using for the main character's vent squirming, while keeping their dry voice signal down, worked way better. Favorite examples of verb usage in the soundscape are as the main character pans his head back up and one of the guard's voices seems to "move" into the vent, as well as the way I treated the alarm as the character bursts into the stairwell.

Final Result

Separated into stems because we had to turn them in that way, and it might give you a better look at what's happening in each part of the whole.

Ambiences Only:

Keynotes Only:

Signals Only:

Final Mix:

What Went Right / What I Learned

  • You shouldn't be scared of recording *in* a location when you want something to sound like it happened there. It seems obvious, I know, but up until this project, I really felt like the safest way to do things was to record sounds clean and close in a soundproof room, then control the way everything was positioned/echoing in a mix later on. For an assignment like this, I saved a ton of time by *not* doing this that way, and even managed to create some more realistic results by doing it. I couldn't have made that stairwell running happen as realistically in any other way but just recording myself jogging up them with two tiny mics clamped to my glasses.
  • A little noise isn't a big deal. Seriously. When recording room tones etc. with our tiny omnidirectional microphones, I had to turn the gain way up in order to get anything to record at all. Doing tihs added a ton of hiss and white noise into the recordings - the mics weren't meant to really be turned that high - but in the final mix, once all the ambiences were in and with everything else going on, you couldn't hear that shit at all. If I had spent any time cleaning those recordings up with Xnoise etc. before mixing them, it would have been totally wasted.
  • The Sennheiser 416T is awesome. So is Panorama.

What Went Wrong

  • In retrospect, the whole project took me way longer than it should have due to my own meticulousness - if I had gotten my shit straightened out with recording planning and done that all in one day, things could have happened faster, but because of this term's schedule, all my recording sessions were happening on and off through the end of the term. As usual, "organization will set you free."
  • I wouldn't have done the fall from the vent/ring-out as dramatically, if I could go back and do it again. It's a bit over-the-top and Gears of War-y, but it helps cover up the fact that there's not much going in that section of the piece besides some footstep scuffles. A straight ring with a less dramatic low-pass might've done it.
  • Not totally totally sold on the fall from the roof as well - I wish I had some closer-sounded traffic sources to gradually fade in to give a better sense of proximity. Struggled a lot with this last section!

Please leave any questions/thoughts/criticism in the comments!

5Mar/110

Term 2 Work, Pt. 3 – Train Car Physics Impacts

Intro to Game Audio

Impacts

Second set of physics impacts for our Intro to Game Audio class, same rules as the first - five lights (10"), five mediums (10'), five heavys (100'), five supers (100'+, hurled). Impact surface was concrete once again.

Train Car

Early impacts planning for the train car recording.

Early impacts planning for the train car recording.

The biggest goals with the train were to make it big at all levels (while still giving myself room to go up), and make it feel cohesive – not like a big pile of assorted junk. I really wanted the sounds to make you feel like a giant hunk of metal was moving around, so I thought a lot about timing.

Since I couldn’t record any sounds from an actual train car and nothing I could get from our props room felt “big” enough to get that vibe across, I resigned myself to using the Foley room stuff for high-mid and high detailing, possibly a little bit of color for the low mids if the pitching worked out well. I suspected I’d be using some really large metal groans to give my sound its defining shape, and that premonition was dead-on.

From what I recorded myself, the car door and some sheet pans were the biggest contributors to the final sound. The car door impacts didn’t do much for me, but I got some great stress sounds by standing on the thing and slowly shifting my weight around. When pitched, these gave a ton of character to the longer impacts. The sheet pans were used for high, ear-splitting scrapes across the concrete that I pitched and used later to convey a really stressful, fatiguing sound of this giant car sliding across concrete.

Had a lot of fun with the supers on this one, as well. Listening to some other students’ impacts inspired me to try to take really novel approaches to the intro/stinger of each super.

With more time, I would've given a little more attention to shaping the overall mix - there's a lot of really aggressive, nasty mid-high frequencies in here that made the impacts mixing a physically fatiguing process. I could do to strip some of that down a bit for repeat listening in a game environment. As well, I would've spent a little more time sculpting the low-end of these sounds to make it feel as big as a real train car might. These sounds in particular are definitely best listened to with a good set of speakers or cans.

All are provided below for your enjoyment!

[Light Impacts]

[Medium Impacts]

[Heavy Impacts]

[Super Impacts]

4Mar/114

Term 2 Work, Pt. 2 – Bowling Ball Physics Impacts

Intro to Game Audio

Impacts

Definitely the longest assignment of the term - each student was given a pair of objects (not literally, just assigned in name) to design 20 impact sounds for at different forces. Five lights (10"), five mediums (10'), five heavys (100'), five supers (100'+, hurled). Impact surface was concrete, with some flexibility here. We had a few weeks for this one and were encouraged to record as much original source material from our Foley/props room as possible.

Much of the below taken from my project post-mortem. Enjoy!

Bowling Ball

The first thing I did when you assigned this to us was to start conceptualizing all the different layers that’d be involved in each of my objects’ impacts, so I had a checklist to work from when it came time to record.

Early impacts planning for the bowling ball recording.

Early impacts planning for the bowling ball recording.

Approach here was to really imagine what was going to happen to the surface itself at the higher impacts, since the ball itself is a pretty one-dimensional sound source. Other than that, I just knew I’d want to record the ball dropping on multiple surfaces so that I could stack and vary the balance of those impacts to fatten up or thin the ball drop sound.

[Light Impacts]

The biggest challenge was trying to get the really impactful, transient crack I wanted for the higher impacts (as the ball would literally split the concrete), while still keeping the hollow thud of the ball sound there as well. I mixed the three different surface drops with some short LFE accents gave me the roundness of the ball, and then tried to use some snappy sweeteners like walnut cracks to give the impression of the concrete breaking. I couldn't find anything nicer than this in our library to fit the task; with a little more time I might've gotten luckier.

[Medium Impacts]

I focused on making sure that the listener was able to tell that this was, in fact, a bowling ball – and not just some solid heavy object – at all velocities. Structuring each fall as a series of bounces helped this a bunch, as did the tail-end rolls I had recorded.

[Heavy Impacts]

I started work on this guy before the train car to try to get my chops up before moving onto the more complicated sounds. Unfortunately, this meant that I was designing these without the aid of a ton of the tricks we'd later learn in Intro to Post Audio Editing, so making variations was slow.

[Super Impacts]

With some extra time, I would've put more effort into varying up the supers - I left off after the first one to get onto to other projects, and by the time I returned I was totally sapped for inspiration, so they're not *quite* at the imaginative level of my train car supers.

My other object was a train car - I'll have those recordings and a little retrospective up in a few days.

12Jan/110

Ye Shall Receive

Just a quick update while I bounce some files out, here: Term 2 is serious business.

It's been only a week since I got back, and I already feel like I've been working for a month. Classes two or three times a day, four final assignments already in the headlights, a bout of the flu this past weekend, and our first triple-class day just now (9AM-8PM). You get what you pay for, right?

The bar's gone way up, way fast. Mastering my recordings from over break has turned into a gigantic ordeal, professional dialogue editing is hard as shit to do convincingly, and the devil, as always, is in the details. I'm starting to get a picture of how much I'll know (and how frazzled I'll be) by the time this program's over: a happy, smart little crater in the ground.

Until next time..