Luca Fusi Sound Design | Implementation

15May/111

Behind – Post Sound

A quick write-up on a post project I took on with a few friends last term.

We three amigos were feeling pretty confident about our workload for Term 3 when we were approached by a graduating student from VFS' Classical Animation department about doing the sound for her final, the story of a day in the life of a young girl at play, and the magical backpack that protects her from harm throughout.

While we probably would have turned down just any old work to focus instead on our assignments, this animation was completely legit -- there was no way we could pass up a chance to get our sound on such a beautiful piece.

So we said yes, and threw ourselves into it amidst the rest of our coursework.

Going in reverse for this one. Here's the end result, and below that is some thinking/work that went into it.

Final Result

Early Planning

Our first look at "Behind" was a draft copy of the piece with temp music dropped in -- nothing locked yet, but close enough to completion that we got what the animator had in mind. The whole piece had a very Miyazaki-esque/Spirited Away sort of vibe to it, and we wanted that to come through unaffected in the final clip. Our job here was really clearly to support the visual and the mood just enough to elevate the movie, then get the hell out of the way.

Some early thoughts we had:

  • Less is more. Both the delicate visual style of BB's animation and the tone of the piece called for a really light touch from the sound department. We've been used to crazy psychic battles, fight scenes and explosions, so trying to create "soft" sounds was totally new for us. We decided early on to keep the sound effects and ambiences from being "over-real" and leave out a lot of the detail we'd otherwise put in.
  • Cede to the music. It's weird to say this, as sound people are usually jostling with the music guys to get their sounds played loudest in the final mix - but again, with a really delicate piece like this one, we had to concede that music could say a lot more than simple effects could to create the vibe we wanted. We were prepared to have our stuff played down.
  • If we can't get the crying working, we'll drop the dialogue entirely. It was a coin toss as to whether the girl was really vocalizing or not as she skips along, but we thought it'd be a great experience to get in touch with our acting campus and cast talent for the role, get some extra dialogue editing practice, etc. -- so we committed to putting dialogue in the piece, with the caveat: if it wasn't believable, if the crying felt false or got in the way, we'd strip it all out.

In all the student works (in-progress and final) I've seen so far in my time here, I've observed that dialogue is kind of a high-risk proposition. When it's good, or just average, it goes unnoticed and we simply absorb it, freeing up our brains to concentrate on the visuals/rest of the sound design that's gone into a piece. When it's bad, it hijacks our attention and basically spoils any chances of walking away from the piece thinking that it had good sound...

Recording and Editing

... Fortunately, we managed to find a couple of voice actors that fit the character we were looking for, and had one of those "Oh my God"/mutual turn-to-each-other moments after our session with Arielle Tuliao, who did the voice of the girl. That was a huge turning point in the process; after editing her stuff into sync and laying it on with our Foley, everything started feeling like it was really going to work out. Everyone else did a great job as well, with Brendan (one of the kids') voices also providing some neat source material for the bag roar when pitched down.

Apart from the music, the great majority of the sound that got played up in the final mix was our Foley -- all of it was captured with the Rode NTG-3 running through an FP33 mic pre into Pro Tools. Would've loved to have used the 416 on all that stuff, but didn't discover it until a few weeks later!

The sessions took us the better part of two days to fully record and edit. Notably "cool" Foley solutions were flicking our fingers with masking tape on them for the butterfly flaps, ripping pieces of cloth underneath a dirt pit for root tears/plucks, and a nice layer of fresh-picked green onions on our surface for some squeaky meadowy footstep details.

My buddy Gwen handled the BGs and SPFX , both of which I thought really had that soft sonic texture the piece needed, and were rich and full without being really showy.

Manuel took the animator's original Music, a simple piano melody, and really filled it out to score the entire piece with -- unfortunately, a lot of those changes didn't make it into the final mix, in favor of the director's starting piece. You gotta expect it.

Final mixes were done by a later-term student and VFS' Matthew Thomas.

Hope you enjoyed! Leave any questions or feedback in the comments.

24Apr/112

Game Audio – Movement Soundscape

The third term of my time at VFS has finished, which means we're 6 months from reality. It's incredible to think about how much I've learned since starting here last October; moreso to try to imagine all the things I've still got to learn. There's enough for several lifetimes, and seeing the work my classmates, our teachers and the wide world of sound on the Internet puts out every day is super inspiring and challenging. It's time to start looking back through my most recent batch of work...

Here's an end-of-term project for our Game Audio II course:

Game Audio II

Reconstructing Movement in Game Audio:

Environmental Sound

The aim of this project is to record source material, and then to re-construct a simulated environment for the user/listener to establish more than one ambient game audio context. The project should take the user on an auditory journey transitioning the user through more than one ambient ‘zone’, as if the character is moving through the game world on a “spline”.

After hearing all the examples of past students' work we were shown in class, I was pretty impressed by some of the ambient shifts these former all-stars had come up with. Some really cool stuff, like underwater to above-water, nice examples of occlusion through shutting a car door in the face of a horde of zombies, etc.

But while they all had lots of imagination and detail, none of them really felt like they were moving to me so much as the environment was changing over time.

I wanted to do better, here; to get movement across at all costs, even if it meant I had to keep my concept a little more grounded in reality. From jump, I thought that this would be a great chance to learn a bit about binaural recording (the project was going to be played back/evaluated in headphones) and to try some in-the-field source recording techniques, stay away from the very close-mic'd dry recordings we'd normally do in our Foley rooms.

The goal was to keep the environments simple and just let the movement speak for itself. And in that respect, I think I did pretty well.

What is Binaural Recording?

Brief aside on what all this "binaural" stuff means:

Binaural recording is a method of recording sound that uses a special microphone arrangement and is intended for replay using headphones.
-- Wikipedia, "Binaural recording"

Sound waves don't just dive straight into our brains via some kind of "line in" jack: they have to move through air to get there. And as they do that, these waves bounce and reflect off your shoulders and chest, wrap around your skull, move through your hair, filter through your clothing, etc., all the while undergoing subtle changes in frequency, volume, and arrival time (e.g. hit your left ear just before your right ear), to name a few.

Obviously, our hearing receptors don't sit in an open, microphone-shaped capsule in front of us, they're buried deep in our ear canals. So by recording via two tiny, omnidirectional microphones positioned roughly where your ears are, you are going to record the sound not "as it sounds" in some pure, idealized sense, but as it sounds to you.

When played back, all of those subtle frequency/delay/arrival time changes you captured by recording this are interpreted by your brain in a way that makes you instinctively feel as though those objects are coming from where they were when you originally heard them. We don't hear "slight loss of frequencies around the 8,000Hz range because those tiny waveforms were absorbed by my haircut," we hear "behind." As an incorrect but illustrative example.

By recording this way we can capture much more detail than just "left" or "right" - we can get above or below, in front or behind, etc. And you can turn two channels of simple stereo into sounding like a whole 360-degree sphere of surround.

And yes, people build lifelike human mannequin heads, complete with density-matched materials to simulate the hard reflections of our skull vs. the soft, absorptive tissue of our outer ears, embed microphones in them and use these devices to do these sorts of recordings. At the fanatical level.

Raaaaaaaawr

This is a popular binaural recording example that might blow your mind - wear headphones.

There's a lot more to say on this than I can teach, so if you're interested, look around. Our ears are amazing.

Deciding on a Scenario

First, I brainstormed a few scenarios - I wanted something simple and real, with the chance to show off a lot of movement and cool perspective changes. The most interesting to me was that concept of a Metal Gear Solid-style infiltration, with some close spaces, hearing people filtered through a vent shaft, and eventual running/scuffling.

I "prototyped" some of my recording techniques by squirming around in a broken vent we have in our props room and adding a little bit of reverb to the signal, and got some really nice, claustrophobic results. I also affixed two omnidirectional clip mics to a pair of classes, hooked myself into a portable recorder and walked around our campus a bit to see if it'd work as a poor man's binaural setup. The results weren't 100% interesting all the time, but a couple of cool moments convinced me that this was an avenue to chase.

Planning and Recording

Vent decided on, I sketched out the rest of the soundscape and the environments it'd take place on - again, sticking to stuff that was close to campus so that I could use a lot of my own recordings. I had to figure out roughly how much time I wanted the character to spend in each environment, plus making sure that I was hitting the assignment requirements in terms of "signals" (in-your-face, attention-calling sounds like guard footsteps and key jingles), "keynotes" (middle-ground emitting sounds like vending machines, vents, a radio) and "ambiences" (persistent background sounds that give some subtle information about where you are).

With only 90 seconds to work with, I wanted to make sure you knew where you were as fast as possible whenever the environment changes. If you were in a vent, you needed to know within a couple of seconds. So I put a lot of emphasis on making those binaural recordings the "bed" of the piece, cranking them in the final mix. Normally, your ears tune out "room tone" as you hang around an environment, but I thought that if I were too subtle with these recordings, the cool perspectivey effects wouldn't come through.

Some things I recorded this way were all of the different room tones I wanted, the character's footsteps, and the stairwell door opens and closes - this let me put all of that easily within "his" perspective without having to do any extra mixing. I also spent a bunch of time tilting myself around like an idiot outside in our school's courtyard to get some really dizzying panning on the nearby traffic for the fight up on the rooftop. That, with some of the low- and high-end rolled off to give a "rooftop" sense of distance, became the ambience for the final fight on the rooftop.

For the main character's breaths, I used a single lav mic and just panned it directly in the center (when little else was) to give it a bunch of presence. Simple but effective!

All other effects were recorded in mono with the phenomenal Sennheiser 416. I love love love this microphone for what it's good for, and it feels great to finally be discovering favorites. It's extremely directional, which means I was able to walk around campus and point it at the tiniest little emitting spots on machines, coolers and various electronics and pick up their distinctive buzz without capturing too much of the rest of the environmental noise. That plus our Foley rooms' new preamps allowed me to grab some extremely clean recordings of all the other stuff I needed: guard boots, cloth punches, key jingles, mug and paper tosses, etc.

These recordings were also almost totally free of our floor's soul-crushing, inescapable 120Hz hum. As a result, I barely needed to treat these recordings with EQ etc. before I started cutting the final piece. Saved me a ton of time.

Finally, the brilliant guard dialogue was written by yours truly in a lame attempt to explain why these two guards just wouldn't shoot the guy on sight, plus give a little extra depth and a sense of threat to the whole piece.

I wanted to experiment a bit more with having my classmates (the two guards) read their lines a bit farther back from the mic and even "off-axis" (not facing the direction that the mic picks up sound in) to try to have that sense of people moving around and not talking directly towards the main character's ear, but stuck to just recording it normally and positioning it in the mix later on. The one "hey!" in the stairwell was recorded a few floors up and in the stairwell, though.

Editing and Mixing

My major discovery for this part of the project was WaveArt's Panorama plug-in, which I managed to convince our IT department to install a trial of on my machine. It's a 3D panner that uses some of those same binaural recording principles to subtly adjust certain frequencies, volumes and delay times in a recording to let you position objects wherever you want within a stereo sound field. And as an RTAS plug-in, everything in it was totally automatable.. which basically let me "draw" the path of all the objects in my soundscape according to where I thought my main user should be. Tooooooo freaking cool.

The Panorama interface.

The only shortcoming I found with it was that it doesn't respond well to really quick pans/head turns, as that's just too much math too fast for the plug-in to crunch. You end up getting a lot of phasey, washy sounds when you do it, so I had to keep the 3D panning stuff slow and gradual. Small price to pay! Being able to place the guard's voices below and increasingly behind the character as he moved forward was amazing, and I'm especially happy with how the quick front-to-back pans on the office objects (mug of pens, paper) worked in the middle of the chase. That plus the whirl-around as the guards open the door on the roof.

The next big victory in positioning everything in this environment came from messing around with multiple reverbs. When I started, I was using a standard low-pass filter (that "muted" effect) to try to make the guards' voices sound as if they were being blocked by the walls of the vent, but it felt forced. Sending their voices into the same claustrophobic, metallic-sounding reverb I was using for the main character's vent squirming, while keeping their dry voice signal down, worked way better. Favorite examples of verb usage in the soundscape are as the main character pans his head back up and one of the guard's voices seems to "move" into the vent, as well as the way I treated the alarm as the character bursts into the stairwell.

Final Result

Separated into stems because we had to turn them in that way, and it might give you a better look at what's happening in each part of the whole.

Ambiences Only:

Keynotes Only:

Signals Only:

Final Mix:

What Went Right / What I Learned

  • You shouldn't be scared of recording *in* a location when you want something to sound like it happened there. It seems obvious, I know, but up until this project, I really felt like the safest way to do things was to record sounds clean and close in a soundproof room, then control the way everything was positioned/echoing in a mix later on. For an assignment like this, I saved a ton of time by *not* doing this that way, and even managed to create some more realistic results by doing it. I couldn't have made that stairwell running happen as realistically in any other way but just recording myself jogging up them with two tiny mics clamped to my glasses.
  • A little noise isn't a big deal. Seriously. When recording room tones etc. with our tiny omnidirectional microphones, I had to turn the gain way up in order to get anything to record at all. Doing tihs added a ton of hiss and white noise into the recordings - the mics weren't meant to really be turned that high - but in the final mix, once all the ambiences were in and with everything else going on, you couldn't hear that shit at all. If I had spent any time cleaning those recordings up with Xnoise etc. before mixing them, it would have been totally wasted.
  • The Sennheiser 416T is awesome. So is Panorama.

What Went Wrong

  • In retrospect, the whole project took me way longer than it should have due to my own meticulousness - if I had gotten my shit straightened out with recording planning and done that all in one day, things could have happened faster, but because of this term's schedule, all my recording sessions were happening on and off through the end of the term. As usual, "organization will set you free."
  • I wouldn't have done the fall from the vent/ring-out as dramatically, if I could go back and do it again. It's a bit over-the-top and Gears of War-y, but it helps cover up the fact that there's not much going in that section of the piece besides some footstep scuffles. A straight ring with a less dramatic low-pass might've done it.
  • Not totally totally sold on the fall from the roof as well - I wish I had some closer-sounded traffic sources to gradually fade in to give a better sense of proximity. Struggled a lot with this last section!

Please leave any questions/thoughts/criticism in the comments!

5Mar/110

Term 2 Work, Pt. 3 – Train Car Physics Impacts

Intro to Game Audio

Impacts

Second set of physics impacts for our Intro to Game Audio class, same rules as the first - five lights (10"), five mediums (10'), five heavys (100'), five supers (100'+, hurled). Impact surface was concrete once again.

Train Car

Early impacts planning for the train car recording.

Early impacts planning for the train car recording.

The biggest goals with the train were to make it big at all levels (while still giving myself room to go up), and make it feel cohesive – not like a big pile of assorted junk. I really wanted the sounds to make you feel like a giant hunk of metal was moving around, so I thought a lot about timing.

Since I couldn’t record any sounds from an actual train car and nothing I could get from our props room felt “big” enough to get that vibe across, I resigned myself to using the Foley room stuff for high-mid and high detailing, possibly a little bit of color for the low mids if the pitching worked out well. I suspected I’d be using some really large metal groans to give my sound its defining shape, and that premonition was dead-on.

From what I recorded myself, the car door and some sheet pans were the biggest contributors to the final sound. The car door impacts didn’t do much for me, but I got some great stress sounds by standing on the thing and slowly shifting my weight around. When pitched, these gave a ton of character to the longer impacts. The sheet pans were used for high, ear-splitting scrapes across the concrete that I pitched and used later to convey a really stressful, fatiguing sound of this giant car sliding across concrete.

Had a lot of fun with the supers on this one, as well. Listening to some other students’ impacts inspired me to try to take really novel approaches to the intro/stinger of each super.

With more time, I would've given a little more attention to shaping the overall mix - there's a lot of really aggressive, nasty mid-high frequencies in here that made the impacts mixing a physically fatiguing process. I could do to strip some of that down a bit for repeat listening in a game environment. As well, I would've spent a little more time sculpting the low-end of these sounds to make it feel as big as a real train car might. These sounds in particular are definitely best listened to with a good set of speakers or cans.

All are provided below for your enjoyment!

[Light Impacts]

[Medium Impacts]

[Heavy Impacts]

[Super Impacts]

4Mar/114

Term 2 Work, Pt. 2 – Bowling Ball Physics Impacts

Intro to Game Audio

Impacts

Definitely the longest assignment of the term - each student was given a pair of objects (not literally, just assigned in name) to design 20 impact sounds for at different forces. Five lights (10"), five mediums (10'), five heavys (100'), five supers (100'+, hurled). Impact surface was concrete, with some flexibility here. We had a few weeks for this one and were encouraged to record as much original source material from our Foley/props room as possible.

Much of the below taken from my project post-mortem. Enjoy!

Bowling Ball

The first thing I did when you assigned this to us was to start conceptualizing all the different layers that’d be involved in each of my objects’ impacts, so I had a checklist to work from when it came time to record.

Early impacts planning for the bowling ball recording.

Early impacts planning for the bowling ball recording.

Approach here was to really imagine what was going to happen to the surface itself at the higher impacts, since the ball itself is a pretty one-dimensional sound source. Other than that, I just knew I’d want to record the ball dropping on multiple surfaces so that I could stack and vary the balance of those impacts to fatten up or thin the ball drop sound.

[Light Impacts]

The biggest challenge was trying to get the really impactful, transient crack I wanted for the higher impacts (as the ball would literally split the concrete), while still keeping the hollow thud of the ball sound there as well. I mixed the three different surface drops with some short LFE accents gave me the roundness of the ball, and then tried to use some snappy sweeteners like walnut cracks to give the impression of the concrete breaking. I couldn't find anything nicer than this in our library to fit the task; with a little more time I might've gotten luckier.

[Medium Impacts]

I focused on making sure that the listener was able to tell that this was, in fact, a bowling ball – and not just some solid heavy object – at all velocities. Structuring each fall as a series of bounces helped this a bunch, as did the tail-end rolls I had recorded.

[Heavy Impacts]

I started work on this guy before the train car to try to get my chops up before moving onto the more complicated sounds. Unfortunately, this meant that I was designing these without the aid of a ton of the tricks we'd later learn in Intro to Post Audio Editing, so making variations was slow.

[Super Impacts]

With some extra time, I would've put more effort into varying up the supers - I left off after the first one to get onto to other projects, and by the time I returned I was totally sapped for inspiration, so they're not *quite* at the imaginative level of my train car supers.

My other object was a train car - I'll have those recordings and a little retrospective up in a few days.

2Mar/114

Term 2 Work, Pt. 1

Intro to Post Audio Editing

BGs, SFX and SPFX

Without a doubt, this course was the term's heaviest hitter (and most prolific source of assignments), running us through the basics of selecting sound for an editing BGs (backgrounds, uninterrupted sounds that flow throughout an entire scene, like birds and wind), SFX (on-screen sounds like a rushing waterfall, a car crash, a heavily layered punch) and SPFX (perceptual or mood setting sounds, otherworldly stuff without a distinct source, sweeteners for dramatic moments).

Opening statement: It's been a long time since I was in school, and it's been longer since I've known a really life-changing teacher, but our instructor for this class has almost single-handedly inspired our class through this term. Really dig his work ethic and emphasis on organization, and it's clear that he loves what he's doing. So, a look at what I managed to produce while under his wing for a couple of weeks.

Son of the Dragon - Fight Scene 1 - SFX

Day one SFX exercise, pretty standard stuff - lots of whooshes and several layers to cover lows, mids and highs for the different hits. You might think that sync is a big issue with fight scene SFX design, but it was the easiest part of the process for me. If I could do this again, I'd get more highs in there and make the table drop a bit bigger.

I didn't do enough EQing to scoop out the bassy frequencies that were going to stack up here, so the final product lacks a lot of punch that would be there with better mixing. The compression on this really dulls it out; early lesson learned.

Son of the Dragon - Arrow Fly - SFX

Second SFX exercise of the term, tough because of the toolset we were still restricted to at this point. Key points we needed to hit were 1) the sense of some mechanism firing the projectiles from far off and 2) good, synced audio cuts between the arrow approaches (rising pitches) and bys (doppler-like effects as they crossed the camera). Used some pitched up F1 car bys for the arrow whizzes, re-used some of my favorites sweeteners and impact sounds from the previous SFX exercise, and slowly raised one resonant part of the EQ on a few wind tracks to give the whistling sense of the arrow getting close. Far-off catapult/arrow launcher guy deal was constructed from a couple of pieces like clanking metal, rattling chains, wood groans and arrow twangs.

Son of the Dragon - Fight Scene 1 - SPFX

Day one SPFX exercise, strung together quickly as king of a kung-fu stinger from some reverse cymbals, piano string scrapes and a rattlesnake tail. Ran a few elements through a lo-fi filter to try to give it that 70s sound. Neat idea, maybe a bit too musical, and definitely more towards the subtle side of SPFX design. The temptation to go big all the time is definitely there - especially since that's our teacher-mentor's style - but I like going towards the less impactful side of things with SPFX design and letting the punches, kicks and sword shings handle the big sounds. At least for this style of film. We'll see if my tastes change.

Born to Raise Hell - SPFX Moment #1

Early SPFX exercise, we were to choose a couple of visual effect-y moments from a clip and throw some abstract SPFX over them. Short and sweet for this one, made from a scream and a couple of resonant metal impacts.

Born to Raise Hell - SPFX Moment #2

Layered here - several breaths (pitched up and down), metal impacts, a bowed cymbal, some repurposed metal shing noises and low rumble (source unknown). Was pretty happy with the way this one turned out.

Son of the Dragon - Market Chase and Fight Scene - BG, SFX and SPFX

Final assignment for the course, three rolled into one - we were responsible for all audio here, with the BGs running most constantly from front to back. Anything missing SFX is likely because it was out of the bounds of the assignment. I mixed these together (they were submitted independently) after the fact to see if they'd all mesh up; usually, you won't have the same person responsible for BG, SFX and SPFX design, but I wanted to see if having one person (me) in all those three roles meant that I'd leave myself some good holes in one mix to fill up with another, maintain cohesiveness of sound sources, etc.

Tons of layers here, with the SFX being the most complicated and the session a total nightmare to get under control when it came time to bounce. This is the first project we'd been able to use elastic audio (dramatic pitching/time-stretching tool) on, and I used it to create the slowdowns and ramp-ups during the water impacts. Tried to differentiate the henchmen and the main character with different tonalities for their whooshes using some hissy sounds for the bad guys and low, raw animal sounds for the good guy's incoming swings.

With the SPFX, tried to keep these as kung-fu themed as possible again by sticking to Eastern and natural sound sources instead of going all the way to The Matrix side of things. I wanted to create the sense of the main character channeling some mystical power source for his moves, hence the spiritual chants. I think they went a little towards the subtle side again, but I'm good with that! SPFX for the table jump-up, slow-mo jump kicks and flying scenes were done with: reversed and crazy echoed Buddhist chants, a lion, dove flutters, monkey chittering, a 909 chap, a bullwhip, finger cymbals, reversed gongs, rattlesnakes and some sine wave rings.

 

Intro to Sound Designing

SPFX

TC Drug Sequence - SPFX

( Vimeo screwed up with this one and appended about 28 seconds of still video to the front of this clip. Skip to 0:28 for the start of the sequence. Sorry! )

Final assignment for our (too short!) Intro to Sound Designing class. I had a vision for this one, and it came together really quickly - very happy with the end result. Recorded several layers of breaths, whispers, gasps and crazy mouth noises, chopped them up and stretched them a bit with elastic audio and went very heavy on the echoes, delays and panning to create this chorus of voices in the guy's head. The siren was elastic audio'd a bit as well to get the pitch wind-down in there and has a chorus effect laid on it. Synth undertone is from an Absynth patch, tuned to fit the 909 bass kick hits at the beginning.

Other sounds include a manipulated lighter, pill bottle, heartbeat and some broken glass. I found myself thinking a lot about frequency content ahead of time on this one, why is probably why the mix came out so balanced.

In the next post, I'll go through my Game Audio impacts and give a quick look at my temp mix for a short section of a student film we had for our Mixing II course.

**All of these were just roughly mixed/mastered before I threw them up - any out of whack levels etc. are intentional in that that's where I am right now. We can watch my mixing ear improve as the year goes on!