Luca Fusi Sound Design | Implementation

21Jan/124

VFS Sound Design Program Review

I was approached a few weeks ago by one of the team from ArtSchoolReviews.ca, who asked me if I'd write up a review of VFS' Sound Design for Visual Media program for their site -- which is about to undergo some dramatic design changes.

As of a couple of days ago, that review is online, and you can read it here:

The Hardest Year - Sound Design for Visual Media at Vancouver Film School

I was originally planning to host a review on this site as well, but as their formatting is just so much nicer, I don't think there's a need for right now.

Many of you have written to me with questions about VFS over the past year or so and I've been happy to help. And while I hope this review serves as a good starting point for a few new generations of prospective VFS SD students, you can still e-mail me directly if there's anything you'd like to know.

28Oct/111

Final Project and Update

Last week, I graduated from VFS' Sound Design for Visual Media program, to an audience of colleagues, parents, instructors and friends. What a ride it's been.

A year ago, that night before I left New York, it felt like I was throwing everything away on a total gamble. I was moving away from friends I'd had for years, my family, my contacts and what have no doubt been some incredible times with Wanderfly to move a coast I'd never known and drown myself in student debt. All to chase the dream, to shut up that little voice that kept telling me I needed to make a change. It's now a year later, and I'm really glad I did. Not in some blanket, everything-has-been-amazing way, but in a very flawed, wonderfully imperfect kinda way. I've changed. I've learned things about myself I would never have known without taking that leap. My eyes and ears have been opened thanks to the wisdom of some incredible teachers. And that far-flung bunch of sound junkies called SD49 have been there the whole time, going from classmates to confidants to lifelong friends.

This was the jist (I think) of my graduation speech as class rep, and while I was worried it all waxed a little too emotional at times, it went over well and it's how I truly feel. It's been an amazing ride, but I can't wait for what comes next.

I owe this place a proper write-up about my time at VFS and things I'd do differently - ex-producer can't help but post-mortem - but for now, a quick look at my final project:

Deus Ex - Human Revolution "Revenge" Trailer Redesign

Everything you're hearing in there, I had a hand in, and as a major Deus Ex fan, it was a total labor of love. I really have to thank Square-Enix and EIDOS Montreal for their permission.

I'll talk a little more about the sounds that went into that trailer when I get some spare time. In the meantime, a bunch of samples of my work have moved to this Work Examples page.

November is film work, job applications and all the sound experimentation I can find time for.

22Aug/110

Nearly There

Months and months later, here we are. I'm nearing graduation from VFS Sound Design for Digital Media program and will tidying this place up quite a bit. It's been a whirlwind year with a lot of lessons learned, so in addition to throwing up all of my latest coursework (at least some of which has to be good enough to show around) I'll have a few closing thoughts on the program. In the meantime, things will likely change and move around. Don't worry, it'll all be back in order before October rolls through.

Cheers to all the prospective students who have found my blog while searching for information on the Sound Design for Visual Media program, you can always contact me via e-mail if you have any in-depth questions.

And a big thanks to everyone who's been reading!

Luca

Tagged as: , No Comments
15May/111

Behind – Post Sound

A quick write-up on a post project I took on with a few friends last term.

We three amigos were feeling pretty confident about our workload for Term 3 when we were approached by a graduating student from VFS' Classical Animation department about doing the sound for her final, the story of a day in the life of a young girl at play, and the magical backpack that protects her from harm throughout.

While we probably would have turned down just any old work to focus instead on our assignments, this animation was completely legit -- there was no way we could pass up a chance to get our sound on such a beautiful piece.

So we said yes, and threw ourselves into it amidst the rest of our coursework.

Going in reverse for this one. Here's the end result, and below that is some thinking/work that went into it.

Final Result

Early Planning

Our first look at "Behind" was a draft copy of the piece with temp music dropped in -- nothing locked yet, but close enough to completion that we got what the animator had in mind. The whole piece had a very Miyazaki-esque/Spirited Away sort of vibe to it, and we wanted that to come through unaffected in the final clip. Our job here was really clearly to support the visual and the mood just enough to elevate the movie, then get the hell out of the way.

Some early thoughts we had:

  • Less is more. Both the delicate visual style of BB's animation and the tone of the piece called for a really light touch from the sound department. We've been used to crazy psychic battles, fight scenes and explosions, so trying to create "soft" sounds was totally new for us. We decided early on to keep the sound effects and ambiences from being "over-real" and leave out a lot of the detail we'd otherwise put in.
  • Cede to the music. It's weird to say this, as sound people are usually jostling with the music guys to get their sounds played loudest in the final mix - but again, with a really delicate piece like this one, we had to concede that music could say a lot more than simple effects could to create the vibe we wanted. We were prepared to have our stuff played down.
  • If we can't get the crying working, we'll drop the dialogue entirely. It was a coin toss as to whether the girl was really vocalizing or not as she skips along, but we thought it'd be a great experience to get in touch with our acting campus and cast talent for the role, get some extra dialogue editing practice, etc. -- so we committed to putting dialogue in the piece, with the caveat: if it wasn't believable, if the crying felt false or got in the way, we'd strip it all out.

In all the student works (in-progress and final) I've seen so far in my time here, I've observed that dialogue is kind of a high-risk proposition. When it's good, or just average, it goes unnoticed and we simply absorb it, freeing up our brains to concentrate on the visuals/rest of the sound design that's gone into a piece. When it's bad, it hijacks our attention and basically spoils any chances of walking away from the piece thinking that it had good sound...

Recording and Editing

... Fortunately, we managed to find a couple of voice actors that fit the character we were looking for, and had one of those "Oh my God"/mutual turn-to-each-other moments after our session with Arielle Tuliao, who did the voice of the girl. That was a huge turning point in the process; after editing her stuff into sync and laying it on with our Foley, everything started feeling like it was really going to work out. Everyone else did a great job as well, with Brendan (one of the kids') voices also providing some neat source material for the bag roar when pitched down.

Apart from the music, the great majority of the sound that got played up in the final mix was our Foley -- all of it was captured with the Rode NTG-3 running through an FP33 mic pre into Pro Tools. Would've loved to have used the 416 on all that stuff, but didn't discover it until a few weeks later!

The sessions took us the better part of two days to fully record and edit. Notably "cool" Foley solutions were flicking our fingers with masking tape on them for the butterfly flaps, ripping pieces of cloth underneath a dirt pit for root tears/plucks, and a nice layer of fresh-picked green onions on our surface for some squeaky meadowy footstep details.

My buddy Gwen handled the BGs and SPFX , both of which I thought really had that soft sonic texture the piece needed, and were rich and full without being really showy.

Manuel took the animator's original Music, a simple piano melody, and really filled it out to score the entire piece with -- unfortunately, a lot of those changes didn't make it into the final mix, in favor of the director's starting piece. You gotta expect it.

Final mixes were done by a later-term student and VFS' Matthew Thomas.

Hope you enjoyed! Leave any questions or feedback in the comments.

3May/112

Say Hello to Kyma

The Kyma timeline environment. Not mine!

The Kyma timeline environment.

In balancing out such a crazy term, a few things were going to have to suffer. I present to you:

Sound Designing 2

Kyma Project - Scanner Duel

Part A
Using a microphone, record a vocal temp into protools to sketch out the sound design for the given quicktime (Scanners duel scene). Use as many tracks as is necessary and any plugin effects or audiosuite transformations that will make your design fit the scene.
Part B
Using any of the processes available in Kyma and any appropriate sound sources, create a sound design for the same scene, using the vocal temp that you created in part A as a template or blueprint.

Kyma Kyma Kyma. Such a wonderful and alluring word! I knew next to nothing about what this thing was or what it did, just that it was a device we had somewhere, buried in one of our back rooms, and that with the right level of arcane mastery, it could be used to create these incredibly techy, otherworldly textures and sounds that you couldn't get anywhere else. I knew we'd spent almost no real class time on it, and that however much each student chose to master the Kyma in the time given to him was solely on him. I was ready.

What is a "Kyma?"

This should start things off:

Kyma is a visual programming language for sound design used by musicians, researchers, and sound designers. In Kyma, a user programs a multiprocessor DSP by graphically connecting modules on the screen of a Macintosh or Windows computer.
-- Wikipedia, "Kyma (sound design language)"

Essentially, the Kyma is a very expensive, very powerful and -- in my limited experience -- very niche sound design tool. It is two parts:

  1. a sort of programming environment where one can set up large processing chains with which to mutilate your choice of source sounds, from loaded-in samples to stuff you synthesize yourself
  2. and the external Paca Rana box that crunches all the 1s and 0s that execute that processing chain. The box is essentially a tiny computer whose every cycle is devoted to rendering sound and nothing else.

All of the components within the Paca itself are of really high quality, and that dedicated processing power means that no matter how much you want to stretch out, granulate and murder that sound, it's still rendered very smoothly and naturally - and without messing with everything else your system has to do on its own, like run Pro Tools and whatever other plug-ins you have set up. It seems like it'd play very well in even crowded sessions.

Not as cute as the real thing.

Not as cute as the real thing.

The Kyma environment as a programming language is custom-designed for sound and sound alone, and genius founder Carla Scaletti has been at it for decades. It teases you in with seemingly infinite possibilities, a playful, hippie-ish sort of branding message and some very cool credits to its name.

If you've seen WALL-E, you've heard Ben Burtt's work on the Kyma in the title robot's adorably lilting voice. Video below, jump to a little past the 6min mark to see some of what the system's capable of in terms of voice construction.

That box, unfortunately, will also run you upwards of $3000 or $4000. And while getting started in that programming environment and making some very out-there sounds with it is pretty quick, it's not something you can jump into and use to go from a pre-planned Point A to Point B. Especially not in the one day timeframe I had left myself for this last assignment of the term.

Start to Finish in a Day

Fuck.

Fuck.

As I said, I was ready to kill this one. I'd heard what this box could do in the work of our badass late-term student, Javier, and I was ready to bend this thing to my will. But the film collaboration happened, the soundscape happened, the days rolled by without me learning much more than how to get the Kyma up and running.

Finally, I found myself at the end of the term, one day between myself and the deadline, and without a plan. Always a great place to be; I love working under pressure!

If you've been reading me, you know I can't work without a little planning. Here's what went through my head before I dove in on this one:

I made the decision to phone in the vocal temp. Yep, it was part of the overall assignment and grade, but I had missed the day's class we had been given to record and clean it up (ironically, because I had been up late learning the Kyma the night before and slept in). With 24 hours to go, it didn't make sense to spend the little time I had left polishing up the blueprint instead of the final actual piece. For better or worse, this needed to spring straight from my head to canvas in one go. Because of this, you're not hearing my vocal temp. At least without buying me a drink.

I watched the scene. Many, many times. Sounds obvious, but if you don't digest the hell out of the clip, even when you're in a rush, you're going to miss the visual cues and anchors you need to hit to create a really impactful sound design. This scene needed a lot of review here because of its extreme length and really slow pacing; there are no bullets fired, no punches or broken panes of glass or anything like that. A lot of it is just two dudes staring intensely at one another. I needed to find the moments where the stare intensified and hit them hard.

After watching and rewatching, and staying deliberately in the dark about the overall arc of the real movie, I decided:

  • This battle needed to feel dangerous, the wounds terrifying. I'd never seen Scanners, didn't know what led up to the point of these two guys clashing, but any time you're facing down a foe so strong that he can get into your head and make you tear your own face off, the audience should fucking fear a guy like that.
  • The psychic battle component should be synth-heavy, push myself a bit into the realm of synthetic sound design, which I (still) know almost nothing about. All my experimenting was getting done on this front, and I was counting on the Kyma to give me some great, unique source material to work with that sounded like nothing I'd ever heard but still resembled an epic battle of the minds.
  • Each character's powers needed to have a distinct voice, so that you instinctively felt when the battle was turning one way or another. Of course, the bad guy's should sound more powerful, a little darker, a little more focused; the good guy's more benevolent, but full enough to take him down.

I'd leave the recorder running and just hope I got something randomly brilliant out of the Kyma. Sad times, but I didn't have the luxury of setting up an elaborate custom processing chain to get myself exactly the sound I needed with so little time left. I ran what I thought was some cool source material (ice cracks, resonant metals) through some intro-level Kyma patches until I stumbled upon something workable, bounced it, then edited from there. The Kyma's contribution ended up being mostly in the form of its really nice granulation functions, which I used (and cranked up) on a slowed down oxygen mask sample that I kept running throughout the scene as a bed for the whole encounter. You'll also hear some of these effects on the wounds the bad guy sustains early on. Though I can't remember why.

In the end, though, nearly all the sounds for this scene came from the Massive synth and a few tweaked presets. I found a few representative synths (bad guy powers, good guy powers, general sub bass, general "scanner duel" atmosphere) and ran through the scene a few times and made multiple passes, scoring the individual layers "live" with my fingers on a bunch of parameter knobs. Recorded it to MIDI, bounced it out to audio, left it to edit the next morning. This got me a lot of good stuff that was already pretty in sync with the pace of the scene, which made editing it pretty easily.

Final Result

That's about all there is to say on this one - editing took it the rest of the way. Some of the plug-ins used here were WaveArts' Panorama for stereo movement, ring modulation, the excellent Waves Doubler and MaxxBass plug-ins, along with a suite of reverbs, eqs and delays. If you have any specific questions, leave them in the comments and I'd be happy to answer!

What Went Right / What I Learned

  • Voice distinction. I feel like I nailed down a sort of personality for each character based on the sound of their psychic attacks; that evil boy sounds evil, and the good guy sounds a little purer.
  • Pain! Was pleased with some of the burning/searing accents on the bad guy's points of focus. This was done mostly with some drastic volume automation on the sizzles combined with pitch shifts/etc. on the sound of his powers right when his expression changed.

What Went Wrong

  • Frequency content. Working with synth presets can be tricky, because they bring a ton of broadband noise/sound with them and sometimes fill up a lot more of the sound spectrum than you want. I didn't have a lot of time to mix this assignment before submission, which means that there are some points where that synth noise really, really builds up and gets harsh. Maybe that serves the scene, but I think it'd need to be tamed a bit before this could go into the mix with BGs, dialogue, sound effects, etc. It's a bit of a spectrum hog right now.
  • Not enough time to really use the Kyma. That device is capable of much, much more than I give it credit for in this assignment, and with a little more time I would have liked to make it do what I wanted, instead of just hitching a ride on the stuff it accidentally did.
  • Injury Foley. I had the brilliant idea to use a pan full of semi-congealed oatmeal and rake my hand across it to simulate the sounds of good guy pulling his face off - but this came a week after I'd finished the project. That sound could have used some love.
  • Flat Fire. With a little more time, I would've stylized the fire sounds a bit more - they were left pretty dry as is, and could have fit better with the texture of the rest of the mix.

The Sound Design community online seems to agree that the Kyma is a very powerful tool -- maybe the best -- for very specific tasks like granulation and cool vocal morphing, but that if you don't have a lifetime to spend mastering it, there are quicker ways to get to the sound you want. I also don't really see anything that it can do that Max/MSP couldn't do better, and for thousands of dollars less, if you're willing to sink time into mastering a sound design-oriented programming language one way or the other.

That all said, looking forward to spending some quality time with the Paca as the year rolls on, and maybe, finally getting it to do what I plan it to (and not simply something cool but unexpected) by the end of the year. I may never see one again, right?

Please leave any questions/thoughts/criticism in the comments!

26Apr/113

An Introduction to Film Sound

I love you.

They're always switching it around, but for our class, Term 3 was our first time getting our hands dirty with production sound and the challenges of recording on a film set. Over the last month, we all spent around 8 12-hour days collaborating with students from the school's Film Production program, trying to make short movies - WITH NO ADULT SUPERVISION. Woo!

It was the first time we'd really been out from behind the editing desk in several months, and a super eye-opening experience. So much so that I'm no longer so sure of where I want to end up, audio-wise. The whole world of sound is just so damn awesome! I thought I'd give a little rundown on some of the things I learned from this experience, plus some background info on how sound for movies happens (as I understand it so far).

Fair warning: the more you learn, the more you'll want to pay attention to it at the expense of, y'know, actually enjoying the film.

How Sound Gets in Your Movies

A beautiful American morning.

Let's take a fairly typical, yet morbid film scene:

INT. HOUSE - MORNING

GEORGE is heading out the door for work. He picks up his briefcase, turns the knob and opens the front door, walking out into the sunshine.

EXT. YARD - CONTINUOUS

BIRDS are chirping. Here he encounters his neighbor, MRS. DANIELS, trimming the hedges.

MRS. DANIELS
Morning, George!

GEORGE gets into his car, throwing the briefcase onto the front seat, and drives off.

INT. CAR - MORNING

GEORGE is driving along LISTENING TO THE RADIO. He takes his eyes off the road to adjust his part in the mirror, and suddenly hears a loud HONK. He looks down to see a car heading straight at him.

CUT TO:

FLASHBACK

GEORGE's entire life flashes in front of his eyes.

BACK TO:

GEORGE CRASHES into the car in front of him and dies.

You can pretty easily imagine how these scenes will sound because, besides the effects that'd accompany his flashback and the life-ending car crash, it's full of things you experience in everyday life. As a result, you probably never stopped to think about how they got into the film.

Before I started into this program - or way back, before I knew I wanted to do it - I wasn't thinking about movie sound much at all. If I did, I probably I still naively thought that maybe the actors were just wearing little microphones that captured everything that happened, and maybe there was a guy at a big mixer with a lot of faders who made sure everything was at max volume before they finally shipped the film.

What Really Happens

The "existence of tiny microphones" and "guy at a big mixer with a lot of faders" have some basis in reality, but the rest of what I knew was completely off. Looking at that scene again from a "where did the sounds come from?" perspective, it might go something like this:

INT. HOUSE - MORNING

GEORGE is heading out the door for work. He picks up his briefcase, turns the knob and opens the front door, walking out into the sunshine.

Footsteps
George's footsteps (the ones you're hearing, anyways) were recorded after the film was shot on a Foley stage by a very coordinated guy in a pair of near-matching shoes on a near-matching surface. This would've been done by a team of at least two people: a Foley Recordist there pushing the buttons to make sure all the audio was captured while the Foley Artist walked the scene out.

Afterwards, these footsteps plus everything else would've been sent to the Foley Editor, who would clean up those recordings (edit out clicks and pops, put the best takes in place) before submitting to the Mixer (guy at the faders), that places it in with the rest of the film's sound and decides how loud he wants it to play.

Hand Touches
The sound of George's hands clasping around his briefcase and his touch on the doorknob would be Foley'd as well, using either a real briefcase or a prop that sounds like one. Same process.

Cloth
The swishing cloth of George bending down to pick up the briefcase? Also Foley, done like this:

Working hard!

The Door
The door opens, creaks and closes (every door in Hollywood must squeak like the most unique door in the world) etc. were probably selected by an SFX Editor, who could've pulled the sound of any random door from his personal library or one of the many professional libraries available. He trimmed the length of this door sound effect to match exactly what the door was doing in the scene itself (or just tried to get close, depending on his schedule) and eventually sent this material along to the Mixer.

EXT. YARD - CONTINUOUS

BIRDS are chirping. Here he encounters his neighbor, MRS. DANIELS, trimming the hedges.

MRS. DANIELS
Morning, George!

GEORGE gets into his car, throwing the trunk onto the front seat, and drives off.

The Birds
Sad news, these birds weren't real. I mean, they were real at one point, but for this scene they were selected from a library by a BG Editor, who is responsible for selecting all the ambient sounds that tell you where the scene is happening. The wind noises, the distant traffic - he picked them too. If there was an off-screen clock or a fridge hum in George's house, he would've been responsible for all that. When the BG Editor is done with all his stuff, he sends it along to the Mixer again.

Mrs. Daniels
Mrs. Daniel's Oscar-winning dialogue was, actually, probably recorded right there when they filmed the scene the first time, maybe even by one of those tiny microphones. Score!...

...Unless there was a plane passing overhead every time they tried to record it (common), or some crazy hobo yelling in the background, or a loud car passing by at the worst moment, or her performance just plain sucked. In these cases, Mrs. Daniels' actress would be called in to record ADR and redo her lines in a studio somewhere after everything was shot. She would be coached through this recording session by an ADR Supervisor while an ADR Recordist pushed buttons in the booth to capture every take, as Mrs. Daniels watched herself mouth the line on screen and tried to deliver it in sync, with the same emotion she did in the original scene.

This line, and any other ADR that needed to be done, would be sent along to a Dialogue Editor, who would choose a best performance from Mrs. Daniels' actress (or combine several), trim the little clicks, pops and mouth noises out of her speech, and subtly time-stretch or compress that audio to match her lip movements perfectly.

Ah, but Mrs. Daniel's line now sounds like it's been recorded in a vacuum, and is totally unbelievable when played sitting by itself in that scene. So the Dialogue Editor's got to take some isolated bunch of ambient noise (more like "ambient quiet") and make it into "fill," which doesn't sound like anything but a mash of what the audio on set that day was like. And that's the point. He puts this bed of "fill" under the re-recorded line from Mrs. Daniels so that it sounds more like it was recorded on set, and when he's finished, he'll send everything along to the Mixer again.

"It's digital, right? We can just take that sound out?"

Everything Else

Foley: George's outdoor footsteps, the hand touch on the door, the sound of him tossing his briefcase onto a car seat, the sound of him sliding his suit against the seat and maybe even his seatbelt buckling. Mrs. Daniel's hedge trimming snips would get recreated either in Foley or selected/edited from a library by the SFX Editor - or possibly done by both parties, leaving the Mixer to choose which version he liked the best.

SFX: the door close, for one. That same SFX Editor would also have to take some combination of a car starting, gear shift, engine reverse, tire crunches, engine accelerations and pull-away sounds from his library of hundreds or more and mangle them into sync with the picture. Easy enough for a car pulling out of a driveway, but imagine being the Sound Supervisor on The Fast and the Furious.

INT. CAR - MORNING

GEORGE is driving along LISTENING TO THE RADIO. He takes his eyes off the road to adjust his part in the mirror, and suddenly hears a loud HONK. He looks down to see a car heading straight at him.

CUT TO:

FLASHBACK

GEORGE's entire life flashes in front of his eyes.

BACK TO:

GEORGE CRASHES into the car in front of him and dies.

The slight hum of the AC in George's car and the occluded sounds of traffic outside are work for the BG Editor.

The sounds of George's car revving up, slowing down and driving are from the SFX Editor.

George's clothing swishes and seat creaks as he shifts around, his touches on the steering wheel and mirror, his hand running through his hair - all Foley stuff.

Music
But the music on the radio? That didn't just get thrown in there because it was the director's favorite song. The director might've had a say, for sure, but more than likely a Music Supervisor, with knowledge of the full film, thought hard about what should be playing in this critical moment when George dies. When he had figured all that out, he had to negotiate with the artist or label to get the rights to use that song in the capacity of the film and broker that deal. Finally, when the music has been cleared, a Music Editor will take the piece and select only the part of the song he wants to play in the car scene, trimming the audio to fit.

He will send it to the Mixer, who might add that "futz" effect (the tinny sound of something playing through a radio) to make it sound like that flawlessly produced track is coming out of the dude's little car system.

The Flashback
Suddenly, there's a loud honk chosen by the SFX Editor (honk = incoming car crash for the audience) and the screen flashes white and we find ourselves in George's flashback.

What was that sound that played when the screen when white? And where did this low, eerie rumble during the flashback sequence come from? Those were designed by the SPFX Editor, the guy who typically makes the sounds that don't really exist.

He might have reversed the sound of a cymbal, sped up a few whale calls, slowed down some of his own recorded breaths and done a whole bunch of other magic to arrive at these sounds. He can do whatever he wants and most of the rules don't apply to him, because we don't know what "having a flashback" sounds like. I mean, we do, from other movies, but there is no real-world flashback we can go listen to out there and compare, so he gets to personalize things a bit. When he is done, his sounds go to the Mixer as well.

If you noticed, all the voices and sounds in George's flashback were probably all washed-out and echoey sounding. The Mixer did that.

The Crash
And the car crash itself? Probably a combination of the SFX and SPFX Editors, especially if the crash happened in slow-mo. If George screams, the they might've recorded that line in ADR, and the Dialogue Editor will send a cleaned-up version of it to the mixer, or the SPFX Editor might just take a random scream from the library and mess with it until it sounds distressed and awesome.

Our Film Collaboration

With all the sound that comes after the film's been shot, what the hell were we even showing up to record?

Ostensibly, just the dialogue, even though we still need to roll the recorder during scenes where no one talks.

The first day or so we recorded, we had a few scenes like this, where we'd record a minute's worth of audio with only 5 seconds of "stuff that makes sound" happening. It was a little distressing, but as the days wore on, we began to feel a certain pride in even capturing those non-moments in the best detail possible.

I think that's one of the "things" about production audio, to try to record everything as if your microphone was the first and last stop for sound in the entire film. Sure, you really just need to try to get the dialogue, but why not take some pride in it?

The film is probably several months away from post-production and all those extra sounds being added in, and way before then, some producer somewhere is going to want to see progress on the film. And that progress will have a soundtrack: yours.

It's basically a two-part process, creating that soundtrack.

Booming

Booming is holding that giant black pole with the fuzzy blimp at the end of it, ever-so-slightly off screen, and maneuvering this into position to capture the best quality sound you can. Before we started this, I thought it'd be pure grunt work, but now I'm starting to see that it's a real art form for the guys who make a living out of it.

You don't just point the thing and stand in one spot until they turn the cameras off. What if the actors are moving around, turning their mouths? Standing up into sitting down? If you aren't constantly tracking the actors' mouths with your mic, a lot of that dialogue will end up getting up recorded "off-axis" (e.g. hitting the microphone from a less than ideal pickup angle), and those lines will lose a lot of body and presence in playback. They will sound "thin." That can be patched up, to an extent, but if it can't, you may be calling those actors in for ADR later. And Mrs. Daniels' rate to fly down to LA and re-record dialogue in a studio is not cheap.

So you move the mic around a lot to follow the actors' mouths (hope you've been studying the script!). But now the director wants to try another shot, start with a wide angle and then come in close, maybe pan around a bit. You need to know what that camera will and won't see at all times and have the good sense to stay out of it - tricky with a lot of motion. Don't forget that your boom pole's probably got a big mess of cables drooping down from it, as well; don't forget that those cables make noise when they clack against it. You'll probably have to find a new position to stand in. You might even have to climb something.

Mixing

Mixing is holding the recorder (or another few pieces of related gear) shown at the top of this post, pressing record when the take starts and twisting volume knobs to make sure everything is coming in at a good volume. That's the simple version. If it's a nice, quiet indoor shot with little movement, just you and one guy with a boom pole, it can be a pretty relaxing affair.

Now you're doing a slightly more involved shot, and you've got the boom operator's signal plus the incoming feeds of two tiny clip mics to balance against each other, it gets a little more involved. Now you're doing that shot on a dock in the pouring rain** and have to worry about the mics getting wet, sputtering and fuzzing out and ruining the entire take - even the perfectly sound boom track - if you don't turn them down fast enough.

You're watching battery levels, always.

The actors go from talking to shouting to pushing each other, thumping one another on the chest right next to your delicate little clip mic, and you have to ride that volume knob the entire way to keep the signal from clipping and becoming distorted.

And if anything in that take doesn't go right for sound, you need to speak up and get it re-shot or let the director know (easy when the director loves the sound crew and you're ahead of schedule; probably not so easy on a multimillion dollar film), or it's your butt on the line when that sub-par sound is playing for the producer in LA who's reviewing the day's takes.

**That rainy scene happened for us, and it was trying as hell. None of us ever want to go back to Cates Park, North Vancouver, ever again.

Final Thoughts

I'm not sure who works harder - production sound guys or editors - and I think it's just safest to say that capturing and creating great sound is equally tough, that you'd have to be equally insane to want to do either one for a living. I can say that I have way more respect for the production sound types after this experience, though.

Film sound guys wake up early, rush around, wait, have to deal with team dynamics, endure shitty weather, stay on their feet all day - but they get paid for it, the weather is sometimes gloriously sunny, and it's sure as hell fun. By the end of our collaboration, our little sound crew was pretty tight with those film students. We had a blast.

Now that I'm back in front of a glowing rectangle and cutting sounds in Pro Tools all day. It doesn't rain indoors, but I can't say which side of the process I enjoy more. I was hoping to rule things out with this program, make my decisions about the future a little simpler, but the doors won't stop opening.

Fingers crossed to land some great work on either end this program's finally out.

24Apr/112

Game Audio – Movement Soundscape

The third term of my time at VFS has finished, which means we're 6 months from reality. It's incredible to think about how much I've learned since starting here last October; moreso to try to imagine all the things I've still got to learn. There's enough for several lifetimes, and seeing the work my classmates, our teachers and the wide world of sound on the Internet puts out every day is super inspiring and challenging. It's time to start looking back through my most recent batch of work...

Here's an end-of-term project for our Game Audio II course:

Game Audio II

Reconstructing Movement in Game Audio:

Environmental Sound

The aim of this project is to record source material, and then to re-construct a simulated environment for the user/listener to establish more than one ambient game audio context. The project should take the user on an auditory journey transitioning the user through more than one ambient ‘zone’, as if the character is moving through the game world on a “spline”.

After hearing all the examples of past students' work we were shown in class, I was pretty impressed by some of the ambient shifts these former all-stars had come up with. Some really cool stuff, like underwater to above-water, nice examples of occlusion through shutting a car door in the face of a horde of zombies, etc.

But while they all had lots of imagination and detail, none of them really felt like they were moving to me so much as the environment was changing over time.

I wanted to do better, here; to get movement across at all costs, even if it meant I had to keep my concept a little more grounded in reality. From jump, I thought that this would be a great chance to learn a bit about binaural recording (the project was going to be played back/evaluated in headphones) and to try some in-the-field source recording techniques, stay away from the very close-mic'd dry recordings we'd normally do in our Foley rooms.

The goal was to keep the environments simple and just let the movement speak for itself. And in that respect, I think I did pretty well.

What is Binaural Recording?

Brief aside on what all this "binaural" stuff means:

Binaural recording is a method of recording sound that uses a special microphone arrangement and is intended for replay using headphones.
-- Wikipedia, "Binaural recording"

Sound waves don't just dive straight into our brains via some kind of "line in" jack: they have to move through air to get there. And as they do that, these waves bounce and reflect off your shoulders and chest, wrap around your skull, move through your hair, filter through your clothing, etc., all the while undergoing subtle changes in frequency, volume, and arrival time (e.g. hit your left ear just before your right ear), to name a few.

Obviously, our hearing receptors don't sit in an open, microphone-shaped capsule in front of us, they're buried deep in our ear canals. So by recording via two tiny, omnidirectional microphones positioned roughly where your ears are, you are going to record the sound not "as it sounds" in some pure, idealized sense, but as it sounds to you.

When played back, all of those subtle frequency/delay/arrival time changes you captured by recording this are interpreted by your brain in a way that makes you instinctively feel as though those objects are coming from where they were when you originally heard them. We don't hear "slight loss of frequencies around the 8,000Hz range because those tiny waveforms were absorbed by my haircut," we hear "behind." As an incorrect but illustrative example.

By recording this way we can capture much more detail than just "left" or "right" - we can get above or below, in front or behind, etc. And you can turn two channels of simple stereo into sounding like a whole 360-degree sphere of surround.

And yes, people build lifelike human mannequin heads, complete with density-matched materials to simulate the hard reflections of our skull vs. the soft, absorptive tissue of our outer ears, embed microphones in them and use these devices to do these sorts of recordings. At the fanatical level.

Raaaaaaaawr

This is a popular binaural recording example that might blow your mind - wear headphones.

There's a lot more to say on this than I can teach, so if you're interested, look around. Our ears are amazing.

Deciding on a Scenario

First, I brainstormed a few scenarios - I wanted something simple and real, with the chance to show off a lot of movement and cool perspective changes. The most interesting to me was that concept of a Metal Gear Solid-style infiltration, with some close spaces, hearing people filtered through a vent shaft, and eventual running/scuffling.

I "prototyped" some of my recording techniques by squirming around in a broken vent we have in our props room and adding a little bit of reverb to the signal, and got some really nice, claustrophobic results. I also affixed two omnidirectional clip mics to a pair of classes, hooked myself into a portable recorder and walked around our campus a bit to see if it'd work as a poor man's binaural setup. The results weren't 100% interesting all the time, but a couple of cool moments convinced me that this was an avenue to chase.

Planning and Recording

Vent decided on, I sketched out the rest of the soundscape and the environments it'd take place on - again, sticking to stuff that was close to campus so that I could use a lot of my own recordings. I had to figure out roughly how much time I wanted the character to spend in each environment, plus making sure that I was hitting the assignment requirements in terms of "signals" (in-your-face, attention-calling sounds like guard footsteps and key jingles), "keynotes" (middle-ground emitting sounds like vending machines, vents, a radio) and "ambiences" (persistent background sounds that give some subtle information about where you are).

With only 90 seconds to work with, I wanted to make sure you knew where you were as fast as possible whenever the environment changes. If you were in a vent, you needed to know within a couple of seconds. So I put a lot of emphasis on making those binaural recordings the "bed" of the piece, cranking them in the final mix. Normally, your ears tune out "room tone" as you hang around an environment, but I thought that if I were too subtle with these recordings, the cool perspectivey effects wouldn't come through.

Some things I recorded this way were all of the different room tones I wanted, the character's footsteps, and the stairwell door opens and closes - this let me put all of that easily within "his" perspective without having to do any extra mixing. I also spent a bunch of time tilting myself around like an idiot outside in our school's courtyard to get some really dizzying panning on the nearby traffic for the fight up on the rooftop. That, with some of the low- and high-end rolled off to give a "rooftop" sense of distance, became the ambience for the final fight on the rooftop.

For the main character's breaths, I used a single lav mic and just panned it directly in the center (when little else was) to give it a bunch of presence. Simple but effective!

All other effects were recorded in mono with the phenomenal Sennheiser 416. I love love love this microphone for what it's good for, and it feels great to finally be discovering favorites. It's extremely directional, which means I was able to walk around campus and point it at the tiniest little emitting spots on machines, coolers and various electronics and pick up their distinctive buzz without capturing too much of the rest of the environmental noise. That plus our Foley rooms' new preamps allowed me to grab some extremely clean recordings of all the other stuff I needed: guard boots, cloth punches, key jingles, mug and paper tosses, etc.

These recordings were also almost totally free of our floor's soul-crushing, inescapable 120Hz hum. As a result, I barely needed to treat these recordings with EQ etc. before I started cutting the final piece. Saved me a ton of time.

Finally, the brilliant guard dialogue was written by yours truly in a lame attempt to explain why these two guards just wouldn't shoot the guy on sight, plus give a little extra depth and a sense of threat to the whole piece.

I wanted to experiment a bit more with having my classmates (the two guards) read their lines a bit farther back from the mic and even "off-axis" (not facing the direction that the mic picks up sound in) to try to have that sense of people moving around and not talking directly towards the main character's ear, but stuck to just recording it normally and positioning it in the mix later on. The one "hey!" in the stairwell was recorded a few floors up and in the stairwell, though.

Editing and Mixing

My major discovery for this part of the project was WaveArt's Panorama plug-in, which I managed to convince our IT department to install a trial of on my machine. It's a 3D panner that uses some of those same binaural recording principles to subtly adjust certain frequencies, volumes and delay times in a recording to let you position objects wherever you want within a stereo sound field. And as an RTAS plug-in, everything in it was totally automatable.. which basically let me "draw" the path of all the objects in my soundscape according to where I thought my main user should be. Tooooooo freaking cool.

The Panorama interface.

The only shortcoming I found with it was that it doesn't respond well to really quick pans/head turns, as that's just too much math too fast for the plug-in to crunch. You end up getting a lot of phasey, washy sounds when you do it, so I had to keep the 3D panning stuff slow and gradual. Small price to pay! Being able to place the guard's voices below and increasingly behind the character as he moved forward was amazing, and I'm especially happy with how the quick front-to-back pans on the office objects (mug of pens, paper) worked in the middle of the chase. That plus the whirl-around as the guards open the door on the roof.

The next big victory in positioning everything in this environment came from messing around with multiple reverbs. When I started, I was using a standard low-pass filter (that "muted" effect) to try to make the guards' voices sound as if they were being blocked by the walls of the vent, but it felt forced. Sending their voices into the same claustrophobic, metallic-sounding reverb I was using for the main character's vent squirming, while keeping their dry voice signal down, worked way better. Favorite examples of verb usage in the soundscape are as the main character pans his head back up and one of the guard's voices seems to "move" into the vent, as well as the way I treated the alarm as the character bursts into the stairwell.

Final Result

Separated into stems because we had to turn them in that way, and it might give you a better look at what's happening in each part of the whole.

Ambiences Only:

Keynotes Only:

Signals Only:

Final Mix:

What Went Right / What I Learned

  • You shouldn't be scared of recording *in* a location when you want something to sound like it happened there. It seems obvious, I know, but up until this project, I really felt like the safest way to do things was to record sounds clean and close in a soundproof room, then control the way everything was positioned/echoing in a mix later on. For an assignment like this, I saved a ton of time by *not* doing this that way, and even managed to create some more realistic results by doing it. I couldn't have made that stairwell running happen as realistically in any other way but just recording myself jogging up them with two tiny mics clamped to my glasses.
  • A little noise isn't a big deal. Seriously. When recording room tones etc. with our tiny omnidirectional microphones, I had to turn the gain way up in order to get anything to record at all. Doing tihs added a ton of hiss and white noise into the recordings - the mics weren't meant to really be turned that high - but in the final mix, once all the ambiences were in and with everything else going on, you couldn't hear that shit at all. If I had spent any time cleaning those recordings up with Xnoise etc. before mixing them, it would have been totally wasted.
  • The Sennheiser 416T is awesome. So is Panorama.

What Went Wrong

  • In retrospect, the whole project took me way longer than it should have due to my own meticulousness - if I had gotten my shit straightened out with recording planning and done that all in one day, things could have happened faster, but because of this term's schedule, all my recording sessions were happening on and off through the end of the term. As usual, "organization will set you free."
  • I wouldn't have done the fall from the vent/ring-out as dramatically, if I could go back and do it again. It's a bit over-the-top and Gears of War-y, but it helps cover up the fact that there's not much going in that section of the piece besides some footstep scuffles. A straight ring with a less dramatic low-pass might've done it.
  • Not totally totally sold on the fall from the roof as well - I wish I had some closer-sounded traffic sources to gradually fade in to give a better sense of proximity. Struggled a lot with this last section!

Please leave any questions/thoughts/criticism in the comments!

28Mar/110

Game Audio – Trailer Music Edit

It's been a while! Our third term has been crazy busy over here, with more post audio work, lots of field recording, some on-set film collaboration stuff, Max/MSP classes and an animation I'm doing the sound for (with a few good friends) on the side. We've just passed a nice block of assignments, so I wanted to come up for air and post a piece of work before diving back in.

Game Audio II

Emotive Game Trailer Assignment: COD4

One of the realities of working at a small-scale game developer (or even some larger ones) is that if you're on-site as the sound guy, you'll occasionally get stuck with some non-developmental audio stuff to do. When the sounds have all been created and implemented and the game's headed for gold, your publisher will want promotional trailers - and you may be the guy to cut them.

For this assignment, we were to choose from a selection of popular game trailers (audio removed) and a handful of available library songs in a variety of genres, then cut, process and edit any number of those songs any way we saw fit to give some emotion to the moving image - accent cuts, create dips and valleys, all that good stuff. The artificial challenge here is that we were able to work only with the source songs as raw audio material, and couldn't bring in extra SFX to do impacts, bass dives or any other tricks we might want to use. We had to get resourceful.

In the end, I used bits and pieces from 5-6 of the potential songs to create the video below. Just a quick look at the original audio before we hit my edit:

SD49 Luca Emotive Game Trailer Sources by lucafusi

It all took off from messing with the pitch of the DnB song and realizing it sounded like a pretty cool beat in and of itself. That track was also the major component of my SPFX during the intro soldier segment and the gunshot to the screen. Here's the final result:

More in the coming days or week. Keeping busy!

14Mar/110

More Plankton

More goods from Phytoplankton after last week's session. It's great to be able to keep our skills pretty fresh with some of the amazing Native Instruments VSTs and standalone kits we've got in the Term 2 classroom, even as the rest of our program sends us further away from them. Not included is the strange, 19-minute soundscape that erupted in the middle of it all..

Locomotive

Four-parter cut up into little movements. If you want the whole thing as a seamless file, shoot me a mail or comment!

The Deep

The Sink

The Dive

The Surface

Once again, Phytoplankton is:

8Mar/110

Phytoplankton

Weapons of choice.

Weapons of choice.

Taking a moment here to link to our SD49 supergroup, Phytoplankton! A couple of friends and I get together on the weekend evenings and abuse our having a classroom that's totally kitted out with all the Native Instruments packages to experiment and jam.

Some good stuff in here from the last few weeks, cut out from a few hours of messing around:

Dinoflagellate

Chicken of the Sea

Deception, Pt. 1

Deception, Pt. 2

Also be sure to check out the Soundclouds/pages (some on the right-hand side) of the other guys (and girl) in the band: