9 Sound for Story

Fictional podcasts, audiobooks, audio dramas, and radio dramas are some of the most common uses of sound design for story outside of audiovisual media. Podcasts are episodic series of fiction or nonfiction news, tutorials, interviews, and stories, named after their popular consumption on portable MP3 players like iPods (today they are perhaps more commonly played on smartphones). Radio dramas are a more traditional format of fiction in single or episodic form broadcast on the radio, originally, but more recently through other media as well: sometimes these are also called audio stories or audio dramas, and include stories on smart speakers like Alexa or the Amazon Echo. Audio-only fiction, including audiobooks, increasingly don’t just have an actor reading the book but include sound design as well. Even nonfiction podcasts take some form of storytelling, so I’ll refer to these collectively as audio stories. Today, many audio stories are witnessing a wave of interest, as portability (MP3 players, phones) and easy access have made listening to podcasts and audio dramas more feasible and popular.

We are not going to focus on the voice here, which deserves at least a book on its own, and neither are we going to focus on narratology and the construction and analysis of story. Rather we will consider how we can use sound to create and support story. While some nonfiction podcasts and audiobooks don’t use sound beyond voice, the most engaging of them tend to use some form of sound design, even if it might be sparse in places.

In this chapter, we bring together everything we’ve learned so far: sounds, recording, space, effects, mixing, and meaning. We’ve already talked about many of the tools we have at our disposal, and we’ve practiced using many of these in our exercises. We’ve looked at how sound can help evoke sensory responses like imagery and touch, and now we’re going to explore how we can use these and other techniques to support storytelling. This chapter will help us explore where, why, and how to use some of these tools in supporting a narrative.

9.1 Functions of Sound in Audio Story

To understand how sound can support story, it’s useful to examine the functions or roles that sound plays in audio stories. Unlike film, television, or games, in audio stories sound is not supporting an image but instead plays a central role. Rebecca Parnell, a sound designer for games, explains what roles she sees sound playing in games specifically, but many of these also apply to sound design for any media:

It can influence how the player experiences a game—how they feel. Do they feel tense? Do they feel scared? Do they feel happy? Does it feel fun? If it’s an arcade game and it’s high-adrenaline and you’re scoring loads of points, how does the sound make the player feel? And that all comes in together, so it’s not just about ticking boxes and, “Oh, this door opens and closes.” How does it sound? Does it close softly? Is it slammed shut? What is the emotional impact behind that sound? And also another element, as well, especially with ambient sounds, is what is the context that you can give within the sounds? What information can you give to the player that you don’t have to spell out in story and dialogue? How can you fill in the gaps? (quoted in Collins 2016, 97)

When it comes to audiovisual media, so much relies on visuals: facial expressions showing character emotion, camera edits and scenery to show scene changes, clear delineation of which character is talking and what role they play, what time of day it is, what the weather is like, what people are wearing, what era it is, and so on. When it comes to audio-only drama, sound has to fill all those roles. In a way, sound design for audio-only media is more complex, because a sound designer has so much more to convey without the image providing support. The many roles that sound plays tend to overlap, but I’ve separated them here so we can think about them in more detail and focus on each one.

9.1.1 Commercial and Aesthetic Functions

At its most basic level, sound plays a commercial role in some audio stories, including a sound signature or audio logo (idents, or mnemonics) at the start, any repeated interface or transition sounds that may be used to change or interrupt segments of a story (bumpers and stingers), and so on—these all have to do with branding the audio drama or podcast, helping to create a sense of familiarity with listeners to associate the work with a particular brand. When we have an episodic series, this branding aspect can be important to remind our listeners that they listened to (and hopefully enjoyed) a previous episode as they remember the sounds.

Related to the branding is the use of sound to improve our appreciation of the work, to make it more enjoyable, “cool,” or indicate a particular genre or lifestyle. A podcast for hip-hop fans is going to sound very different, be paced differently, and use different mixing from a podcast designed about knitting for grannies, for instance. A newspaper ticker-tape sound is a classic news signifier; a gloomy but goofy evil laugh is a classic B-movie style horror signifier.

Exercise 9.1 Ident

Make an ident—a sonic signature, or audio logo—for a podcast series. Think about what you are trying to convey. Here are some suggestions to get you thinking:

  • An audio logo for a game company’s podcast
  • An audio stinger for a new phone advertisement
  • An audio bumper for a quirky edgy news story podcast

Exercise 9.2 Audio Aesthetics

Find some sounds and music that would be suitable to accompany a podcast for horror fans, and then repeat the exercise for a podcast for car racing fans. What did you choose, and why? How does it relate to the brand the podcast might want to convey? What effects did you add to the sounds, and why?

Exercise 9.3 Thinking about Tropes

The classic news ticker sound and the evil laugh are common genre tropes. What other tropes do you associate with specific genres of story? Why do you think those tropes developed? Try turning a trope on its head—a horror news show, or a comedy horror show. What do you have to do to the trope to flip it?

9.1.2 Setting the Scene

With audio stories, we need to know fairly quickly where we are supposed to be located (in time and space), since we can’t see the location the way we can in a film or game. Setting the scene or environment to represent a particular location geographically or temporally is an important function of sound. We’ve spent a lot of time listening to and thinking about our own soundscapes and the elements that go into our environment’s sonic uniqueness. Can you think of the most important of those sounds? What shorthand signifier could you use to represent your particular location in space and time? What is the foreground, midground, and background, and how are these constructed in the mix? Where are the sounds in space, and why are they placed there?

Environmental sounds can also be useful to indicate a change in narrative—such as a shift to a new scene. Some simple differences could indicate a change from day to night, for instance, by quietening down the ambience and adding crickets. Weather, time of day, geography, indoor/outdoor, season, ecosystem, and the like can all be signposted by using sound. Think in terms of foreground, midground, and background. What are the key sounds in the scene, and how do the midground and background support these?

Soundscapes also help create mood—they’re not just about time and place but play a role in shaping that time and place as well. What does the ambience also say about the mood? Hard rain is different from soft rain, which is different from rain with wind. Adding reverb can put us at night, but also in a lonelier space. A soft wind in the desert is different from a howling wind in the desert. Adding distortion can make the surroundings sound more aggressive and closer to us. Think about how the ambience can tell us not only about the place, but also about how we are supposed to feel about that place.

Exercise 9.4 Impossible Soundscapes

Make an alien soundscape. What sounds did you choose, and why? What effects did you choose, and why? How would you change the soundscape from good alien world to bad alien world, and why? Make a future soundscape. What sounds did you choose, and why? How would you change the soundscape from good future (utopia) to bad future (dystopia)?

Exercise 9.5 Combining Place

Combine elements of two different soundscapes you’ve recorded into a new soundscape to create a new place. Which elements did you choose to keep, and which did you throw away, and why?

9.1.3 Subjective Perspective

We looked at proxemics and the point of audition previously (chapter 3). Thinking about where you want the audience to be “located” in relation to the characters is important. Give them a perspective. Do you want it to be close and intimate, or are they at a distance? First person, third person, or omniscient? The perspective may change multiple times throughout a longer production. How will you alter the subjective position of the audience to enhance a story? What elements will you use to alter the subjective perspective? Sometimes, an audience is real or imagined in audio stories, for instance in the use of a laugh track. Listen to some audio stories and think about what the creators did in regard to their audience.

Exercise 9.6 Changing Perspective

Imagine we are in a scene where a serial killer is stalking someone. Shift the perspective from the victim’s perspective to that of the killer and back. What did you do to shift the perspective, and why?

9.1.4 Structural Functions

Sound can be used to enhance or demarcate formal structure. Links, segues, and reverb tails can all signify a change in scene. For instance, tailing out the ambience of one scene and bringing in another scene with a hard cut can help your audience rapidly grasp a quick change of scene. Using a soundmark (for instance, a church bell) in the ambience can help the audience to know when we’ve cut back to that location after going somewhere else in the story. In this way, sound can be used to enhance continuity across scenes. For instance, if a character goes into a flashback, we might bring back the original ambience when the flashback is over to bring the listener back into the scene.

Sound can also be used to indicate a change in the narrative: if we’ve gone from a narrator’s current day into the past and into the scene itself, for instance. And sound can be used for foreshadowing. Foreshadowing is a warning sign in advance of something happening at some point in the future. For example, the sound of thunder, even though the sky is clear, indicates that something bad is going to happen (i.e., a storm is coming, literally or metaphorically). Sound can be used to foreshadow events in story. But sound can also be used to set up an expectation related to a more immediate event. For example, the sound of a plane or bomb dropping from the sky sets up an expectation that we are going to hear a crash/explosion. We can use expectation to confuse the listener by disrupting that expectation. Foreshadowing, on the other hand, probably won’t work if we disrupt it, because it’s often only later that we recognize that the event was foreshadowed. Violating expectations, and then resolving that later, can provide an important sense of tension and release.

Another role related to the structure is the use of sound to pace the story, to take the audience on a journey. Changing the pace can create a sense of urgency or anxiety, for instance, or can be used to demarcate scenes. Listen to an audio story and think about how changes in scene are indicated in terms of sound. What types of transitions or segues are used, and how and why? How is silence used to create tension, emphasis, and focus?

Exercise 9.7 Structural Functions: Scene Segues

Use two very different soundscapes you’ve created or recorded. Try some different techniques to cut between the two as if they are two different scenes. What did you try, why, and what impact did it have?

9.1.5 Creating Characters

We learn so much about characters in audiovisual stories by their posture, their expression, their clothing choices or sense of style, and so on. None of those are available to us when we’re just using sound, but we can still create a sense of who a character is. We can tell the difference between someone wearing a leather jacket and a ski jacket, for instance, by the sound their clothing makes. We get a sense of their gender by the way they walk and what type of shoes they wear, but we can also learn more about them by their footsteps: are they injured? Do they have a physical disability? Are they angry or in a good mood? There are all kinds of things just simple movements can tell us about characters. We can use leitmotifs—repeated sonic patterns, usually musical but not necessarily, that stand in for a character, to avoid having to use their name to indicate their presence in the story. The stereotypes and generalizations discussed in chapter 8 can come in handy, although we must also be careful not to offend. We can use panning or other spatial aspects to differentiate characters in the story if there are actors with similar voices, and place those characters into the space using the worldizing techniques discussed in chapter 4. Listen to some audio stories: How else are characters differentiated? How exaggerated are the sounds compared to real life? What are the characters feeling? Sound designer Walter Murch explains the use of sound to describe the inner life of a character in The Godfather:

That example of the elevated train in Godfather is something that’s primarily an emotional cue. There’s rhythm to it but only to a certain extent, and story-wise it’s a little ambiguous. “What is that sound? What is it doing in the film?” There’s not an easy answer to that. But emotionally you absolutely understand what that sound is there for. Because there’s nothing in the picture that is anything like a train—although reasonable that a train might be heard in that part of the Bronx—the emotion that comes along with that sound, which is a screeching effect as a train turns a difficult comer, gets immediately applied to Michael’s state of mind. Here is a person who is also screeching as he turns a difficult corner. This is the first time he is going to kill somebody face to face. He’s doing what he said he would never do. (quoted in Jarrett and Murch 2000)

When it comes to nonhuman characters, sound design really gets to shine. Sound designer Stephan Schütze explained to me the complexities of designing creature sounds:

What I would try to do is, I’d try to look at a creature and not look at it and go, “Oh, it looks like a bear, therefore, I should use a bear sound.” I’d try to look at it and think, what does that represent to me? Is it something that is a threat? Is it something that’s a threat because it’s a carnivore, or is it something that’s a threat in the way an elephant is just simply because it’s so big and it’s potentially unpredictable? And often it’s not going to run me down to eat me, but it might run me down to protect its own herd or to protect itself. But those are going to be very different sounds.

. . . So I think what you’re trying to get, first of all, is you’re trying to get what is the emotional impact you want that creature to have, and what is its behaviour like? How is it going to behave? Then you can start to think, alright, is it a big creature or a small creature? Now by and large, sound just works based on physics. A mouse is generally going to create a high-pitched and probably fairly quiet sound because it’s got small vocal chords. It’s like a short violin string, whereas an elephant, it has larger vocal chords, it has a larger set of lungs behind it. It has more bulk so it’s likely to be a bigger sound. It’s not always going to be the case, but it is something that as humans, emotionally, we respond to. If I build something that’s supposed to be some giant Titan and it squeaks at you, unless I’m trying to purposely have some sort of comedic value to this thing, it’s likely to go against what people are used to. So you need to sort of at least keep that in mind. So again, I’ve come to the creatures and I’ve decided what sort of emotional impact do I want to have.

Then it comes down to what sort of textural behaviour do I have, and I’ll use the example of the dinosaurs. When I worked on the Jurassic Park game many years ago, I had to create the sounds of the dinosaurs. Now the obvious ones like the T-rex and the raptor, which you know from the movie, well, they were pretty much, we just used those sounds. But we had quite a large range of big herbivores. Now, the thing about herbivores, they’re herd animals. So I had to work out their type of behaviour. Now, herd animals, generally you put one herd animal somewhere, they don’t actually talk very much, because herd animals communicate. And so working out what types of communication they’re going to have. Herd animals generally will have warnings to the rest of the herd, so there needs to be this kind of like, perhaps family, sort of like, “Hello, where is everybody else?” to, “Oh dear, there’s something really, really bad about to happen, let’s get out of here.” And so then we’ve worked out the vocabulary, so I’d probably done a lot of this before, I’ve even started playing the sounds.

So then I get to the point of like, alright, well, what do I want to do? And I mention that I think bird sounds, pitching through those extremes, can be really quite amazing. And bird sounds, when you pitch shift them down a couple of octaves, or even more, you can get incredible textural sounds, and the thing is, I don’t have to build. I don’t have to artificially build a vocabulary. Because if I decide, for instance, we have a bird in Australia called the Peewee Magpie. It’s like a small black-and-white bird. And they’ve got a range of sounds. But their range of sounds will be threat sounds, communication sounds. I’ve got all these sounds. The vocabulary exists, so I can pitch shift it down and I’m still going to get the sense of threat in a warning sound. I’m going to get the sense of community in a “Hello? Where are you?” sort of thing. I’m going to get the sense of urgency of “Oh, I’m hungry, I want to go eat” from the babies. And so in some ways, it’s almost cheating, in some ways. It’s kind of like I’m stealing another language and I’m just manipulating it to use for my particular thing. Now, I may need to chop and change some of those things, and that’s the beauty. When I pitch shift them down, I might lengthen them, I might get longer phrases that I can then pull apart, and in a procedural way I can turn them into a language. (quoted in Collins 2016, 197)

Exercise 9.8 Creating Characters

Create a nonverbal dialogue between two characters, such as a woman and a man. You can pick the place and situation. How can you differentiate the two characters with sound (not using any voice at all)?

9.1.6 Believability and Immersion

Constructing a believable world is critical to creating a sense of immersion for your audience, allowing for or encouraging the suspension of disbelief, and adding realism. If sound designers were just mimicking the real world, sound design would be a lot less work. But we sometimes put sounds into scenes where they may not in reality belong, because their absence would make it feel unreal. Nowhere is this more obvious than in representations of outer space, which of course is in a vacuum and so has no sound. I’ve previously called this cinematic realism—it’s not realism, but it’s what we’ve come to expect based on our past experience of screen media. It’s believability, more than realism.

We’ve come to expect the “whooshing” sound of kung fu punches and kicks, the screeching of tires (even on gravel roads), the metallic “tshing” of a sword removed from a scabbard, and so on. Their absence would be noticed, and going for realistic punches might sound really boring for an audience used to hearing the dramatic hits of movies. Frogs make a variety of noises, but what sound do we associate with frogs? “ribbit.” But the “ribbit” is actually just one specific species of frog located in Hollywood, called the Pacific Tree frog. A Hollywood movie needed the sound of frogs and sent their crew out to record frogs, and they came back with the Pacific Tree frog sound’s “ribbit,” which many people think all frogs should sound like, even if frogs in the rest of the world don’t make that sound! The result is if you go out and record real frogs in the region where the program is set, the audience may not understand what the sound is. Again, it comes down to believability, not realism.

Physics can also be different in the auditory world. We know sound travels at a speed of 343 meters per second at 20°C in dry air. If we shot a gun toward a cliff one kilometer away, it should take nearly six seconds to return to us, but if we left a six-second gap to have an echo off that cliff in an audio story, the audience would likely be really confused. What we’ve become accustomed to based on our experiences of media can override reality, in other words, so understanding what conventions already exist—doing our research—is critical to believability.

Nick Wiswell, who designs the sound for many car racing games, explained the importance of finding the right balance of creativity and accuracy. As the explanation shows, a lot goes into all aspects of sound that play an important overall role in the audience’s experience.

Everybody has an opinion on the authenticity of a car sound. Some people have driven it in real life. Some people have seen it driving around in the streets or on a racetrack. Some people have watched a YouTube video. Some people have seen it in Hollywood, in a movie. None of those things sound the same. So, what is authenticity? Authenticity is your personal perception of how that car should sound. So we have to take all these things into account when we’re trying to design. What is an authentic sound? Because you’ll have people who will say, “You’ve got the sound of this car wrong. Look at this YouTube clip,” that was recorded on somebody’s cell phone that’s just a big distorted mess. But they’ll say, “This is what that car sounds like.” And to them, that’s their authenticity bar. Where somebody else will like, “I own this car, I drive it every day and it doesn’t sound how it intended, how I remember it, or how I feel it should sound.”

All I know is I was there, or one of my team were there, to record the car. We have the recording of the car, so if I wasn’t there at the session, my perception of the authenticity of that car is the recording we got back, but what if the recording we got doesn’t capture something that people think of when they think of that car? So, one of my biggest jobs is to try and take all of these pieces and say, “Well, this The Fast and the Furious movie made the car sound this way, but this YouTube clip says it sounds like this, which is completely different. I’ve got this recording, that doesn’t sound like either of those two. There’s a guy in the office who used to own this car, and he says, ‘This is how it should sound.’” So all of them are right, and none of them are right all at the same time. (quoted in Collins 2016, 306–308)

9.1.7 Connotation and Metaphor

Sounds can have many connotations, and sound effects can be used in subtle or quite obvious ways to fill in the blanks in what is said, or add slant or bias. For instance, we could use a politician saying “There is absolutely no risk of war” and have the sound of guns and bombs in the background to give the impression that we think the politician is lying. The sound can stand in for something else and through this relationship reveal something about character, plot, theme, product, and so on, as metaphor. A metaphor, if you’ve forgotten your high school English class, is a figure of speech in which a phrase (in our case, a sound) stands in for or is symbolic of something else.

Walter Murch has written a great article on sound and metaphor in film, and while most of it applies to sound and image, one great quote stands out: “The metaphoric use of sound is one of the most fruitful, flexible and inexpensive means: by choosing carefully what to eliminate, and then adding back sounds that seem at first hearing to be somewhat at odds with the accompanying image, the filmmaker can open up a perceptual vacuum into which the mind of the audience must inevitably rush” (Murch 2000). The use of sound on its own, without image, can serve an equally important role as metaphor. A rising sequence of frequencies has different associations from a falling frequency: we could use rising frequencies to represent a training sequence in a story about a boxer, or falling frequencies to represent that boxer’s fall to the mat after a knock out.

Exercise 9.9 Sonic Metaphors (Partner/Group Exercise)

Come up with a list of sonic metaphors, and compare them with friends if you can. Are you “on the same page” when it comes to metaphors?

The juxtaposition of sounds with each other can lead to a form of closure. A concept from Gestalt psychology, the idea of closure is that our mind fills in the blanks when given certain parts of a whole. We could call the phantom fundamental (section 2.5.1) a form of closure, since our brain is filling in the blank of the fundamental by using the harmonics of the tone. But we could expand the concept of closure to include how the brain can fill in information that isn’t there to provide a sense of completion. We may fill in the auditory ambience that gets ducked to focus on other elements of a soundscape, for instance. In the last chapter, we touched on intersensory integration, and it’s often suggested that “the best pictures are in the head”: part of closure may involve filling in the visual blank of radio listening with sound. As Nick Wiswell describes above, we don’t have to use all the sounds in a car to represent a car—we can just use a few and our minds will fill in the rest.

We can also use hyperbole—the obvious exaggeration of a sound to make a point, counterpoint and contrast for irony or humor. We can use anthropomorphism to enhance a listener’s feelings toward or empathy with a nonhuman character (think Wall-E).

Exercise 9.10 News Story in Sound

Take a news story from the day and tell its story in just sound: how might you use sound in some of the metaphoric or ironic ways just mentioned to emphasize or criticize the journalist’s point of view?

Exercise 9.11 Anthropomorphic Sound

Grab a stuffed teddy, robot, creature, action figure, or nonhuman doll from a junk store, and give it a series of sounds. Imagine animating this figure: how would you anthropomorphize the sound of it, and why?

9.1.8 Mood and Emotion

There is a saying in the video game industry, by composer Marty O’Donnell: “Music makes you feel, sound makes it real.” I disagree with Marty, because sounds can make you feel as well! In fact, sounds can express emotion, underline emotion, as well as induce emotion: by this distinction, I mean it’s possible for sound to tell us how a character is feeling without necessarily making us feel that way, but it’s also possible to make us feel that way. We can hear a character is angry without feeling angry ourselves; but some sounds can actually induce a particular emotion in us.

We looked at some effects and talked a bit about how reverberation can feel warm, phasing can feel psychedelic or mimic the effect of drugs or mental disturbance, overdrive can feel warm but when pushed can feel more like anguish/anger. It’s not just the choice of sound and the effects we use; as we have discovered through our journey, other aspects of sound impact feeling. The proximity, loudness, and directionality of a sound will also have some impact on its affect (see Tajadura-Jiménez et al. 2010). The context in which the sound is heard will also influence the perception of the sound. It is extremely difficult, then, for us to determine the effect of a sound on every person in every context, but we work with generalities and with our intuition that we develop over time and practice.

Joanna Orland, a game sound designer, describes how the selection of sound in the ambience can influence emotions:

I believe that audio plays a huge role in the emotional effect on the player in the game. Audio is what is manipulating them emotionally. We can use almost psychological tricks to ensure that they are feeling something without them even noticing because when audio is done well, no one notices it. It’s only when it’s done wrong or poorly that people take notice. I believe, with audio, if your audio is well thought through, you can use emotional tricks. So for example, with the horror genre, there’s quite an obvious pattern that film does with the soundscapes, you know, the environments are always very abstract and hyper-real, which is an exaggerated form of reality, and the character sounds are always very naturalistic and realistic. And I believe it’s a psychological trick because the player, if it’s a horror game, or the viewer, if it’s a horror film, can feel the emotional attachment to the character, because they recognize the sounds as human and as fragile, almost because it’s very naturalistic, and almost frail, in their human elements. But then when this character is placed in this really hyper-real environment, there’s a tension that’s created because this character that they believe in, because they’ve got an attachment to them, is put somewhere very unfamiliar, which is scary and unnerving because this environment sounds really alien, and they don’t know what can happen to this character which they’ve got an emotional bond with. (quoted in Collins 2016, 94)

Emotions are highly complex, and the use of “mood maps” to pigeonhole emotion is controversial; however, mood maps can be an effective tool to help us to define general categories for discussing sound’s affect and think about our use of emotion in sound design. There have been similar attempts to categorize and label emotions, such as Plutchik’s “wheel of emotions.” A mood wheel can be useful to think about what characters are feeling, and what you want the audience to feel at any point in the story. Plutchik’s wheel, shown in figure 9.1, puts emotions on spokes that radiate outward from most to least intense (see, e.g., Donaldson 2017). As you progress in your sound design skills, you’ll come to associate emotions with your own bag of tricks that helps you to convey that emotion.

Plotting the key emotional points in the story and aligning scenes with moods can help you think about how to emphasize key moods. We talked a bit before about using dynamic range for emphasis. What are the peak emotions, and what are the peak sounds? At what point do they appear in the story, and how do they ramp up? How does dynamic range draw the audience’s attention toward that particular event? (As discussed above, dynamic range and amplitude are not the only ways to show emotion, of course.)

Exercise 9.12 Sonic Mood Board

A mood board is a common design tool where a collection of colors, textures, and images are gathered to represent a particular mood that a designer is trying to explore for a product or client. For instance, if I were trying to create the feeling of serenity or calm I might use images of calm lakes, being snuggled inside an igloo, a candle, a clear sky, soft fluffy textures, and so on. We can create mood boards with sound, too. Find sounds that represent a certain mood because of their signification, their associations, their textures, and so on. Don’t use any vocalizations (no voice, no “emotes,” or vocal utterances), and no images are allowed. Use effects to enhance or alter sounds to really bring out the mood. It’s fairly easy to choose strong emotions like anger, but how would you sonically represent something like “wistful” or “pensive”?

Figure 9.1

Plutchik’s wheel of emotions (based on Plutchik 1991).

Exercise 9.13 Tension and Release

Create a new mood composition. This time, focus on building up tension for the first part of the piece, then give the listener some release at the end.

9.1.9 Information, Feedback, and Reward

Finally, in some types of audio stories (interactive stories, for instance), sound may play an important role in providing some form of feedback to the listener—they clicked on the right button, they collected a good or bad object, and so on. Providing some information about what is going on in the story, and giving feedback to a user in any kind of interactive story, can be one of the most important things that sound can do. This feedback can also come in the form of some form of sonic reward for performing a correct action. In an audio-only game or interactive story, for instance, undertaking a particular action that results in winning or scoring points should be accompanied by some auditory reward. This “audio bling,” as I like to call it, provides positive reinforcement.

Exercise 9.14 Sonic Storyboard

Create a sonic story of about 30 seconds using only sound (no music or dialogue). Document the effects and techniques you used. Some examples of the types of stories you might describe include: 1. You arrive home and scare your cat or dog, who knocks over your favorite vase. 2. It’s the bottom of the ninth inning, and you’re up to bat. 3. Alone in a parking garage, you fear someone is following you. 4. You are lost in the woods, and spend the night in an abandoned barn.

Exercise 9.15 Sonic Storyboard 2

Take your sonic storyboard. Now assume that the entire story takes place in a dream or hallucination. How will you change the sound?

Exercise 9.16 Sonic Storyboard 3

Flashback or flashforward from your story to create the next scene. How will you show the perception of time passing (hint: don’t use a clock ticking!). How did you transition from the first scene to the next?

9.2 The Mix

It’s important to note that although we’re not talking about voice or music in this book, these audio aspects may fall under the purview of a sound designer’s job on an audio story team. A sound designer may be responsible for dialogue recording and/or editing, and selecting or commissioning music. Even beyond that, the sound designer may be responsible for thinking about all of the other aspects of dealing with dialogue and music in the mix: How are sounds interwoven with dialogue and music? Are they given their own space? How do sounds or ambience frequencies interact with dialogue and music—have the sounds been selected or mixed to sit in a different acoustic space? Are there places where there is no distinction between the music and sound—that is, can we use sound design as music or in place of music? Where, why, and how?

Regardless of what your overall role is on the team, if you’re responsible for sound design, you need to be thinking about the music and dialogue and how your sound will fit with it. How is ducking used, and where? How might you use EQ instead of ducking to achieve the same aims? How else might you deal with mixing the audio components together? How are the files being distributed and how is your audience listening to the material? All of the elements of mixing need to come together, because great sound design can be ruined by a bad max.

9.3 Audio Research

We touched on reference material in relation to mixing in chapter 6 and above. Reference materials are the examples from media that we may first turn to when researching a project. If we are assigned the task of creating the soundscape for an alien planet, one of the first things as a sound designer that we might do is see what other people have done in the past. We may watch movies, read books that describe the sound of distant planets, play video games set on distant worlds, and so on. While it’s certainly possible to try to create an entirely new take on alien planet soundscapes, we know there are some conventions that audiences rely on (see above). Understanding how those conventions have been used in the past is key to understanding how much we can push our own design. Joanna Orland, a sound designer for games, explains:

What I do is, I start off by creating an audio vision, which is what the audio’s goal is for the game that I’m working on. For example, when I worked on Wonderbook: Book of Spells, that was a magic game, so I had to figure out what the sound of magic was. So I did a lot of research, I watched a lot of films, read some books, and I discovered a pattern in mostly the film stuff that I watched. Basically, in a lot of films and popular culture, the magic sounds are sourced from nature. So I decided for Book of Spells that was what we were going to do as well. So I created this whole audio vision that the sound of magic would be sourced from nature. And then, as a sound designer, I had to then create these sounds that fi t into this world that was being sourced from nature. I think we do look to film quite a bit as an example of how our audio should sound, because the mainstream is pop culture, and film is a popular media, and that’s what people are used to, these conventions that we hear are defined by the films that we watch. So with games, we don’t want to do something so unfamiliar to people. We want them to believe in our vision, so a lot of the time we will use familiar tropes and sounds and methods that film may use. . . . It’s effective, it’s proven, it works. So we’re not reinventing the wheel in games. (quoted in Collins 2016, 93)

When films are being made, they will often use temp tracks for music—music that is the feel or atmosphere that the director is aiming for. That often gets sent to the composers so that they have an idea of the type of music the director wants. Sometimes, the temp tracks even end up staying in the film, if the director or others form an attachment to seeing the music. When it comes to sound, it would be very rare to get temp track ambience or other sound design, but we can still think along those lines: that there is a template that exists for what we want to do. We may not always want to rely on the template—after all, we want to flex our creative muscles—but taking a risk on a completely new approach isn’t always our job as sound designers. We want to evoke a certain location or mood quickly and effectively. Most of the time, we actually don’t want to be noticed (other than by other sound designers!), so we can’t stick out too much.

9.4 Audio Story Analysis

Analyzing how other people are effectively (or ineffectively) using sound is one of the best ways to explore sound design for story and do your research. The more you train yourself to be thinking about sound design, the better equipped you’ll be to approach your own sound design. Below is a list of just some of the questions you should be asking: and not all questions will apply to every audio story.

Beyond these more technical questions are the personal ones—what did you like about it, and why? What would you have done differently?

Exercise 9.17 Audio Story Analysis

Get cracking on your own analysis. Remember you’re going to have to listen to it several times for different things as you go through it, so pick one that isn’t too long.

Exercise 9.18 Comparing Approach

Listen to two popular podcasts or radio dramas and compare their approaches to sound.

9.5 Spotting a Script

Many writers describe sounds in their stories in order to evoke sounds in their readers’ minds. Whether you are recording a fiction or nonfiction audio story, understanding where, why, and how to use sound is critical to its success. Scriptwriters sometimes write some key sounds directly into scripts. For instance, here is one part of the script for BBC radio series The Wire for the episode “The Startling Truths of Old World Sparrows” by Fiona Evans (2013, 11):

STAN’S KITCHEN. DAY.

DOG WHINES.

STAN

What’s the matter baby? Do you want to go out? She doesn’t like the snow, especially when it’s deep. With her being so small. Poor thing . . .

STAN UNLOCKS AND OPENS THE BACK DOOR

Her belly gets wet. Go on, go out for a wee.

KIDS LAUGHING. A WHEELY BIN IS KICKED OVER. THE KIDS CALL HIM NAMES

STAN GOES OUTSIDE. SHOUTING AFTER THEM.

KIDS RUN OFF.

Oi! What do you think you’re doing? Oi . . . Don’t you come near my house again, do you hear me?

Bloody snow! Can’t run in this can I?

What shall I do? I’ll clear the path. That way I can run after ’em next time.

Note how the scene is delineated by all CAPITALS, BOLD AND UNDERLINED. The character is delineated with an UNDERLINE AND ALL CAPS. Dialogue is not bold, all capitalized, or underlined. ACTION IS ALL CAPS.

In this single scene, we have two locations, which is unusual, although technically the scene is inside Stan’s house, and just in the doorway/just stepped to the outside, but we need two ambiences to delineate the space: inside and outside, even though it’s one scene. Some writers may have separated those scenes out into two distinct locations, or they may be thinking like a camera person and imagining the entire scene is shot from the kitchen, with some view through the doorway.

We have several characters, even though only one has dialogue: Stan, the dog, and the kids. We have some action events: dog whining, door being unlocked opened, wheely bin being kicked (a wheely bin is a garbage or recycling container on wheels). These are the key spot sounds, or foreground sounds, in other words. But so many other sounds are also a part of the scene that we need to include: footsteps, wind or birds outside (depending on time of day and weather, based on context from the rest of the script), kitchen sounds, and the like.

Now we can think in terms of adding emotional impact or the things that are not there. Let’s suppose Stan is talking to himself, not to someone else. What might his home sound like? Maybe the faucet drips because he’s not attentive to those types of things. Maybe he stumbles a little and kicks the dog bowl as he walks over to the door. Maybe the door is creaky and old-sounding. Maybe the lock is not one simple lock but has several stages that need to be unlocked. Maybe the door has a draft stopper that drags on the floor as he opens it. Maybe the floor of the kitchen is wooden and creaky, or maybe it’s tiled and hard. Maybe it’s blowing a gale outside, or maybe it’s a cold crisp night. What about the kids? Are they all boys? All girls? What approximate age—little children sound different from young teenagers. What’s in the wheely bin as it’s tipped over—paper recycling is going to sound different from glass or garbage. These are the creative decisions we have to make that can relate to the mood we want to convey.

When you are spotting a script, during the first read, it helps to read the whole script, paying attention to objects, actions, environments, emotions, and so on. Circle keywords that indicate where and what sounds are linked to specific characters, objects, or actions. Think about the tension and release points. Write down your thoughts as you read.

Exercise 9.19 First Spot

Find an audio drama script (there are many online scripts available: make sure it’s an audio drama, not a television or film script). Read through it and circle the keywords that would help your sound design. In this case, it’s the location, the characters, and the key foreground sounds.

On the second read-through you can think more thoroughly about what sound can bring to the story. What is the overall theme of the story, and what sounds do you associate with that theme? What sonic generalizations or stereotypes do you associate with the time and place, the story, the emotions, the characters, the key objects? Can you adjust your sound effects to enhance the mood or emotion of the key sounds you’ve circled or added that might convey story? Using a mood wheel, mark off which parts of the script associate with which moods. What is the overall arc of the emotion, and how can you convey that with sound? What is the peak emotional point and how can you convey that? How do places and times shift in the script, and how can you convey that with sound? What is the overall genre, and what sounds do you associate with that genre? If Stan is in a horror movie, for example, that’s a different sound than if Stan is in a comedy.

Go through the questions presented in the analysis above, and consider how you might apply what you learned from analyzing other works to your own work here.

Exercise 9.20 Second Spot

Conduct a second (and third and fourth!) read-through on your script, marking out the key aspects you can consider with relation to sound design.

9.6 Cue Sheets

After the first spot, we can make a cue sheet for the script. A cue refers to when a director would point to a person in live radio, recording, or film, indicating whose cue was coming up, so it distinguishes sound, music, and dialogue. We should have a separate cue for each distinct music, dialogue, or sound event. Don’t forget we need time in between dialogue to fulfill the functions of sound in the audio drama, and don’t forget to include silence where appropriate. Here is how we might approach the previous scene by adding cues (cues are usually numbered in a cue sheet, as well):

In the cue sheet, music and sound are underlined. The whole thing may be boldface (these are based on conventions from hand-typing things out and relying on a sometimes poor photocopier quality in the old days), so underlining differentiates dialogue from sounds. You’ll note that not all sound effects were included. I also split the sound effects up in cue 7 and 8, because these are two quite different cues. If Stan were to cower and run inside, we might approach cue 7 differently. I also didn’t list all the sounds we discussed above, but I included a few that weren’t in the script to remind me. We can fill in the blanks on those as we go.

When it comes to music cues, these may or may not be up to the director, the composer, or to you as the sound designer. Let’s talk about music cues.

We started with a bridge: a bridge is a music cue that plays between scenes without dialogue. Sometimes it’s referred to as act in or act out, as it brings the actors in or out of a stage in a theater production. We also had a fade in bed: we know what a fade in is, and a bed is music that plays under dialogue or action. These are sometimes listed as source bed (in the diegesis, or story-world that the character hears, such as a radio on in their home), or just music bed, which is composed music that the characters don’t hear (nondiegetic). Fade ins differ from fade unders, in that a fade in finishes before dialogue or sound effects start, and fade unders don’t finish until the other sounds have already started.

We also had a sting: this is sometimes called a stinger or bumper. In the old days, when something dramatic was going to happen, the organist would play one or two notes repeatedly to underscore important dialogue and create tension. It’s also used to refer to a quick “blam!” music chord or clip that draws attention quickly then gets out. Bumpers tend to refer to the music specifically used to start a scene or break for commercial, but in a drama, a stinger tends to be a dramatic moment, and may signal the start of a new scene.

Some other music cues we haven’t used here:

There are other things as a sound designer that we may want to add to the cue sheet that weren’t in this scene. If there is a walkie-talkie, or talking from behind a door, we may want to add [FILTER] or [REVERB] to the dialogue line. We may want to also add any emotes—any brief vocalizations, like [GASPS]—as Stan opens the door.

When it comes to sound effects, there are a few things you may encounter that we haven’t talked about before: walla-walla, or just walla, is the term for crowd mumbling, as you would hear in a busy place like a train station, restaurant, and the like. It’s part of the ambience but sets the scene as being busy with people. A ramp is a music cue that leads up to vocals.

In addition to a cue sheet that may be written by someone else, we may get production notes: “Cue 4 on Page 8 should be all in heavy reverb,” or some such note. It is our job as the sound designer to either follow the direction or make a case for why the direction isn’t adequate for our vision, and, in that case, we should record several versions to show the director what we mean and why it’s superior to their vision.

Exercise 9.21 Creating a Cue Sheet

Using the script you used in the previous exercise, go through and create the cue sheet for the entire script. You will need some time to really think about what you’re doing, so don’t rush it.

9.7 The Asset List

Once we have a sense of what sounds we’re going to need, the next step is to create our asset list—a list of all the sounds we need to find, record, make, design, and so on. How we divide or categorize these is up to us, unless we have a producer who has already decided what they need to share with a team of people. We may, for instance, create lists of sounds we know we’re going to have to rely on a sound library for, and sounds we know we can record ourselves. We may also divide them up by scene, or by type of sound (spot, ambience, etc.). Asset lists are typically spreadsheets that help us to keep track of what sounds we need and whether we’ve got them ready to go into production. Asset lists are also great places to start a naming convention for our project. If we know we need “Josie’s footsteps,” then when we slate and name our files, we can name the files something like josies_footsteps_indoor.wav, or Josies_footsteps_02.wav, and so on. Spreadsheets can also be color coded, filtered, and so on, so we can use them to track which sounds are ready, which are rough and need redoing, which are not yet finished, and so on. The conventions we choose are up to us (and our team), but developing and sticking to one convention is really useful. An asset sheet might look something like table 9.1.

Table 9.1 Sample asset list

Scene Sound Filename Category Notes Finished?

Fireplace

Fire crackle

Fire_01.wav

Ambience

Loop. Still needs reverb.

—

Fireplace

Dog whimper

Dog_02.wav

Spot

Get John to voice this.

—

Exercise 9.22 Asset List

Grab your cue sheet from exercise 9.20 and create an asset list for the project.

Exercise 9.23 A Writer’s Description

Ray Bradbury’s story “The Foghorn” describes the sound of foghorns (1983, 434–436): “I’ll make a voice like all of time and all of the fog that ever was; I’ll make a voice that is like an empty bed beside you all night long . . . like trees in autumn with no leaves . . . a sound like November wind and the sea on the hard, cold shore. I’ll make a sound that’s so alone that no one can miss it, that whoever hears it will weep in their souls . . . and whoever hears it will know the sadness of eternity and the briefness of life. . . . The Fog Horn blew. And the monster answered. A cry came across a million years of water and mist. A cry so anguished and alone that it shuddered in my head and my body. . . . Lonely and vast and far away. The sound of isolation, a viewless sea, a cold night, apartness. That was the sound.”

Make the sound of this foghorn based on his description. If you can think of other sounds that are described in books you’ve read, try to create those sounds too. In many cases, we may not be able to talk to the writer to ask them what they were thinking; we can only go by their written descriptions and our own imagination.

Exercise 9.24 Recording a Script: Radio Drama/Fiction Podcast

Take your final cue sheet and record it. If you can’t find actors, you can always read it yourself. We’re just focusing on the sound effects, so the voice is less important here. What did you find in recording your script that you needed to change from your initial spotting session, and why? What did you add/subtract?

Reading and Listening Guide

Clive Cazeaux, “Phenomenology and Radio Drama” (2005)

Cazeaux argues against the idea that by not having image radio drama is an incomplete or “blind” artform. Drawing on phenomenology, Cazeaux shows how sound design in radio drama offers us a complete experience by using the way intersensory perception helps sound to evoke images and touch.

Neil Verma, Theater of the Mind: Imagination, Aesthetics and American Radio Drama (2012)

Verma’s book covers a history of radio drama in the US from a variety of perspectives, including a particularly in-depth exploration of subjectivity and the point of audition (a term he takes issue with). Verma gets quite detailed in descriptions of studio set-ups and microphone techniques used by radio, and overall the book is very useful for another perspective on sound recording for radio, particularly in ways you might analyze and think about radio.

BBC Academy Podcast: “Working with Sound” (May 25, 2017), and “How to Create Stories with Sound” (March 23, 2017)

A BBC podcast series about creating podcasts: a few episodes are specifically about sound and have clips of interviews with some of their sound designers.

Rob Rosenthal, HowSound

A Transom.org podcast about radio storytelling, and many of its episodes are about recording and sound design for podcasts: https://transom.org/2005/walter-murch/.

Radio Drama Revival

A biweekly podcast that’s been going for over a decade now. Each episode discusses the craft of radio drama, but also presents dramas and interviews the people involved in creating them.