5 Sound Effects

We’ve already looked at some space- and time-related sound effects, or digital signal processing (DSP) effects. But there are lots of other digital effects we can play around with that have different impacts on our sound. A simple analogy is with photos and Instagram filters: you have an image, but then you use filters, or effects, to modify the image to enhance or alter that image. We can create different feelings or enhance or alter a sound by adjusting the sound with effects. Effects can add a lot to our subjective experience of sound, and are a staple of sound design. In this chapter we will focus on frequency effects, such as modulation, equalization, and filters. There are other effects that we’re not going to cover in this book, and you’ll find yourself gathering software plugins over time that are designed for specific purposes, but we’ll cover some of the most common effects. I encourage you to download and explore other effects available. Now that you have a basic understanding of sound waves and digital sound, the documentation that each effect has on the Audacity website should start to make some sense.

5.1 Tremolo and Vibrato

Tremolo modulates the amplitude of a sample back and forth, giving it a “trembling” effect (tremolo is Italian for “trembling”). You can modulate the speed at which the tremolo works, and also the depth, or how much amplitude is adjusted. Tremolo can give a feeling of “liveness,” pulsation, or movement (figure 5.1).

5.1.1 Digital Tremolo

You may need to download the tremolo plugin to use tremolo. Generate a sine wave as we have done many times now (Generate > Tone). Apply the tremolo effect by selecting Effects > Tremolo (just select the default for now and hit OK). Notice the amplitude of the wave has changed to a wavy pattern, although the frequency is the same (figure 5.1). Open the Plot Spectrum (Analyze > Plot Spectrum) to check that there is no change in frequency.

Audacity has several settings for tremolo:

Waveform type alters the shape of the amplitude pattern: triangle creates a triangle wave pattern and square creates a square wave pattern.
Starting phase sets where to start in the waveform cycle, so zero is the start of the cycle (as the waveform rises toward its peak).
Wet level sets the depth of the amplitude sweep from zero to maximum of the original amplitude volume.
Frequency controls the speed of the tremolo’s oscillation (how “trembly” it is), not the frequency of the file.

Exercise 5.1 Tremolo

Try some different tremolo effects on different types of sounds. How does it change the feel of the sound? Can you find examples of tremolo effects in some music you listen to? Why do you think the artist chose to use tremolo in those songs? See if you can find an example of its use on vocals!

5.1.2 Vibrato

Tremolo is often confused with vibrato. Vibrato modulates a frequency back and forth (from the Italian vibrare, “to vibrate”). Vibrato gives a pulsating change of pitch, and can add depth and interest as well as added emphasis or expression. The “whammy bar” on an electric guitar is a vibrato effect, although sometimes people call it (wrongly) a tremolo arm.

5.1.3 Digital Vibrato

You’ll need to download an extra plugin to get the vibrato effect in Audacity. Apply the effect using the default settings (Effects > Vibrato > OK). This plugin will currently only allow us to apply vibrato to a small sample (~2.3 seconds), so you will have to select a small part of the waveform. Unless we’ve zoomed in, it won’t look like anything has changed, but listen to the change. If we zoom in, we can see in the waveform how the vibrato effect has changed the waveform: the amplitude remains the same but the wavelength of the wave (i.e., the frequency) has shifted (figure 5.2).

Exercise 5.2 Vibrato

Try out some vibrato on different samples. Where do you think these might be useful for sound effects, rather than music? Find some examples of vibrato used in music. Why do you think the artist chose to use vibrato in those songs?

5.2 Pitch Shifting and Auto-Tune

A pitch shifter does what its name implies—it shifts the pitch (the musical term for frequency) up or down. Pitch shifting in music is most commonly used in association with other effects, like ADT or chorus, or to correct a singer who is slightly off-pitch, but it can also be used for all kinds of sound design purposes, like creature vocalizations. We know that changing frequency will change the length of time, since the oscillations become tighter or longer. A sound of 60 Hz (remember, that’s cycles per second) altered to be 30 Hz (30 cycles per second) should be twice as long, because the cycles are being stretched. Audacity’s pitch change plugin will attempt to adjust the time length to match the original while still changing frequency. This can create sometimes interesting, and sometimes unwanted, auditory artifacts (figure 5.3).

An obvious pitch correction with only part of the signal corrected results in an effect that sounds like an Auto-Tune (since this is what an Auto-Tune does!), and has more recently become a stylized popular vocal effect to deliberately distort vocals. Although it’s used generically to describe that obvious pitch correction, initially made famous by Cher’s “Believe” (1998), Auto-Tune is a brand name, a processor developed by Antares Audio. In the case of the famous Auto-Tune effect now common on songs, the pitch correction speed is set too fast, creating that little distortion in the adjustment. Auto-Tune takes a few milliseconds to kick in on each note, giving the voice the slightly swirly sound.

Exercise 5.3 Auto-Tune

Find three examples of songs that use Auto-Tune: it shouldn’t be hard with pop songs today! Why do you think the artists are using the effect—what impact does it have on the listener?

Exercise 5.4 Pitch shifting

Take three sound samples and pitch them up or down in small or large increments (Effects > Change Pitch). Audacity gives two options for changing frequency. The first is for musical pitch, and it will guess the pitch your sound is closest to, and ask you what pitch you want it to be. You can use this, or use the second half of the window below where you can deal directly with frequency (which is more detailed, granular correction). How might you use this effect in your own sound work?

5.3 Equalization

An equalizer or EQ adjusts the amplitude of specified frequencies within a track. Whereas volume knobs boost the entire sound file, an EQ will let you boost or attenuate (reduce) frequency bands within that sound file. Imagine if we recorded a track and found that the bass seemed flat. We can EQ the mix so the bass is increased. Remember our frequency response charts (section 1.3.1)? If we know there is a dip in the 500 Hz range of our output, we can EQ it back up. You’ve probably used EQs on your stereo system, as a simple knob that adjusts the bass and/or treble, or as faders (slider inputs) (figure 5.4).

There are different types of EQs, and they can vary greatly in precision or visualization, but commonly have some of the same properties. The goal of an EQ is to modify the frequency spectrum, by boosting or attenuating specific frequencies or groups (bands) of frequencies. These are adjusted on a slope, rather than a harsh notch of frequencies, so if you pick 500 Hz to boost, it will boost on a slope up to and down away from 500 Hz, not just 500 Hz.

The slope is typically set using dBs per octave (a musical term representing a range of frequencies): the greater the dBs, the harsher or tighter the slope. The bandwidth is the set of frequency bands (the width of bands) being adjusted by the slope. A wider bandwidth will boost a wider range of frequencies to either side of the selected frequency.

The Q value is a ratio of the center frequency to bandwidth: as we increase Q, we lower the bandwidth, or sharpen the range what is being adjusted. The Q value is not shown in the standard Audacity equalizer plugin, but you may find it on other equalizers.

If we open up the first standard equalization effect in Audacity (Effects > Equalization), we will see something that looks like a blank Fletcher–Munson graph. The graph is flat because this is your starting point. It is not a spectrogram, so it does not represent where our frequencies are currently in relation to each other. There are a number of presets we can apply under the Select Curve dropdown. Select the Telephone preset and take a look at your graph now. It is showing that anything below about 130 Hz is going to be reduced to –30 dB and we won’t hear it. There is a slope up to 400 Hz where we will hear everything in our original file at its original amplitude, and then at about 2,500 Hz it begins to slope down—or drop off—again. Preview the file and listen. We can adjust the curve by dragging the tiny Bezier points. Beside EQ Type on the bottom left, select Graphic instead of Draw, and we’ll see some fader sliders: now we have no Bezier points but we can adjust frequencies by moving the sliders.

Exercise 5.5 EQ Presets

Try some of the different presets (select curve) built in to the Audacity equalizer. What does each preset do? Write down the effect it has on the frequencies as you both hear and see them.

Exercise 5.6 The Best Frequencies

Grab a long sample or one you’ve recorded of some length and play with the EQ until you think you’ve made a significant improvement to the file. What frequencies did you boost or cut and why?

5.3.1 Digital Wah-Wah

The wah-wah, sometimes known as the “Jimi Hendrix” effect for his heavy use of this technique, is a variable EQ, where the boosted frequency alters according to a sweep of the frequency band. The wah-wah is actually an instance of onomatopoeia, based on the effect it makes. The original intent was to mimic a trumpet using a mute (a device stuffed into the end of the trumpet to adjust the amplitude and to a lesser extent the timbre/tone). A wah-wah gives a weird in-and-out wave effect as the frequencies are swept, using a low-frequency oscillator, or LFO. If we apply it to a single 500 Hz sine wave, we get a really weird-looking wave-form (figure 5.6): in this you can see the amplitude moving in and out in a wah-wah-wah-wah.

Exercise 5.7 Wah-Wah-What?

Apply the wah-wah to some sound samples and note what you discover about the sound and its effect on you. Try to find some musical examples of the wah-wah effect: usually you’ll hear it in psychedelic rock or funk from the 1970s.

Exercise 5.8 Supersonic Hearing

You may notice the EQ band goes up to 40 KHz on professional equivalent. Knowing that human hearing cuts off at about 15 KHz for most adults (20 KHz if you’re young), why would the bands go up to 40 KHz? This is a mysterious perceptual phenomenon in that even though people can’t hear above 15 KHz, they appear to hear the difference between a file where the upper frequencies have been chopped off and a file where they haven’t! You know where your hearing cut-off is. Boost the frequencies above that cut-off in an EQ and see if you can hear the difference in the sound file. Give an A/B test to a friend to see if they can hear the difference (an A/B test is a simple comparison between two objects where the person being tested doesn’t know which one is which).

Exercise 5.9 Turn It Down!

Earlier we discussed a weird perceptual phenomenon related to our hearing : bass requires a higher amplitude to appear at the same perceptual volume as the mids (see section 1.3.1). The mid-frequency range becomes flattened out as we increase amplitude, leaving the perception that low and high frequencies are louder, or boosted in the signal. We can create the illusion of loudness by boosting the highs and lows, and leaving the mids where they are. Try it out for yourself.

5.4 Filters

Filters can work like EQs, in that they can boost or reduce (or even remove) frequencies in a set range—in fact, some EQs, called shelves, are essentially filters. A filter circuit behaves more like a sieve, allowing some frequencies through (“passing” them) and holding back others. Filters aren’t designed to boost a signal, they only cut.

Low-pass filters are designed to allow all frequencies below a particular cut-off frequency to pass through, while preventing other frequencies from passing (figure 5.7). Low pass filters have the effect of creating a “muffled” sound, much like what we might hear underwater, behind a window or door, or from a distance.

High-pass filters are the opposite of low-pass filters. They cut frequencies below a set point and allow the higher frequencies to pass through (figure 5.8).

Band-pass filters cut the highs and lows, and let a middle band of frequencies pass through. These can be useful for isolating a sound (figure 5.9).

Notch filters (sometimes called “band rejection”) cut a notch out of the frequency spectrum. These can be really useful when we have an annoying sound (e.g., tape hiss, air con) and we know its frequency (figure 5.10). We used a notch filter earlier when removing the fundamental to show a phantom fundamental (section 2.5.1).

There are other types of filters, which are less commonly in use. High- or low-shelf filters boost or reduce frequencies in a particular range, but don’t cut out frequencies. Comb filters are what gives the phaser its distinctive sound, but instead of sweeping the amplitude, comb filters are a harsh on-off, like the teeth of a comb. We can imitate a comb filter by making a “FFFFFFF” sound with our mouth, and putting our flat hand on and off our mouth. It has a quick on-off pattern caused by the continued interference on the signal. There isn’t a built-in comb filter in Audacity so we have to download one to explore it.

Filters are sometimes used for effects (such as the comb filter), but even more often they are used to correct audio files by eliminating certain problems, such as a buzz or a hum, pops, tape hiss, or other unwanted background sound. If we can figure out the frequency of the refrigerator or air conditioner in the background on our recording, for instance, then we can filter out that frequency without messing with the other frequencies.

Exercise 5.10 Exploring Filters

Choose different filters and set different cut-off frequencies to hear the differences in how they impact the sound file. Here is where you’ll find the Q value we talked about earlier: the slope of the curve.

5.5 Modulation: Ring Modulation and Vocoder

A ring modulator is a type of modulation effect that creates a metallic or robotic sound. Ring modulation is created by taking two sound signals and multiplying them together to create two new frequencies that are the sum and difference of the inputs. So, if you were to put in two sine tones at 500 Hz and 600 Hz, the output would be tones at 1,100 Hz (600 + 500) and 100 Hz (600 – 500) played at the same time with the original tone—in other words, we’re not just hearing the combination tones in our ears (section 2.5.1), but they’re now part of the file. So if we generate a sine wave at 500 Hz and select a modulator at 600 Hz, we get a new waveform that, if we open up the plot analysis graph, we can see has generated sounds at 100 and 1,100 Hz. You’ll need to download a plugin for the ring modulator. (Generate > Tone > 500 Hz, Effects > Ring Modulator > Modulation frequency: 600 > OK) (figure 5.11).

Usually we aren’t inputting sine waves, but one sound source (known as the modulator), which is then mixed with a sine or square wave (known as the carrier) as the secondary input signal. The result in more complex sounds is that the harmonics created tend to not be even, so the result can sound metallic, hollow, and bell-like, particularly at higher frequencies, which produce more audible resonance.

Exercise 5.11 Ring Modulator

Record your voice and some other sounds, and apply ring modulation. How does the ring modulator affect the way it sounds? Go back to the section on difference tones (section 2.5.1) and create the difference tones with your ear and then with the ring modulator on a sine wave. What is the difference between what is generated by the ring modulator, and what is generated by your ears?

A vocoder, or voice encoder, also uses a carrier and modulating wave, and can sound a bit like a ring modulator. Vocoders work across frequency bands, so a broad frequency spectrum is required, particularly in mid to high content (which is to say, it won’t really work on a sine wave, but needs a sound with lots of harmonics). Typically the modulator is a voice, but it can be used on other sounds. The modulator is split (grouped) into many bands of frequencies, and each band is sent through a filter set to the same frequency that was analyzed. The carrier wave is then sent through the same band of filters, and filters out elements in the original signal by imposing the dynamics (an envelope) and spectral content of the modulator. The effect is pitched voice (if using voice), where the input signal will follow the synthesizer signal’s frequency.

Exercise 5.12 Vocoder

Grab some sound samples or record your own voice. You’ll need a stereo file for applying the vocoder. If you didn’t record in stereo previously, you can change this in the drop-down menu beside your microphone. Apply the vocoder. What is the difference as you hear it? How might you use this effect?

5.6 Distortion: Overdrive and Fuzz

Overdrive, fuzz, and distortion create an artificial distortion of the sound, effectively clipping either the fundamental or the harmonics of the sound. The effect is a result of the sound being overdriven—the sound volume is pushed too high, causing the signal to distort. The result is the addition of and/or boosting of harmonics to the sound—a little bit of distortion and it can sound “warm and fuzzy,” a lot and it can sound harsh and “dirty.”

Some people distinguish between overdrive, fuzz, and distortion, with overdrive typically described as a “soft clipping,” or subtler effect, and distortion as “hard clipping,” a crunchy sound, and fuzz as even harder or more aggressive clipping and saturation of harmonics. Often, however, these terms are just used interchangeably. Technically, overdrive and fuzz are both types of distortion, where overdrive is subtler and is “cleaner” at lower amplitudes, boosting saturation as the volume increases. EQing (boosting) the harmonics of a sound, rather than boosting the main signal, is one way to create a saturated, distorted effect without increasing the overall amplitude.

Soft clipping doesn’t cut the waveform off so much, as instead it rounds it out at the cut-off (0 dB: figure 5.12), whereas hard clipping, as we saw in chapter 2 (section 2.4.2), cuts the tops and bottoms of the waveform off.

The human body overdrives its voice at times of anger or anguish: we have to yell quite hard to overdrive our voice. The resultant effect of overdrive, then, can be a perception of anger, anguish or excitement. Adding a little distortion—boosting the harmonics of the sound—can also increase the perception of loudness without actually increasing loudness, which can lead a sound to feel either warmer (if used in moderation) or angry/loud. Because a lot of harmonics will push a sound toward loud noise, the effect is used a lot in genres of heavy metal, punk, and industrial music.

5.6.1 Digital Distortion

Audacity has many settings for distortion. There are preset types that will alter the harmonics or other aspects of the distortion type. The distortion type will determine which parameters you can adjust in the window below: some sliders will be disabled for some distortion types.

Clipping level: peaks greater than the level set are clipped off.
Drive: the amount the signal is amplified prior to being clipped.
Make-up gain: the amplification of the final output.
Distortion amount: the amount/strength of distortion.

Exercise 5.13 Distortion

Try some different distortion effects and note for yourself how the sound appears to change (Effects > Distortion > Distortion Type). Change the settings for each type, as well.

5.7 Summary and Bonus Exercises

We’ve looked at a wide range of effects now. Knowing what effects do to a sound and the impact that has on the perception of that sound is really important, and is a staple of sound design. While many of the effects are based on analog tape-recording techniques and effects pedals that were experimental in the 1960s and ’70s, today sounds are usually recorded dry and digital effects are used to adjust the sound to what we want. In this way, we can have more range and more easily adapt effects to what we want. Quite often, effects are used not in isolation but in combination with one another. To create a walkie-talkie effect, for instance, we can use a band-pass filter to filter out the highs and lows, and add distortion to “dirty up” the sound. Try to experiment with combinations of effects to mimic various effects you’ve heard in movies, songs, and so on.

We’ve now got a lot of experience applying different effects, so let’s put them to creative use with some exercises!

Exercise 5.14 Guess the Effect (Partner Exercise)

Grab a partner. Use the same sound sample and manipulate it using a different effect. Swap your results, and see if you can then mimic your partner’s effects by guessing what they did. If it’s too easy, add a second effect on top of the first, and try again. Then a third effect, and so on.

Exercise 5.15 Change a Sound

Take a sound and put it through as many effects as you need to turn it into another sound. For instance, try to change the sound of a kitten into the sound of a dog. Try to change the sound of a running car into the breathing of a person. Log your steps in effects chosen.

Exercise 5.16 Build a Sound from Memory

Think of a sound from memory. Record or download a similar sound and then use effects to try to adjust the sound until it matches what you had in your memory. Log your steps in effects chosen.

Exercise 5.17 Glitch

“Glitch” a sound: apply some effects on a sample so that it appears to have been a glitch. A glitch might be a tape deck sound that suddenly slows down part of the track, or a digital glitch like a quick skip, aliasing, or other sounds that make it sound like a mechanical error.

Exercise 5.18 Clean Up Aisle Audio!

What happens if we have a sound effect sample that has some artifacts we don’t want? We can edit it knowing what we now know about filters and effects. Find someone’s really bad recording online (for instance, a podcast, a YouTube video) and try to improve their sound as much as you can. Don’t forget to write down your steps taken.

Exercise 5.19 Subtraction

Take one of your field recordings. By only subtracting sounds from it using filters in different places, turn it into something else (you can’t move or alter the sounds in any other way).

Reading and Listening Guide

Jonathan Weinel, Inner Sound: Altered States of Consciousness in Electronic Music and Audio-Visual Media (2018)

Weinel’s book explores altered states of consciousness (drugs, dreams, trance, etc.) and how they are represented in and potentially induced by audiovisual media, with a focus on music. A fascinating read of how we virtually re-create physical sensations.

Dave Tompkins, How to Wreck a Nice Beach: The Vocoder from World War II to Hip-Hop, The Machine Speaks (2010)

Tompkins follows the history of the vocoder from its invention in 1928 to today. The title takes its name from a misheard vocoder recording: “How to recognize speech.” It’s a bit haphazard, but with some interesting historical gems and lots of photos.

Karen Collins, “Sonic Subjectivity and Auditory Perspective in Ratatouille” (2013)

In this article, I explore how a subjective perspective is created for the audience in the film Ratatouille. For example, in one kitchen scene, the sound designer Ben Burtt creates a subjective position for the audience by the use of low-pass filter and reverb effects when we are situated “with” the mouse under the metal colander.

Stalker (dir. Andrei Tarkovsky, 1979)

The story for this film is about an expedition led by a guide taking two clients into a mysterious restricted site known as the “zone.” The scene to focus on takes place on a railroad cart from the known, natural world into the zone. One man asks the guide if they will get caught, and is told that the police won’t go in the zone because they’re afraid. The question “Afraid of what?” hangs in the air for about four minutes as the visuals just cut back and forth between the two clients. The scene is carried by the sound of the rail car: as they get deeper into the zone—and as we move deeper into their mindset—an increasing amount of effects are placed on the sound, so that by the end of the scene they are saturated with phasing and filters and reverb.

Music Listening

Psychedelic drug effects were common in 1960s psychedelic rock music. For a sample, see the Spotify playlist on the studyingsound.org website. What sound effects did they use?