24 Marwaread Mary Farbood Sketches Music
What you see is what you hear.
—Marwaread Mary Farbood125
Mary Farbood’s Hyperscore is a beautiful piece of software that links sound to images—color, shape, line, texture. It offers a way for someone entirely without musical training to compose music, intuitively visualizing and editing musical structure, linking sound and vision.
Farbood started as a pianist—but a bad case of tendinitis brought an end to her musical career. Instead of continuing at Juilliard, she went to Harvard, where she studied computer science and took up the harpsichord. She moved on to the MIT Media Lab, where she had the chance to follow up on her interests in both computer science and music.
At MIT, she studied under Tod Machover, a dynamic composer and professor of music. One of the aims of the Media Lab is enablement—enabling adults and children not trained in musical composition to write music. Farbood’s Hyperscore played a key role in Machover’s Toy Symphony project, in which children composed musical pieces, some of which were performed by orchestras.
Farbood is not the first to try combining visuals and sound. Earlier graphic music-makers include Morton Subotnick’s Musical Sketch Pad, enabling users to “sketch” musical works by selecting and combining the sounds of different musical instruments, and William Duckworth’s PitchWeb, an interactive work of music and art. Hyperscore uses graphics instead of musical notation to teach the essentials of music composition.
The first versions of Hyperscore involved simply generating scores by pushing a button. In later versions, users have more control. The software maps complex musical concepts onto colors, shapes, and textures, enabling users to control pitch, dynamics, melodic contour, and harmony.126
The user makes this happen in two stages. You click on notes, then drag and drop them onto a screen in the form of teardrops. Their horizontal extension determines how long the note is held, pitch is determined by their vertical position on the screen, and their position along the horizontal determines the order in which the notes are played. The result is a melody, to which you assign a color.
The next step is to create compositions from this melody or melodies of different colors. This takes place on another screen, which Farbood calls the harmonic palette. If you’re using a single melody designated as blue, then by clicking and dragging you will generate a blue line on the palette. A straight line gives a single pitch. Lines at an angle give variable pitches. Drawing many lines, some overlapping, results in cacophony. You can use an automated harmony line to impose a chord progression. To start, the harmony line is horizontal, but you can curve it by clicking and dragging. The inbuilt chord progressions are taken from a database of chords from Bach chorales; Farbood is currently enlarging the database to include chords from jazz and popular music. She also uses Markov chains to generate series of harmonic functions. In this she has been much inspired by David Cope.
Users enjoy drawing lines freehand on the computer screen, manipulating them until a satisfactory melody emerges: you literally sketch a piece of music. As Farbood writes, Hyperscore “tries to push the concept of ‘what you see is what you hear’ in a score as far as possible.”127 What Farbood is looking for is a “computational model that includes the various features of tension.”128 A work, be it of literature or music, has tension if it possesses a narrative arc: a beginning, a middle, and an end. This requires repeating melodies in different ways, looking back over a work. “People have an intuitive idea of what tension is,” she says. They look for the “release” when a chord is resolved or when melodies are reprised in various ways. The audience listens with anticipation for elements inherent in musical structure, such as expression, dynamics, and harmony, and feels fulfilled and moved at a dramatic ending. Thus tension “enhances emotional content.”
Farbood has currently stopped work on Hyperscore to study the extensive literature on tension in cognitive psychology so as to understand better how people perceive music. “There has never been a model that takes all this into account and combines them into an empirically valid tension curve,” she says. She is exploring from a cognitive perspective what we actually do when we listen to music and has put together a model that focuses on short-term memory and windows of attention. She plans to return to Hyperscore soon and enlarge its scope so as to create a version with “very high-level ways of assisting composition.”129