37 The World’s First Computer-Composed Musical: Beyond the Fence
Could you get a computer to write a musical?
—Catherine Gale1
In 2014, Catherine Gale, previously a researcher in computational biomedicine and now a producer/director at Wingspan Productions in London, made two television programs that, at the time, seemed unconnected. The Joy of Logic, a foray into logic, explored Alan Turing’s belief that both brains and computers are information-processing systems, which means that one day computers will be able to reproduce human thought. The other, Our Gay Wedding: The Musical, celebrated Benjamin Till and Nathan Taylor’s wedding on the first day that same-sex marriages became legal in the United Kingdom. Till and Taylor, both musicians of note, composed the music and lyrics.
Thus inspired, Archie Baron, Wingspan’s creative director, came up with the visionary idea of staging a musical composed as much as possible by computers, with computers generating the ideas and writing the lyrics and music. In 2015, Gale recalls, “Sky Arts was looking for big, ambitious, unusual projects, and they had a development fund for genre-busting subjects.”2
Baron’s inspired idea entirely fit the bill. Besides the musical itself, he envisaged two one-hour programs telling the story of how it was put together. To everyone’s delight, Sky Arts took the bait and commissioned it. Gale began by putting together a consortium of scientists from around Europe and hired Taylor and Till to curate the lyrics and the music, which the computers would compose, and assemble them into a show.
The first step was to identify a recipe for success—the key ingredients that made a hit musical. Using machine learning, James Robert Lloyd, Alex Davies, and David Spiegelhalter of Cambridge University conducted a predictive big-data analysis of successful musicals. They analyzed 1,696 musicals and 946 synopses, extracted the key features, and identified the four most popular themes: a journey, aspiration, love, and a lost king. Cast size, backdrop, and emotional structure also played a part. The protagonist, they concluded, would have to be female. The most popular periods for the setting were Europe in the 1930s and 1980s. There would have to be a death, a happy ending, and plenty of dancing. They also carried out surveys, asking audiences to watch fifty-two representative shows and report their reactions in a sort of computer-assisted market research. The viewers elected to have a happy ending and a peak in positive emotion about halfway through act 2.
The next player was the marvelous and quirky What-If Machine, designed by Simon Colton (who developed The Painting Fool) and his team at Goldsmith’s University in London.3 The What-If Machine is a spinner of story ideas. It takes an unlikely scenario and plays with it. It’s obvious that a dog can’t ride a horse—but what if it learned? What if there was a cloud that had bars in it instead of water? Rather than enjoying its shade, you could buy a beer there. Thus it brainstorms plots and ideas—many outlandish, but some tantalizing.
The data from the Cambridge thematic analysis and the seven basic plots as drawn up by academic Christopher Booker—rags to riches, the quest, rebirth, and so on—together with musical synopses for hits and flops were all translated into what-if sentences and fed into the What-If Machine. Six hundred plot ideas came pouring out. At this point, the human element came into play. Taylor and Till sifted through the ideas and homed in on one: “What if a wounded soldier had to learn to understand a child in order to find true love?”
They typed this phrase, along with other statistically important elements of a hit musical, such as love and loss, into a search engine. One of the responses that came up was the songbook from the Greenham Common Women’s antinuclear peace camp in the 1980s. From this, Taylor and Till created a story line about a soldier posted at Greenham Common nuclear base to recover from his wounds, where he befriends the mute child of a protestor and falls in love with the child’s mother. They also came up with a name for the musical: Beyond the Fence.
But how to develop the plot? For this, Taylor and Till turned to Pablo Gervás at the Universidad Complutense in Madrid. Gervás’s work involves generating a whole story from a single plot line dreamt up by, for example, the What-If Machine, using his storytelling algorithm PropperWryter. PropperWryter builds sequences of plot elements, such as boy meets girl, boy loses girl, villain kidnaps victim. It knows that if it inserts one of these elements into a plot, that will introduce tension, and also knows that a story can only end when all tensions have been resolved. Gervás adapted it to suit a musical and worked up the core narrative arc of the new show, adding data from musicals and text from literary sources, annotated by music students and the Wingspan Productions team to indicate how they should be used.4
Next came the music. Bob Sturm and his team at Queen Mary University of London conducted a computational analysis of the music of musicals. They fed in seventy-seven full-length recordings of musicals, over 130 hours of audio in all, and isolated elements such as loudness, brightness, tempo, dynamics, and key, trying to work out what kinds of music characterized a hit and what kinds inevitably led to a flop.
Nick Collins at Durham University then fed all this data—chord changes from hit musicals and rules for combining chords taken from his own experience as a musician—into his computer composition system, based on Markov models, which he dubbed Android Lloyd Webber.5
François Pachet also fed songs from the Greenham Common songbook together with the songs Nick Collins had generated into his Flow Machine to create more songs.6
Android Lloyd Webber could produce two hundred lines of music in seconds. But humans—namely, Taylor and Till—had to review it. They played through every line, listening for melodies that they liked. Till, a paper-and-pen composer, found this disturbing. He told Collins that when he wrote music himself, his soul was in it. Collins replied, “My soul is in the program which generated the music. When you die your music dies with you. But my program will go on producing music forever.”7
For the lyrics, Alex Davies and James Robert Lloyd fed seven thousand lyrics from musicals and ten thousand poems, with parts of Wikipedia thrown in, into their lyric-generating artificial neural network.
Clarissa the Cloud Lyricist, as they called their algorithm, spewed out lyrics that Taylor and Till, in conjunction with Kat Mace, an editor of computer lyrics, used to fashion into songs. Trawling through the music and lyrics was painstaking work. Although each line Clarissa produced more or less made sense, it was rare for two consecutive lines to make any sense at all. In the end, only about 25 percent of the lines in the songs were produced by machines; in one song, only 6 percent was used. The rest were crafted by Taylor and Till.
Gale recalls how Taylor and Till were determined to make the show “as good as possible with what they had, rather than injecting new material into it.”8 If they had stuck with lyrics written by the computer, it would have been an entirely different show. As an experiment, in the first rehearsal the cast was given a piece called “Time Is on the Tree,” made up entirely of computer-generated lyrics and music. Gale sent the lyrics to Collins, who generated music for them. The first few lines run:
Time is on the tree
He’s a man who can shame
A shadow of a melody
Me has so that the shade.
Difficult to sing, much less to memorize! Taylor and Till emphasize that “in a musical nothing is left to chance.”9 Every note, every word must support the story, and many people are involved—writers, musicians, cast, producer, director, choreographer, and more.
And so, after a month of rehearsals, the show went on. Beyond the Fence opened at the Arts Theatre in London, on February 22, 2016. It ran for fifteen performances, and over three thousand people came to see it. Audience members were asked to complete questionnaires, and there was an overwhelmingly positive response. Strangely, some were entirely unaware of the role computers had played in the production.10
Critics focused on the experimental nature of the production and the fact that it was created by machines. But despite their initial reservations, the reviews were generally positive. “A unique experiment in musical theatre composition,” wrote the reviewer for the Stage. “Despite my reservations I was won over,” said the Independent. “Extremely moving and emotional,” concluded West End Wilma.
Leading lights of London journalism and theater, such as Ian Hislop, editor of Private Eye, and Alain Boubil, lyricist for Les Misérables and Miss Saigon, went in as skeptics and emerged converted. The music and lyrics were “comparable to what any of us have written,” Boubil remarked.11
The main flaw that critics homed in on was that the musical was if anything too mainstream, not to say bland; it fulfilled every criterion for a best-selling musical: hardly surprising, as its creators had programmed their computers to do exactly that. Clearly, neither the computers nor their operators had any idea of what the Greenham Common protest had really been about and simply used it as the background for an anachronistically feel-good musical.
The computer scientists involved felt that there was too much human curating, although most were impressed by how difficult it was to produce a West End musical. Adhering to the demands of this particular genre, François Pachet tells me, forced the algorithms to work within “the category of songs for musicals, a category with lots of constraints that they wanted to have. They did it and did it well; see the reviews. But we really wanted songs that had some surprising unconventional elements. We wanted to push the artists beyond their comfort zone.”12
Bob Sturm, who carried out the initial analysis of the music for the project, tells me that he “never imagined it would be as successful as it was. … At the beginning I thought it was going to be the strangest musical in the world.”13
Pablo Gervás, who developed PropperWryter, was also uncomfortable with the large amount of human intervention. “The actual musical is far removed from what the computers generated,” he says.14
Nick Collins, inventor of Android Lloyd Webber, agrees. But he found the experiment rewarding and pioneering, too. “Musical theatre composition itself has not been the prime subject of previous research in algorithmic composition, but it deserves wider future investigation.”15
Till, the more critical of the writer/lyricist team, was inspired by the possibilities of machines and people working together. It can “lead to ideas that we would never have known,” he says.16 “Nathan Taylor cried when Clarissa was turned off,” Gale recalls.17
Regarding Android Lloyd Webber, Collins later received a “legal letter from a well-known musical composer concerned at the use of a parodic version of his name, and seeking to stop this under trademark law.”18 What the composer found derogatory was the association of his name with a “mechanical process,” implying that his music was not the result of a “creative process.” Collins points out the contradiction in trying “to stop a program on commercial grounds from producing output that could be confused with that of a human, and at the same time being so worried as to denigrate the program’s capabilities in emulating creativity.”19 Collins provided a “gentle response”—a piece made up of music by Android Lloyd Webber and lyrics by Clarissa.20 As he says, what all the computer scientists involved in Beyond the Fence wanted was for computer modeling of music to throw some light on human musical creativity.
Indeed, in Beyond the Fence, machines and humans share equal billing, as is clear in the playbill in figure 37.1.
Playbill for Beyond the Fence, February 2016.