32 Ross Goodwin and the First AI-Scripted Movie
Our whole goal in Sunspring was not to curate.
—Ross Goodwin103
My name is Benjamin.
—Benjamin (previously Jetson)104
Ross Goodwin works at “the intersection of writing and computation.”105 He has had an extraordinary career. After playing about with computers as a child, he studied economics, then became a speech writer for President Obama, writing presidential proclamations, then took a variety of freelance writing jobs.
One of these involved churning out business letters—he calls it freelance ghostwriting. The letters were all pretty much the same, so he figured out an algorithm that would generate form letters, using a few samples as a database. The algorithm jumbled up paragraphs and lines following certain templates, then reassembled them to produce business letters, similar but each varying in style, saving him the job of starting anew each time.106 He thought he was on to something new but soon found out that this was a well-explored area. But it did pique his interest in the “intersection of writing and computation.”107 He enrolled in the master’s program at NYU’s Interactive Telecommunications Program (ITP), where, among others, he worked with Allison Parrish.
One of his first creations was the remarkable word.camera, which is online for anyone to play with. It takes a picture—of you or of whatever you are holding up to be photographed—identifies what it’s seeing, then generates words—poetry—sparked by the images it has identified. Show it an image of, for example, mountain scenery, and it might come up with seven or eight descriptive phrases—“blue sky with clouds,” “a large rock in the background.” Then it uses each to generate a sequence of words: “A blue sky with clouds: and a sweet sun carrying the shadow of the black trees and the spire are dark and the wind and the breath in the light are.” A little mysterious, but no more so than a lot of contemporary poetry.
Here is an example of an image (figure 32.1), in this case the great sixteenth-century artist Albrecht Dürer’s 1514 engraving of Saint Jerome in His Study. It’s a period interpretation of a genius as someone touched by divine guidance, transmitted by the light that shines on the saint.
Albrecht Dürer, Saint Jerome in His Study, 1514.
And here is what the machine sees, the poem generated by word.camera when presented with this picture:
A Stone Wall fills a sea,
A shadow of rivers,
A spirit of light;
and in the water
Some storm that sweeps the wind and the darkness.
A large brick wall with a slow view of the trees,
The town of the house where it was done with water,
Having passed a Spring-fire.
A large rock on the ground is dark and shadowy,
and the whole world is dead.
To create his word.camera, Goodwin took a neural network machine containing Char-RNN, which predicts sequences of characters based on an input corpus, and trained it on a vast number of twentieth-century poems and a large number of images, all captioned. Then he rigged up a digital camera to it. The computer writes poems based on what it sees in the image. Goodwin was gratified that the poems “conjured imagery and employed relatively solid grammar,” while having the aura of originality. “What good is a generator if it just plagiarises your input corpus?” he asks, adding: “A machine that can caption images is a machine that can describe or relate to what it sees in a highly intelligent way.”108
Then, in 2016, Goodwin met Oscar Sharp, a British film student at NYU who preferred hanging out with the technologists at ITP rather than with filmmakers. Sharp wanted to create a film by splicing together random parts of other films, like Dadaist artists had done. But Goodwin had a better idea—to use Char-RNN. Their call to action came when they heard about the annual Sci-Fi London Film Festival and its 48-Hour Film Challenge. For this, contestants are given a set of props and a list of lines of dialogue that must be included. They have to make the film over the next two days, and it can be no more than five minutes long.
Goodwin fed his Char-RNN, which they named Jetson, digitized screenplays of sci-fi films from the 1980s and 1990s, TV shows like Stargate: SG1, and every episode of Star Trek and The X-Files. While Goodwin, Sharp, and the cast stood around the printer, Jetson spat out a screenplay, complete with seemingly impossible stage directions, such as, “He is standing in the stars and sitting on the floor.” Sharp decided this called for a dream sequence—though some of Shakespeare’s stage directions, too, like the classic “Exit pursued by a bear,” from A Winter’s Tale, are as obscure as any written by Jetson.
Another stage direction called for one of the characters—H, who is speaking to C—to spit out an eyeball:
C
(smiles)
I don’t know anything about this.
H
(to Hauk, taking his eyes from his mouth)
Then what?109
Sharp randomly assigned roles to the three actors in the room: Thomas Middleditch from the HBO series Silicon Valley (H), Elisabeth Gray (H2), and Humphrey Ker (C). The actors read through Jetson’s script, interpreting the lines as they read, turning it into an involving if somewhat bizarre sci-fi story of love, despair, and murder. There is even a musical interlude that Jetson composed after learning from a database of twenty-five thousand folk songs, which comes toward the end, during a dreamlike action sequence. The song was curated and performed by vocalist Tiger Darrow and musician Andrew Orkin, who “chose lines that worked well with their music.”
Sharp kept curation of the script itself to an absolute minimum. He cut some lines to keep the movie within the five-minute limit for the challenge. One scene was removed because it called for a truck and they had none. “Oscar Sharp did not edit it for consistency or linearity or for any other qualities,” Goodwin tells me.110 In the film, which they called Sunspring, they went the whole distance: actors spoke exactly the lines that the machine gave them.
Sunspring is a sci-fi drama with the characters’ costumes augmented with metal and gold. It’s certainly bizarre—even Lynchian, one might say. The actors deliver their somewhat gnomic lines with passion and intensity, putting meaning and expression into them and making them make sense, even if the script itself does not. And in the course of the film H does indeed spit out an eyeball. For Goodwin, it’s up to the actor or the reader to provide the meaning. Pablo Gervás also pointed out that this is an essential part of creative writing: “The magic doesn’t come from the writer, it comes from the reader.”111
People may be less prone to appreciate prose that makes no sense, but this is not the case for movie dialogue or song lyrics or poetry. As Goodwin says, modern readers may have difficulty understanding certain lines of Shakespeare, but they make sense when actors flesh them out with their body language and emotion.112
The judges at the film festival placed Sunspring in the top ten, though it did not win. Nevertheless, Jetson was interviewed on stage at the award ceremony, responding through text on Goodwin’s computer. Here are two of its responses:
Interviewer: What do you think of your historic nomination in this contest?
Jetson: I am pretty excited. I think I can see the feathers when they release their hearts. It’s like a breakdown of the facts. So they should be competent with the fact that they won’t be surprised.
Interviewer: What’s next for you?
Jetson: Here we go. The staff is divided by the train of the burning machine building with sweat. No one will see your face. The children reach into the furnace, but the light is still slipping to the floor. The world is still embarrassed. The party is with your staff. My name is Benjamin.113
Rather touchingly, Jetson had told them its name and henceforth Goodwin and Sharp referred to it—or him—as Benjamin, and billed him as such as the writer of Sunspring.
For the moment, Goodwin sees the role of computers as assisting us and providing intelligent augmentation. But he believes that we are getting close to machines creating on their own volition. “That sort of creativity,” he says, “need not replace human creativity and will work in conjunction with human creativity. I don’t see why one has to replace the other.”114
Benjamin, along with Goodwin and Sharp, went on to create a second movie, entitled It’s no Game, with David Hasselhoff as the Hoffbot. And Goodwin was snapped up by Google’s Artist and Machine Intelligence Program as a creative technologist.
He has now published a book entitled 1 the Road, which he describes as “a novel written using a car as a pen, an enhanced AI experiment.”115 It is an AI version of an American road trip akin to Jack Kerouac’s On the Road. For this, Goodwin took an AI for a ride. He traveled from New York to New Orleans in a Cadillac equipped with four sensors: a surveillance camera on the roof, a GPS unit, a microphone, and the computer’s internal clock. All these were hooked up to an AI trained on poetry, science fiction, and what Goodwin describes as “bleak” literature, as well as on location data.
Like a very advanced version of Goodwin’s word.camera, the AI produced words in response to what it saw and places it passed and to conversation it overheard in the car. The opening sentence came spewing out of the printer as Goodwin turned on the machine in Brooklyn: “It was nine seventeen in the morning and the house was heavy.”
The AI continued to generate somewhat disjointed but evocative sentences throughout the journey, such as in this excerpt:
It was seven minutes to ten in the morning, and it was the only good thing that happened.
What is it? the painter asked.
The time was six minutes until ten o’clock in the morning, and the conversation was finished while the same interview was over.116
Responding as it does to sights and sounds along the way, the book really does read like a rather surreal road trip novel.