16 Hod Lipson and Patrick Tresset’s Artist Robots
We’re like alchemists of the past, driven by the passion for knowledge.
—Hod Lipson169
In Isaac Asimov’s I, Robot, the inspector asks a robot whether it has ever turned a canvas into a masterpiece. The robot replies, “Have you?” It was this exchange that inspired Hod Lipson to begin working with robots.
As director of Columbia University’s Creative Machines Lab, Lipson has done a great deal of work on developing “self-aware” robots. He has developed software that can derive complex equations and thus enhance research, as well as robots in the form of simple structures that can replicate themselves with available parts, learn on their own, and evolve over time without human intervention. But his private passion is to develop a robot that paints. He has now built three painting robots: SEURAT, Pixasso, and PIX18. He built the first, SEURAT, with his then graduate student, Carlos Aguilar.170
The robots consist of an articulated robot arm, a digital paint simulator that predicts how a paint stroke will look if it is painted on a canvas, and an algorithm that instructs the robot arm in the correct brushstrokes to replicate a given painting. Lipson shows the painting robot a digital image, which it analyses and represents with actual brush strokes and paint on canvas. From time to time, he stops the process and culls those images that are too far removed from the original. Finally, the robot arm produces an image pleasing to the subject and to the operator of the program.
The results are rather spectacular. Figure 16.1 shows a painting created by PIX18.
Tear, inspired by Roy Lichtenstein’s Frightened Girl. Lipson attributes the work to PIX18 and himself thus: PIX18/Hod Lipson, 2016.
Since Aguilar graduated, Lipson has not been able to persuade another graduate student to work on his robot art project. “Engineers shy away from art,” he says. “If you pay for an engineering degree, you want on your résumé that you were designing rockets, et cetera—serious stuff. But if you’ve made oil paintings, maybe you won’t find a job. So this is one project I’m doing myself, almost as a hobby.”171 Lipson’s second robot, Pixasso, painted large paintings in his basement.
When he moved from Cornell University to New York City, he built PIX18. Lipson firmly believes that computers can be creative and have volition. Just as the invention of photography drove adventurous artists to invent impressionism, so too “AI will push the art world to invent new forms of art,” he says.172
He would like to shift the balance of creativity “almost entirely to the computer,” so that eventually it paints out of its own life experiences using its own imagination, which will require machines to have emotions, too. Why should “creativity be the last bastion of humanism?” he asks.
“GANs are absolutely the path forward for machines to create their own things,” he says. GANs also include an internal critic to eliminate the mass of unwanted material that pours out when machines generate art. Lipson speculates that eventually the art produced by machines will be impossible for us to understand. “But that won’t be for a thousand years from now.”
Robot art is becoming increasingly popular. Besides Lipson’s robot artists, there is the German collective robotlab creating vast drawings173 and the Czech artist Federico Diaz turning out enormous and spectacular pieces of sculpture.174
And in London, there is Patrick Tresset and his sketching robots, collectively called Paul. Paul is a robot arm attached to an old-fashioned school desk, inside which is a laptop that is its “brain.”175 Its arm holds a pen, and there is a camera “head” that looks at and photographs the sitter. Then Paul gets to work.176 The process is akin to Lipson’s, except that the images the robot uses to assess its progress were drawn by Tresset himself. The software is designed to emulate the perceptual and cognitive processes involved when artists sketch faces.
Tresset instructs me to sit absolutely still while Paul sketches me. I sense something awry here and decide to move. Paul carries on sketching; there is no effect on his movements, as in figure 16.2. In his wry French way, Tresset admits that asking subjects to sit still while Paul’s camera moves is “a bit of theatre; the camera just needed to look once at you.”177
Paul sketching the author, 2017.
It’s rather endearing to see a classroom of Pauls all working away busily together. Tresset programs them so that some are eager to work, whereas others seem hesitant and shy. In this way, he says, he goes beyond the art dimension to create “little pieces of ‘theatre’ that touch people. I would say that I am inspired by Jacques Tati, Samuel Beckett and Antoine de Saint-Exupéry.”178 Tresset tells me that galleries pay his robots “like performers. … And the bookings go quickly.”179
Recently, experiments have shown that people, as one would expect, tend to appreciate art made by computers more when the computer is housed in a robot that looks human, which suggests it has a mind like ours. This is the case with Tresset’s robot, Paul. People are more prone to appreciate Paul’s art if they stay in the room while their portrait is being drawn, rather than leaving and coming back later to see the finished work. It seems the process of creating the artwork is important for us to appreciate art produced by robots, as is the robot having human characteristics.180
Tresset is adamant that machines “cannot be creative. Real art is done by human beings. Drawings done by robots are not art.”181