Computer Art Morphs into Media Art
One November evening in 1980, people passing through New York’s Lincoln Center were intrigued to find a giant screen occupying a wall there. It was full of larger-than-life people, staring back at them, even talking to them. When the New Yorkers tried waving or jumping up and down or pressing up close to the screen, people on the screen responded. Finally someone asked these apparitions where they were. A figure on the screen answered, “Los Angeles!”
The passersby in New York and at the Century City Mall in LA were all participants in a giant work of media art created by two artists, Kit Galloway and Sherrie Rabinowitz, who called it Hole in Space. Akin to a wormhole which could bring two distant places smack up against each other, it used a high-speed satellite link, and showed that people interacted with strangers at a distance in a far more relaxed way than if they were actually face-to-face. They danced, they clapped, they shouted exuberantly (they had to, to make themselves heard above the noise), they exchanged phone numbers.
The Hole in Space opened up for just three days. Even by the second day, the crowds were vast. Everyone wanted to be in on it.
It was art—not in a gallery but in the street. Computer art had entered the media age.
BY THE 1970S computers were no longer a mystery, but they were still large and expensive. The first really revolutionary development was when the Apple II appeared at the end of the decade. This was a personal computer capable of color graphics. The first modems came out at almost the same moment, allowing digital signals to be transmitted over phone lines.
With the 1980s came an explosion of technological advances. A new generation of personal computers came on the market, less expensive and easier to use, with new software enabling artists to create digital images. In 1982, Adobe Systems appeared with a new line of digital-imaging software, followed by the compact disc. Larger graphics cards were developed leading to Adobe Photoshop, making it easier for artists to experiment with digital art. In 1986, Steve Jobs purchased the Lucasfilm Computer Graphics Division from George Lucas and formed Pixar Animation Studios.
By the 1990s, developments were speeding up. First came hypertext mark-up language (HTML), then Tim Berners-Lee created the World Wide Web. Browsers allowed both text and image to be viewed, and Net Art took off. Interactive art began to incorporate digital technologies.
In 1979, the first Ars Electronica festival of electronic art took place in Linz, Austria. In 1980, the architect Nicholas Negroponte founded the MIT Media Lab to foster the interplay between design and technology. Universities in the US began to offer courses in digital art. New York’s School of Visual Arts, which opened in 1947 as the Cartoonists and Illustrators School, was in the vanguard, instituting in 1986 an MFA degree in computer art. The students who arrived were already adept at the latest technology. “They have computer tools already and so the focus at SVA is on art-making and aesthetics,” says Bruce Wands, the current director of computer education. “Creativity is their number one focus.” Today the SVA has 4,000 enthusiastic students and a teaching staff who are all practicing artists.
Media art and music: Bruce Wands
Bruce Wands is something of a Renaissance man. He’s not only an artist/technologist but also a musician. As part of his current work he records ambient sounds, chooses those that are most melodious, then superposes actual music on them. He is also working on representing geometrical shapes using the proportions inherent in classical Buddhist art, using 3D software to create abstract images that nevertheless convey a spiritual content.
When I enter Wands’s office at the School of Visual Arts in New York’s Midtown, the first thing I notice is his desk, piled high with reprints, preprints, papers awaiting grading, and his own work in progress. When I sit down, he almost disappears from view. Immaculately coiffed, he speaks like the entertainer he once was.
In 1975, after thirteen years on the road playing guitar and bass and taking care of various bands’ multichannel electronic music, he joined the MA program at the Newhouse School of Public Communication at Syracuse University wanting to learn to operate recording studios as well as to study the more technical aspects of music. His instructor was a student of Roy Ascott, Joseph Scala, who taught him to program a powerful IBM 705 mainframe computer to generate images. In those days they still used punched cards to generate line drawings from a plotter. Primitive though this seemed, says Wands, “I saw the future at that point. Computer graphics was it—the wave of the future. You draw better with a computer.” One of his works, dating from 1976, is a white-on-black image of a geometrical form like a snail shell, drawn on a plotter: Heartline.
From Syracuse Wands went to New York, where he created and produced computer animation for the billboard display in Times Square and developed the computer-animated opening for NBC’s Saturday Night Live.
Wands is a great believer in social networks and in Net Art, which, he believes, has gone far to eliminate such “preconceived notions as science being logical [while] art is not. . . . A generational shift has occurred.” Today’s students simply don’t think of making art with computers as different or unusual; they automatically use computer science in their work. They don’t label a project as artsci, but see art and science coming together into something that demands a better classification.
4.1: Bruce Wands, Heartline, 1976.
Wands believes that creativity is intuitional and experiential and emerges through experimentation. As an artist/musician, he first conceives a piece, then writes it down, then experiments. It is in improvisation, freewheeling it, where he sees creativity coming in. “To me improvisation is a non-cognitive process where ‘I let the music play me,’ rather than me playing the music.” As a media artist, he works with algorithms capable of producing new images and sounds with every cycle. Nevertheless, “the artist has control over a wide range of programming parameters, thus giving him the ability to create images and sounds he never imagined.” In traditional art, conversely, the artist has to create the permutations with no computer to help him.
The role of the artist is more to select than to create the perfect image, as in digital imaging, “where high resolution live action video can be shot and the final photograph is selected from the footage.” Whatever seems intuitively to be right must be aesthetic. For Wands the creative thrust is in the process which to him is “primarily intuitive, rather than intellectual.” He points out that creative mistakes often produce the greatest work. Experience primes the mind to hear the melody in, for example, a musical scale wrongly played.
In all of his work, Wands explores and pushes the boundaries of music and art with the help of computers, bringing art and technology ever closer.
Super-real animations: Ken Perlin
In 1982 a groundbreaking science fiction film called Tron hit cinema screens, starring Jeff Bridges as a hacker who is trapped inside a computer and has to battle with the Master Control Program, an artificial intelligence that controls the mainframe, to get out. The film was one of the first to make extensive use of computer animation and included spectacular depictions of the bizarre world inside the computer.
The animation was extraordinarily natural, way ahead of anything that had been achieved so far, but Ken Perlin, who helped to create it, was not completely satisfied. To him, it still had too much of an unnatural machine look. So he began work at New York University’s Media Research Laboratory, which he had founded, and developed a means for producing more natural textures. This works through algorithms that add random textures, known as procedural textures, to the surface, in the same way that natural textures appear random. This is akin to the random addition of signals or noise, hence the name Perlin Noise.
Procedural texture takes advantage of the natural limitations of our brains and perceptions. To create an animated image, Perlin proceeds one step at a time, building up the picture until the image is like an Impressionist painting. Blurry though it is, we can still see the scene sharply. Perlin elaborates. “Take Jane Austen. Bits of her characters are not real, but that makes no difference. She presents enough bits of characters that it will convince you they are real.”
Perlin’s fame spread far beyond academia when he won an Oscar in 1997. The citation read in part, “To Ken Perlin for the development of Perlin Noise, a technique used to produce natural-appearing textures on computer-generated surfaces for motion picture visual effects. The development of Perlin Noise has allowed computer graphics artists to better represent the complexity of natural phenomena in visual effects for the motion picture industry.”
“Needless to say, my mom was very happy,” says Perlin with a characteristically wry smile. Perlin exudes energy. Casually dressed, he is shining, despite the fact that he had been up all night editing. By the end of our interview I wonder if he ever sleeps. Bruce Wands describes him as a genius.
Perlin’s research ranges over art and computer science, with the emphasis on computer graphics, animation, games, physics, and mathematics, plus an active interest in science education. To start out with he was interested in the arts, but eventually “opted for mathematics because [his] English teachers were terrible. How does anyone end up doing what they do?” After a science internship he went into computer graphics. For him this was “where it’s at—inventing new things, aesthetic expression, doesn’t get anywhere better than that.”
“Look around,” he says, “look at something in the world, and ask, ‘What can we do with this? Can it lead to a computer game, a sculpture? Can its properties lead to something new mathematically?’ ”
Some artists say that art is art and science is science. Perlin’s reply is that they start with the hypothesis that there are two sorts of people, artists and scientists. President Obama is always referred to as America’s first black president, but in fact his father was black, his mother was white. The dichotomy is meaningless, just like separating artists and scientists. “Einstein was an artist. The entire universe just happened to match the one he had in his head. Categorization underplays the power of the human brain.”
Perlin adds, “There is a willed ignorance by people who claim no connection with another discipline which they know little or nothing about. They have never engaged in the other discipline and are ignorant of methods and expectations there.” “Great discoveries are made by people with knowledge of other disciplines.”
As for him, does he consider himself artist or scientist? “I’m a researcher,” he replies. For him, as for others, the labels “artist” and “scientist” are obsolete.
Screen magic: Rick Sayre
Rick Sayre, too, is an animation maestro, a supervising technical director at Pixar Animation Studios. He has been involved in various capacities in creating the mesmerizing screen magic of films like Toy Story, Monsters, Inc., and The Incredibles. In 1996, he won a Technical Achievement Award from the Academy of Motion Pictures Arts and Sciences for the Dinosaur Input Device (DID)—now known as the Digital Input Device—that helped bring the dinosaurs in Jurassic Park to life.
Sayre is a bit of an extraterrestrial himself. He looks every inch the sorcerer, with shoulder-length graying hair framing a long, thin face, high forehead, and chromium white complexion, always in black, adorned with earrings, painted nails, and ornate rings on every finger.
Sayre studied electrical engineering, computer science, film, and theater, a highly unusual combination, at the University of California, Davis, completing his studies at Berkeley. “For me, science and technology were only a means to an end,” he says. “I was always interested in the effect it had on me, on how it created emotional states; in a way it’s like the history of magic. That’s what I was always interested in—magic and spectacle, performances and theater . . . particularly that technology allowed me to produce something that was unique, that had never been seen before.” He was working on an animated short film with two classmates when Ed Catmull of Pixar offered him an internship. It was 1987, the year after Steve Jobs had established Pixar, and computer animation was just opening up. It was the perfect moment.
Sayre was inspired by, among others, Oskar Fischinger, the German-born nonfigurative animator. The Nazis condemned his colorful abstract animations as “degenerate art,” and he saw the writing on the wall and left for Hollywood. His talent was recognized by Paramount Pictures as well as by Walt Disney, but his art was years ahead of its time, entirely different from the Mickey Mouse cartoon style of the day. Other major influences were Douglas Trumbull, a special effects pioneer who worked on Stanley Kubrick’s 2001: A Space Odyssey, and Don Coscarelli, known for the cult horror films he worked on such as The Phantasm and Bubba Ho-Tep, “doing shoestring visual effects.”
The film industry operates as “hive mind,” says Sayre. At a typical meeting, people “rattle off things as aesthetic touch points.” Somebody might shout, “Ken Adam,” suggesting that the film being discussed should have the high-gloss, futuristic look that Adam created for the classic early Bond films. Or perhaps it should have the feel of a Japanese anime.
Aesthetics can be put in and played with afterward. “Pixar can try something—test it out before a viewing audience and then alter it. Did it have the intended effect? If not, we change it.”
Sayre goes on, “I don’t have a precise definition of aesthetics. For us, an element of aesthetics is when there is intention behind the work and the intention is to create a certain emotional response in the viewer and that emotional response is going to be motivated by the story and is also going to be influenced by specific desires of the director. For example, we want [Mr. Incredible] to be somewhat stylized, we don’t want him to feel like any particular person, we want him to have this generality and iconography of comic book characters, but not of a particular actor, a means to become, not physical texture, not detail but feeling alive.”
When creating human characters for The Incredibles, the filmmakers considered Pixar’s most successful human character to that point to be Al McWhiggin in Toy Story 2, the greedy store owner who schemes to steal Woody. McWhiggin was depicted with skin covered in pimples and stubble. This worked in close up, but the detail was lost in medium and long shots. For The Incredibles, the director decided against any complicated texture. Simplicity was actually more demanding, in scientific terms. Keeping Mr. Incredible’s face simple required a focus not on skin texture but on the way real human skin responds to light. “It was the most complex of any skin we had ever done—inspired by physics, medical physics, and mathematics.”
Outside Pixar, Sayres collaborates with sculptors, metal fabricators, musicians, dancers, and performance artists to produce interactive artworks. In 1991 he worked with the robotic artist Chico MacMurtie to create Tumbling Man, for which they won the Prix Ars Electronica Distinction. Tumbling Man is a life-sized robot rather like a metallic skeleton, with spine, limbs, hands, feet, and a vestigial head, equipped with a computer. MacMurtie and Sayre were wired up to operate him by means of computers. When Sayre raised his arms, for example, the robot did the same thing. Problems arose when Sayre and MacMurtie made contradictory movements. Trying to obey both sets of orders, the robot ended up rocking back and forth on his back like a tortoise or doing somersaults. Neither Sayre nor MacMurtie knew who was in control. It was the robot who decided which of their instructions could best be utilized.
Sayre is interested in combining the big with the small, the abstract with the figurative. He is concerned that Hollywood, with its blockbusters, has become complacent about animated effects. The interesting developments tend to be in “peripheral areas, particularly underground things like the Demoscene which are really vibrant.”
He directs me toward the website pouet.net, a “wretched hive of scum and villainy.” The Demoscene is a computer art subculture producing computer programs which run audiovisual presentations in real time, typically with no spoken words, flagging up whatever’s new in that world. Members compete to show their latest demos at parties or at large meetings advertised on the website. It is alive with ideas, with creative people bursting with enthusiasm.
Sayre has always found the Demoscene exciting. One member who calls himself Gargaj reports on a gathering in Stuttgart in 2007 where “we met Rick Sayre, one of the technical leaders of Pixar, who exhibited a strange interest in the three foolish kids who were talking about demos endlessly.” Sayre invited the group to visit him at Pixar. They were somewhat doubtful about whether he really meant it, but when they turned up he was true to his word. Despite their cool image, they were thrilled.
Pixar recently hired a member of the Demoscene, who played a key part in how greenery looked in the animated movie Brave. “Fresh blood is a great thing,” says Sayre.
A place where the future is lived: the MIT Media Lab
The MIT Media Laboratory is one of the major world players in design technology. The online course catalogue for the program in Media Arts and Sciences sets out its ambitious program as focusing “on the invention, study, and creative use of new technologies that change how we express ourselves, how we communicate with each other, how we learn, and how we perceive and interact with the world.”
The Lab was the brainchild of the architect Nicholas Negroponte, together with Jerome Wiesner, ex-president of MIT, physicist, and longtime advocate of science education. It was part of the MIT School of Architecture and Planning and opened its doors in 1985. It is intended to be a “place where the future is lived, not imagined.”
The futuristic new building was completed in 2010. The interior is almost entirely glass, box-shaped with a central atrium surrounded by laboratories and offices. The glass interior makes all research visible and encourages networking and collaboration, though some people I spoke to said that while this layout is good for working in groups, for creative work, which requires solitude, they tend to rent studios elsewhere. Nevertheless, the atmosphere is electric, the air alive with ideas.
The director of the Media Lab, Joichi Ito, is a Japanese American who, at forty-five, radiates boyish enthusiasm. He speaks rapid-fire, with a mind like a parallel processor; he’s with you while simultaneously running through other scenarios. Occasionally he stops speaking and his fingers dart over the keyboard of his ever-present Mac.
Born in Japan in 1967, Ito moved to Canada with his family and then to Detroit when he was three. His mother raised Joi and his sister and then, at thirty-five, went to work as a secretary in Energy Conversion Devices Inc., where his father was a research scientist. She rose to vice president, then left to become an entrepreneur. Her get-up-and-go attitude was what fired him.
Ito’s sister became an academic, but Ito found formal education not to his liking. “I always retained the notion of neoteny—the retention of childlike attributes in adulthood,” he says. One of his many enterprises is a venture capital firm which he named Neoteny Co. Ltd. As a committed neotenist, he took the road of the nonspecialist, of continual learning. He studied computer science at Tufts University, near Boston, but dropped out “because I felt I could learn it on my own and better.” He enrolled at the University of Chicago to do physics, but disliked the emphasis on problem-solving rather than understanding concepts and dropped out again.
A succession of jobs followed, which included stints as a disk jockey and the manager of a nightclub, and shuttling back and forth between the motion picture industries in Japan and Hollywood. He finally became a venture capitalist, investing in a string of Internet companies in the USA and Japan in the 1990s at a time when the world of Internet start-ups was wide open. Then, in 2011, MIT asked him to direct the Media Lab. Someone who had dropped out of college twice was a controversial choice, but as the press office wrote, “Ito, 44, is recognized as one of the world’s leading thinkers and writers on innovation, global technology, and the role of the Internet in transforming society.”
“The choice is radical, but brilliant,” says Larry Smarr, who directs Calit2, the California Institute for Telecommunications and Information Technology, a research institution similar to the MIT Media Lab. “He can position the lab at the edge of change and propel it for a decade.” Ito’s understanding of finance was essential for the Media Lab as it struggled to return its funding, which had fallen radically in the downturn of 2008, back to the levels of the dotcom boom. His intellectual qualities were also outstanding. He was outspoken in support of interdisciplinarity and a focus on creativity. He is also well known as an activist, campaigning on behalf of the individual versus government.
The Lab offers a degree in media arts and science—not fine art, which is “not very collaborative,” but art which is about process, the building of things. “Process, not end result, is of concern,” says Ito. In the Lab, artists work with scientists and engineers on an equal footing. To be accepted as a student at the Lab requires a high degree of technological know-how. The Media Lab “is not a tech shop for artists,” says Ito. It is not at the service of artists who would like to have their creations mechanized. Nor is it a place for procrastination. “People spend very little time sitting around doing PowerPoint presentations. Things are different at the Media Lab. People just do it!” Ito puts this credo even more directly. “Shut up and build it, then we’ll talk about it.”
Glamour geek: Neri Oxman
Neri Oxman is an excellent example of the type of person who thrives at the MIT Media Lab. She is interested in load-bearing and has an “aesthetic fascination with forms in nature—form generation as given by nature.” She is researching new uses for concrete, aiming to make possible a bold new architecture going beyond the type of building that can be made today using reinforced concrete. This encompasses engineering and art: engineering in the use of materials and art in the creation of aesthetic objects. Engineering focuses on problem-solving, but in design and art there is not always a particular problem to be solved. She is interested in “problem-seeking, as opposed to problem-solving. Problem-seeking or problem-discovery is a more difficult road to follow than trying to solve already existing problems.”
After studying architecture at the Technion Israel Institute of Technology and medicine at the Hebrew University in Jerusalem, Oxman did a PhD in design computation at the Media Lab. Oxman teaches at the Lab as well as directing the Mediated Matter research group. She was featured on the cover of the high-tech magazine Fast Company, which described her as ushering in the “era of glamour geeks,” and added, “smart is sexy.”
In order to look into the way concrete shapes and supports buildings, Oxman studies how calcium shapes load-bearing bones, a process that “produces unbelievable patterns.” In her research, she “tries to spec out algorithms that describe this conversation between matter and distribution of loads.” This is best understood as algorithm plus context. The algorithm contains aesthetic reasoning: will calcium produce certain shapes for certain distributions and can this be applied to the distributions of concrete? The context is the conditions confronting the architect.
Oxman comes up with an illustration. Ansel Adams, the pioneering photographer, wrote books explaining his techniques. But even if you use these techniques, you won’t produce photographs like those Adams took; the recipe is there, but the results will differ. The recipe—Adams’s description of his technique—is a creative algorithm. Every time it is used, it produces something different. Creative algorithms are often used to power extruders to produce three-dimensional art. Algorithms are everywhere, they are in the air, they are the zeitgeist. At the turn of the twentieth century, the zeitgeist was the quest to redefine the classical, intuitive ways of understanding space and time. Einstein and Picasso both responded to this. As for the zeitgeist today, “now, with the Internet, it’s about the group, the community,” about collapsing distances and times. “I miss the old world,” Oxman adds.
One of Oxman’s best-known products is a chaise longue she calls Beast. Its single organic-seeming surface serves as the structure as well. It’s a wonderful undulating creation which adapts to different bodies, i.e., different loads, varying its density and flexibility. “Beast lets go of boundaries such as those between structural support and comfort support. All is blended via an algorithm.”
4.2: Neri Oxman, Beast, 2008–10.
Oxman has used her studies in the distribution of materials suited to bear loads to create a variety of products including the prototype “carpal skin,” inspired by the patterns on animals’ coats. It is made up of organic material distributed so as to protect against carpal tunnel syndrome, a painful nerve condition in the wrist.
In her research comparing the way calcium shapes bones to the way concrete shapes buildings, Oxman works with material scientists and biologists. She studies biological specimens using CT scans (computer tomography, which uses x-rays and computers to create detailed images). The images have to be set up differently for material scientists and for biologists, who have different requirements. Eventually, education will have to be revamped to accommodate new fields of research such as hers. Biologists will have to be trained in material science and vice versa. “A da Vinci stage has to be reached. The greatest scientists of all time were artists,” she says, repeating the last words emphatically.
Art has been redefined, and so has its role. Oxman mentions “iPad fever. You build a tool box and use it over and over again and it is shared by everyone.” She “doesn’t believe that that is the role of art, which is to question a certain reality rather than to provide entertainment.” Design is art, she says. The best way to think of design is as “translation between disciplines.”
A peek into the future: Michael Bove
“The Media Lab is an idea factory,” says Michael Bove. With his khaki trousers, button-down shirt, immaculately parted hair, and MIT graduation ring, Bove looks conventional enough, but behind this starchy appearance lurks a restless and creative mind. In his office every available surface is covered in books, papers, and gadgets.
Walking around his laboratory is like taking a peek into the future. Students and researchers sit around asking “what if?” questions, such as, “What if the world were literally in touch?” They’ve already come up with a possible way to achieve this—prototype Touch Gloves, made up of sensors containing information that is shared when you shake hands with someone else also wearing Touch Gloves.
Can you transform an object into a phone? Bove showed me the prototype for an object which looks like a bar of soap with “dimples” on it and a screen. It becomes a phone if gripped one way, a camera if gripped another way, and a recorder if gripped yet another way.
Can you use technology to teach baseball? Bove’s team rigged up a baseball with sensors to teach a beginner to throw a fastball or a curveball.
Can we capture our dreams without having to jump out of bed and look for a pad of paper? Pillow Talk is a voice-activated recorder sewn into a pillow. The dreamer can store his narration on disk to be replayed later.
“Look, I don’t know if this is art. I don’t know,” says Bove. If anything, his team’s highly imaginative inventions could be called interface art, in which the performer or subject triggers a computer through movement.
Bove found his calling early. As a high school student in the late seventies, he built a personal computer, attached it to a digital camera which he had also built, and started producing computer graphics. At MIT he specialized in electrical engineering, but he was also interested in art and hung around the architecture department. Nicholas Negroponte, soon to found the Media Lab, met him and was impressed by him. Bove was the first student to do a PhD at the Lab. He now heads the Object-Based Media Group.
Bove also collaborates with artists on holographic art, holograms being two-dimensional surfaces that interact with incoming light so as to make three-dimensional shapes appear outside the plane of the surface. He is currently looking into developing a holographic television which would project images into your living room as a “cloud” which could be viewed from any direction without the need for 3D glasses. The holographic images could literally take over your living room—though this is far in the future.
Bove’s chief collaborator on this project is the Australian holographic artist Paula Dawson. She has contributed to scientific papers on holography and, Bove has noted, has developed a “totally radical way of making large scale optical holograms,” digital holograms which are computer-generated using a “holoprinter” device.
One of the problems in developing a holographic television is to do with questions of perception in viewing it. For a start, where exactly is the picture plane? In a painting, it is the canvas, the physical surface, but in holography it is the nonmaterial plane between the electronics producing the image and the viewer. Then there is the matter of how best to develop the image’s three-dimensionality and color.
Dawson makes suggestions from her perspective as an artist about how the image should best be displayed. A major stumbling block is how to define the boundaries of the image. Looking at the image close up (in the middle of your living room), it would need to be contained in a frame, such as a task bar with decorative side columns, similar to a computer screen (see Insert).
Bove is enthusiastic about the collaboration. “Apart from the fact that Paula is creative and energetic and it’s generally inspirational for my group to get together with her, she has particular interface and display requirements for her holographic drawings that have caused us to rethink certain aspects of our research and system design for interactive holographic video displays.”
Dawson and Bove provide an excellent example of an artist-scientist collaboration working to benefit both.
Digital rhythms: Joe Paradiso
Joseph Paradiso’s lab at the MIT Media Lab is an electronic madhouse. Musical synthesizers, old and new, share floor space with ever more sophisticated equipment and computers. Tall, lean, and gray-haired in black jeans, a checked shirt, and sneakers, Paradiso pours out words rapid-fire. He is, to put it mildly, an electronic genius. At one point he was offered three prestigious positions at the same time—at CERN in experimental particle physics, at the Draper Lab at MIT in space science, and at the Media Lab. He chose the Media Lab because “it’s all about creativity here.”
Alongside his scientific research, Paradiso is a musician who builds and plays synthesizers and jams with ad hoc groups. His original massive synthesizer, probably the largest in the world, with a complex configuration of wires connecting its hundred-odd modules, has the place of honor in his lab and is slated for the MIT museum. Paradiso began work on it in the early 1970s and it took him a decade and a half to complete, starting from scratch, scrounging around electronic surplus stores for parts. Paradiso has also designed Musical Instrument Digital Interface (MIDI) systems for operating several instruments from a single control system.
He is generally involved in seven or eight projects at the same time within the Responsive Environments Group he directs. One of the group’s prototype inventions is a pair of shoes housing batteries that charge as you walk, thus generating energy. Another is a pair of shoes with a sensor system that creates musical tones in response to the wearer’s movements. Rhythmic movements result in pleasing sounds, encouraging the wearer to move more gracefully.
4.3: Joseph Paradiso, Expressive Footwear, 2000.
Paradiso’s group has also invented a Digital Baton, which generates the sounds of different musical instruments depending on the pressure with which you hold it. It’s a technological magic wand.
At the frontier of media art: Peter Weibel
“Today art is an offspring of science and technology,” says Peter Weibel. Weibel is the chairman and CEO of Zentrum für Kunst und Medientechnologie (Center for Arts and Media Technology, known as ZKM) in Germany, a curator, an art theorist deeply versed in the classics, art history, and intellectual currents such as structuralism and psychoanalysis, and a professor who regularly gives workshops at universities across the world. From 1988 to 1995, Weibel was artistic director of Ars Electronica, a digital arts organization that produces a gigantic annual fair at Linz devoted to science- and technology-influenced art. He’s also a practicing artist. In the late 1960s, he was a member of the Viennese Actionists, a group of wildly bohemian artists who practiced what they called the politics of transgression, using their bodies as canvases, covering themselves with paint and even cutting themselves with razors. He continues to be a video artist. His 2003 video work Venus in Fur, showing historical portrayals of women by male artists, is the only work of contemporary art at the Belvedere, the National Gallery of Art in Vienna.
Weibel speaks authoritatively on a huge spectrum of topics, ranging across art, science, technology, art theory, and theater. I meet him in Vienna at the opening of a multimedia installation showcasing the art-based research project Quantum Cinema—A Digital Vision, of which he is the director. The aim is to use digital art to explore the quantum world as well as visualizing higher-dimensional geometries. The conference takes place at the University of Applied Arts, where he is a professor.
Weibel has a huge, charismatic presence. His independence of mind springs from a turbulent youth. Born in Odessa in 1944, he was a displaced person after World War II, living in refugee camps in terrible conditions. Growing up in state institutions, he turned inward and read voraciously. He devoured Dover Publications’ science books and George Boole’s An Investigation of the Laws of Thought, which turned out to be not about thought at all but about Boolean algebra, a form of mathematical logic. Boole’s book led Weibel into information theory. Then he discovered the French poets, in particular, Paul Valéry, who had studied books by the nineteenth-century English scientist Michael Faraday. Weibel did likewise. In school he enraged his science teachers by telling them that what they were teaching was wrong because they had failed to include quantum physics.
In the early 1960s, he studied medicine at the Sorbonne in Paris. Always intellectually restless, he also began reading works by the linguist Roman Jakobson, the anthropologist Claude Lévi-Strauss, and the psychoanalyst and philosopher Jacques Lacan. Medicine began to seem rather narrow once he realized how art and technology played off each other, and in 1964 he began making films. He also immersed himself in the history of the cinema.
On returning to Vienna in the mid-1960s, he found himself acknowledged as an expert on cinema. It was then that he became involved with the Viennese Actionists. It was obvious that the best way to revolt against society and the establishment was as an artist. Science was too well established. Still, Weibel found the technology involved in developing and editing film too cumbersome. Then in 1969, video cameras became available. Now he could express his “ideas about space, time, and relativity much easier,” showing them in real time using simultaneous images side by side.
Weibel always works, he says, “in dialogue with scientists.” In the 1970s he corresponded with Claude Shannon, a towering figure in information theory, with Marvin Minsky, the director of MIT’s Artificial Intelligence Lab, with John McCarthy, who was well known within computer and cognitive science, and with René Thom, an internationally renowned French mathematician. Recently, he approached Anton Zeilinger, a physicist at the University of Vienna known for his experiments on entanglement. Once two particles on the atomic level interact, they become entangled: they forever sense what has happened to the other, like the Corsican brothers in the novel by Alexandre Dumas, separated at birth but affected ever after by the other’s experiences. Einstein called this “spooky actions-at-a-distance.” Zeilinger’s group and others have shown that it is real. Weibel was convinced that what Zeilinger was doing “could be wonderful media art” and invited Zeilinger to speak on his experiments, complete with videos, at the 2012 Documenta Festival in Kassel.
4.4: Peter Weibel, Imaginary or Virtual Tetrahedron. Closed-circuit video installation, 1978.
To Weibel’s way of thinking, aesthetics is something our minds construct from the complex interplay of our senses. In his video Virtual Tetrahedron, he shows a metal frame with no particular shape, but then an actor appears and inserts a screen with a strategically placed line on it, and a tetrahedron magically appears. He also manipulates two projectors to produce a virtual tetrahedron. In neither case does the tetrahedron—one of the five Platonic solids deemed to have a degree of perfection—actually exist. It is an optical illusion, entirely constructed by our mind. “It exists only thanks to the media.”
“Aesthetics is a general medium of questioning the world,” he says, in this case of playing with perceptions to bring out unexpected results, hidden information. “For me aesthetics has therefore nothing to do with beauty, but with information,” he continues. “Many of my works are not directly linked to reality, but to aesthetic problems.”
Weibel has found that art historians and curators are often hostile to artworks involving science and technology. “It is ridiculous that art historians are not educated in science and technology because artists are much more advanced.” He senses fear and resentment against ZKM, which successfully showcases science- and technology-based art.
He has crusaded for science to be included in the art history curriculum. The example he likes to give is of “how photography deeply influenced painting,” arguing that many of the effects artists now use were inspired by photography, that there was “no single painting out of focus before photography.” He also refers to the artworks of Andy Warhol and Gerhard Richter, created by manipulating photographs.
Like it or not, our world is driven by science and technology, which are based in theory. So, he says, “I give art a strong theoretical, political, and social urgency. And I make difficult shows.” Art, he says, is “theory-dependent because it wants to be part of the modern world.”
Daring and imagination: Dunne and Raby
Fiona Raby and Tony Dunne are a dynamic globetrotting husband-and-wife team who oversee the imaginative and daring Design Interactions group at the Royal College of Art in London. According to Dunne, the philosophy behind the group is “just about anything goes. Let your mind wander.” “We are interested in using design as a medium, to ask questions and provoke and stimulate people, designers, and industry,” he has said.
In selecting students, they place a premium on creativity, imagination, and innovation, and look for candidates who are “highly committed to playing a significant and meaningful role in the shaping of our technological future.” Incoming students need not “have a high tech savvy,” says Raby. “They can be pure artists.” The biggest thing they need is imagination.
The Design Interactions group is a playground for ideas, where “technological dreams” can be played out. Raby and Dunne’s goal is to introduce people to technologies and “mess up all those very nicely thought-out technological systems. There is a need for the irrational, illogical, and contradictory to be considered equally.”
Both Dunne and Raby lived in Tokyo for three years in the late 1980s, and were inspired by its heady mix of technology and imagination embedded in everyday life. “I don’t think we ever recovered!” says Raby. Coming home, they found “no playfulness in England.” They frequently return for inspiration and in 1994 established the design partnership Dunne & Raby.
“Can designers be involved in biology?” asks Raby. An early project in the field of design biology was Biojewellery by Tobie Kerridge and Nikki Stott, design researchers at the Royal College of Art, working with Ian Thompson, a bioengineer at King’s College London. In the future it will be possible to grow bone tissue outside the body to be used in reconstructive surgery. But supposing it could be used to make jewelry? Supposing a couple could express their love by wearing rings made of each other’s bone tissue?
4.5: Tobie Kerridge, Nikki Stott, and Ian Thompson, Models of Trish and Lynsey’s Rings, 2006.
To carry this out, the group used bone tissue from wisdom teeth extracted from a particular couple, mixed it with precious metals, and cultured it on a bioactive ceramic used as a scaffold for growing cells. The results—two rings of bone mounted on etched silver—were exhibited at Guy’s Hospital, London, in 2007. The process involved many different disciplines: materials engineering, cell biology, oral surgery, medical imaging, computer-aided jewelry design, graphic design, interaction design, product design, fine art, media relations, journalism, science communication, sociology, and ethics.
Commercial design is not particularly speculative, in that commercial designers have to sell their products, but the work at the RCA’s Design Interactions group very often is. Students look into the idea of growing hair on a host for future use, for example, or alternative uses for an Anglepoise lamp.
David Benqué, another member of the team, has a project he calls “Acoustic Botany, A Genetically Engineered Sound Garden.” His starting point is that forms in nature evolve not to suit particular human needs but to satisfy irrational desires. “Beautiful flowers, mind-altering weeds, and crabs shaped like human faces all thrive on these desires, giving them an evolutionary advantage.” So perhaps we could genetically modify plants for the same purpose. Supposing, for example, there could be “a controlled ecosystem of entertainment [which has an] aesthetic relationship with nature.” In the future, synthetic biology might be able to actually make this possible. For now, Benqué sculpts the swelling shapes of fantastical plants and fruits wired up to produce haunting melodies reminiscent of the music of the spheres.
Projects such as these help to break down barriers between disciplines. Teams made up of biologists, engineers, mathematicians, chemists, and designers work together. “They all sit at the same table and communicate with each other, appreciate each other’s skills,” says Raby. “While the designers may know nothing about science, they sit with scientists, ask questions. They question dogma.”
Synthetic biology explores the design and construction of entirely new biological systems. It is not biology-influenced art, but media art. Alexandra Daisy Ginsberg is a key player in this field and gives dazzling TED lectures. In 2009 she and another designer, James King, joined a team of seven undergraduates at the University of Cambridge in a project to introduce color into bacteria. Taking genes from a variety of organisms, they designed sequences of DNA known as BioBricks, which they inserted into E. coli, a bacteria common to our digestive system, treating the cell like a computer and the cell’s DNA like computer code—in other words, reprogramming the cell.
The result was a new bacteria which the team called E. chromi, capable of producing colors in response to specific chemicals, which made it a potential diagnostic tool. When E. chromi is inserted into a patient’s digestive system, it can change the color of the feces if certain diseases are present, thus sounding the alarm. It could also be used to check whether water is safe for drinking by turning it red if it isn’t.
Ginsberg and King speculated on other uses for the new technology—dreaming dreams, as Ginsberg had learned at Design Interactions. Perhaps E. chromi might be used in food additives or personalized medicine, in antiterrorism, in organisms that excrete oil or plastic, or for analyzing or encouraging the emergence of new types of weather. The E. chromi project won the grand prize at the International Genetically Engineered Machine competition at MIT in 2009.
Synthetic biology is rife with ethical issues. Who owns the new genes? Where do we stop? Ginsberg sees part of her role in the team and in the field as “that of a provocateur—an activist.” Synthetic biology “tampers with nature,” making her ponder the “seductiveness of technology, blurring the boundary between the natural and unnatural, outrageous scenarios such as engineering myself.”
In media art, artist and scientist are fused together. Using complex mathematics, media artists generate pleasing images. These artists are of necessity technologically savvy, and closer to the scientist than those schooled in the traditional fine arts, an area which is becoming increasingly marginalized. In this new avant-garde, the labels “artist,” “engineer,” and “scientist” are growing increasingly irrelevant, often replaced by “researcher.” These artists distance themselves from the traditional art world, which shies away from their art, and instead take advantage of the upsurge of venues that exhibit and sell artsci, such as ZKM in Karlsruhe, Ars Electronica in Linz, GV Art in London, Le Laboratoire in Paris, Science Gallery in Dublin, and Documenta, which takes place every fifth year in Kassel. All these allow artists working on the fringes of art, science, and technology to bypass the traditional art markets. Five years ago there were only a few biennials dedicated to science. Now there are over fifty.
As to how artists can finance themselves, says Peter Weibel, “Private industry will finance them.” In this way, “art will become independent, no longer dependent on markets.” Property rights will become an old-fashioned notion in an open-source world where all software is on the Web and cannot be copyrighted. Ultimately, he predicts, there “will be an empire of data.”
What has been emerging is an emphasis on the process rather than the end result, especially in terms of the creativity involved, which is often guided by algorithms. Aesthetics honed by experience enters as a way of selecting the proper route and product. Despite the highly mechanized methods that media artists use, they place a high premium on intuition and imagination, which resist explanation using logic. As Weibel put it, “Today art is an offspring of science and technology.”