This essay was for a large anthology on dance and the moving image. It followed a time when I was making many presentations, and encountering many of the same questions in different contexts. These were the answers I ended up wishing I’d made at the time.
The virtual dance collaborations I’ve been involved with have triggered unexpectedly intense interest not only in the dance and computer fields, but also in the popular press. People have the sense that the intersection of art and technology is now particularly important, especially when their point of overlap is the body. As a result, I’ve been invited to a lot of conferences, where my goal has been to wave off any mystification that might obscure the work. If there is going to be any mystery here, let it be in the art itself, not in its technical realization or in vague theoretical formulations taking off from it.
The most interesting moments at those conferences come when questions are posed. Usually they’re raised by members of the audience, though sometimes they occur to me on my own afterwards, in my hotel room or on an airplane. Here are some I keep thinking about.
Why the interest in abstracting motion from the body?
It’s true that motion-capture is a process of subtraction, of taking away. The infrared cameras have eyes only for the reflective markers worn by the performing bodies, and not for the bodies themselves. Right away we lose all vision of muscle and flesh, and with that all sense of effort as well, since we can no longer make out the actual and sweat of the performing body. The face also vanishes, and with it the expressions that signal intention and feeling. Thoroughly stripped away are the dancers’ stage presences: their physical beauty and charisma. What good is this?
One way to answer that question is by posing others. Is there beauty in motion seen all on its own, without seeing the body that created it? Do the virtuoso performers on stage distract us from a more ineffable beauty that we sense only vaguely when watching them? Can we force it into focus by squinting, as it were – peering through some new lenses that technology has just given us?
But this talk about seeing motion without the body is misleading. Granted, motion-capture recording produces mere data sets, collections of numbers – but software immediately converts these into visible form, into something that makes sense to our eyes. At the very least, we translate the captured motion into white dots corresponding to the markers placed on the body. As soon as those dots start moving, we sense the body implied by them, a curiously palpable form in the black void of the screen.
Most often the motion-capture data sets are then translated into a solid 3D simulation of the body. In the program we use, Character Studio, this body is called a biped (which inspired the Cunningham dance title). Most conventional animators model the bipeds into ever more realistic figures, simulating flesh and muscle and even face. The more realistic the appearance, however, the more artificial the feeling. The digital double falls well short of the real thing, making us most aware of the technology’s inadequacy.
So when I first started working in 3D I asked myself, why photographic realism? Isn’t it true that drawing lets you see and feel more? Doesn’t what you leave out of a picture show as much as what you put in? My aim in my work with Shelley Eshkar was to make a hand-drawn body: an expressive figure that you could see through like an x-ray; an animated frame that fully revealed the motions mapped onto it.
Where we sought complete realism was in the motions themselves. After all, as Cunningham wanted to know, who’s interested in a head that can spin around three times? (Answer: Hollywood.) Eshkar and I kept abstracting our figures further – into dots, sticks, curves, poles – but we took care not to distort the underlying motions: no hundred-foot leaps, no impossible pivots.
We insisted that the motion be true, but you can question that. True in what way? Motion capture is accurate, perhaps, but when a performer’s motion is captured, doesn’t it suddenly occupy a radically different kind of time? Isn’t it now close to timeless, the very opposite of real performance, whose essence is that it’s “water running through the hands,” as Cunningham says? And haven’t we also made it “spaceless,” since we can now put it down virtually anywhere? (I admit to feeling that the dematerialization of the stage space in BIPED was our biggest accomplishment there.)
“Is that me?” Bill T. Jones wanted to know on first seeing his motion-capture dots on the screen. Good question: who is the figure that’s moving, once the motion is disembodied? All of [Ghostcatching][1] spun out of that enigma, as we multiplied Jones’s identity through a series of spawns, one figure begetting the next. By contrast, Cunningham cheerfully ignored the issue. In our first collaboration, [Hand-drawn Spaces][2], he let his chance operations combine the movements of Jeannie Steele and Jarrod Phillips, obliterating any differences in body size, sex, and self.
In every dance conference, a few hands are raised with questions about notation. I think it’s clear that motion-capture, combined with video, will soon replace paper-based notation schemes like Laban and Banesh, which few can read anyway. But what are the criteria for good notation? Cunningham says that it must be visual: dancers learn by looking. I wonder whether it shouldn’t also be creative. That is, can’t dance notation be as powerful as music notation, which not only lets a musician play, but also lets a composer compose?
This motion-capture session was part of a longer exploration of dance preservation, initiated & funded by the Estate Project for Artists with AIDS.
Notation brings us back to the question of identity. I remember ballet master Suki Schorer’s reaction to a trial motion-capture we made of Balanchine’s Harlequinade . It’s actually useful, she said, that you can’t make out the individual body, only the motion. Students will start concentrating on mastering the movement itself, not on imitating the star.
What do choreography and computer science have to tell each other?
When I ask filmmaking students at liberal art universities what they know of Alan Turing and Joseph von Neumann, I usually get puzzled looks. And blank stares when I mention John Cage and Merce Cunningham in computerdom.
How curious that at mid-century we had Cage / Cunningham in the arts and Turing / von Neumann in mathematics addressing many of the same questions. To my knowledge, there was no crossover between them, but a multitude of their preoccupations –random access, chance, emergent structure – are amazingly similar. One of my students, trying to illuminate this overlap, unearthed a wonderful quote from Turing:
An interesting variant on the idea of a digital computer is a ‘digital computer with a random element’. These have instructions involving throwing a die or some equivalent electronic process… Sometimes such a machine is described as having free will.
This relates to a question I’m frequently asked about Cunningham’s legendary use of chance operations. What exactly does he do, and why? I can only report on what I saw during our Hand-drawn Spaces collaboration, after we’d motion-captured 71 phrases of movement. To put them together into choreographed sequences, he simply rolled his dice. He trusted that chance would find things that would never have entered his mind otherwise.
John Cage started using chance operations to get beyond the self, to transcend his personal predilections and habits. Questioning the effectiveness of this practice, I came across a remark by his mentor, Marcel Duchamp:
Your chance is not the same as mine, is it? If I make a throw of the dice, it will never be the same as your throw. And so an act like throwing dice is a marvelous expression of your subconscious.[3]
On its face, this statement is nonsense. Are dice influenced by the person rolling them? No. Yet doesn’t Duchamp aptly describe his own work here, and also Cage / Cunningham’s? No matter how random their constructions, aren’t the resulting works inescapably personal? To reduce this to absurdity, has Merce Cunningham ever rolled the dice and had George Balanchine or Cab Calloway or a whirling dervish emerge in a dance instead of himself? Is such self-transcendence even conceivable by these methods?
Perhaps not. But let’s look back at Turing for a moment. In the same way that Cage / Cunningham think chance will give their artworks greater autonomy, Turing wonders whether chance can endow his computer with free will. Can Turing succeed where Cage / Cunningham fall short?
With the exception of William Forsythe, many of whose pieces at the Frankfurt Ballet derive from sophisticated dance algorithms.
What’s missing from Cage / Cunningham (and from so many who followed) is complex contingency, which comes only by building networks of IF –> THEN relationships. For chance to be powerful, its effects must ripple down through many possible branches. . In reality, isn’t it rarely the case that multiple outcomes are equally likely?
To be more specific, in Hand-drawn Spaces , we saw that when Cunningham needed to find a dance phrase to follow another, he rolled his dice and accepted that choice. This worked because he’d created all 71 phrases to go with each other and because the Character Studio software generated logical transitions between any two phrases seamlessly.
One of Character Studio’s creators, Michael Girard, now imagines doing all of this cybernetically. He asks: What if one had, say, 71,000 movements to choose from rather than 71? (You may object that increasing the number of phrases also increases the number of possible interconnections by several orders of magnitude, but that’s what computers are for.) In the database of movements, each phrase is classified according to its speed, rhythm, style, etc, after which one can start to apply some rules – that is, to choreograph.
In a recent email, Girard imagined several scenarios. What if one computed a piece such that dancers could perform ever-faster motions in an increasingly tight space? Or divided a company into three groups and experimented with rules like this:
Blue group follows Green spatially and Red rhythmically while Red follows Blue spatially and Green rhythmically.
Choreographers routinely explore such rules and patterns as well, of course – and perhaps more subtly – but not across such a potentially vast range of phrases and permutations and conditions. “The ocean of possibilities here,” Girard wrote, “is so deep it’s hard to fathom.”
This system reminds us of something, but what? … It is not unlike the Big Blue supercomputer that finally outplayed Gary Kasparov. However, while chess is a closed system (albeit incalculably large) with a fixed number of pieces and rules, choreography is open: You can add as many new dancers and movements as you like. Still, the brute force searches of chess and motion-capture databases do resemble each other: both exceed human capability.
But there is another man/machine contest that’s even more to the point here – the Turing Test of artificial intelligence, for which we can now imagine a dance variant. Could our hypothetical system choreograph a dance such that no one could tell whether it was the product of human or artificial intelligence? And if it passed this test, would it not then constitute the autonomous work that Cage / Cunningham aspired to as well? Or even if it did not pass?
A good starting-point to a vast & vastly interesting subject is The Recursive Universe by William Poundstone (Morrow: 1985). Excellent freeware for running cellular automata is easily found online. My favorite is Life32 by Johan Bontes.
Real autonomy is an attribute of living things, which is where von Neumann comes in. First to wonder whether artificial life forms (“automata”) could evolve mathematically, he asked whether one could devise an algorithm for self-reproduction – in his view, the true test of life. He invented a marvelous playing field to run such tests: an infinite two-dimensional grid in which logical games called cellular automata could play themselves out. The most famous of such trials was John Conway’s game Life, a revelation to nearly everyone who has encountered it, including me. From three simple rules governing the life and death of abstract “cells” on the grid can come self-organizing and self-reproducing systems, often of extraordinary complexity.[5]
I can almost hear some of you objecting here, asking “Isn’t this becoming too one-sided computer science over art?” The truth is this: however fascinating such artificial life experiments are, so far (for the ones I’ve seen at least) they don’t resonate emotionally. Having spent a lot of time playing with such systems, I reluctantly concluded long ago that they don’t qualify even as unwitting, unintended works of art. One’s engagement with them is simply too shallow an experience.
. . .
Walking on the swarming sidewalks of a city like New York, have you ever had the feeling – this is a personality test – that you were: (a) an unconscious performer in a complex but unacknowledged dance; or (b) a cell in a von Neumann-like self-organizing system? Either sensation could have come from your simultaneous recollection of having looked down earlier, from a high office or apartment building, at the transitory patterns formed by the pedestrians below, who couldn’t help but remind you of ants. Down on the street, however, you can forget about overall patterns for a moment and concentrate instead on the singular, shifting, unrepeatable beauty of each “ordinary movement” as it unfolds before you. This is to look at the street with dance eyes, I suppose – though long before I’d acquired such a thing, I was finding the same beauty with film eyes, shooting a Manhattan intersection in Super-8 for my film Colourblind etc (1977). How extraordinary the negotiation of traffic, crosswalks, and crowds by each and every pedestrian, whether a six-year-old skipping toward the WALK sign or a businessman leaning into the onrush of cars, trucks, and buses to hail a cab.
Twenty-four years later, I am again addressing the complexity of the street, but this time looking down at it from bird’s eye view – “looking,” that is, with a computer rather than a viewfinder. It’s difficult to talk about a work-in-progress, for who knows exactly how it will turn out? But the piece, called [Pedestrian][3], is another collaboration with my old partners-in-crime Amkraut, Eshkar, and Girard. Our plan is to replace streetlamps in the city with high-powered projectors beaming looped sequences of crowd simulations directly on the sidewalk below. The intelligence of the software that Amkraut and Girard have developed is opening the door to an entirely new way of working, which is less like constructing a work than growing it.
To be more specific, imagine that you can set a simulated crowd in motion, instructing its dozens of members to move toward a point on the other side of the screen. Each crowd member draws on its own repertoire of motion-captured movements – walks, turns, stutter-steps, runs, pauses – to negotiate this task, so that you have young and old, men and women, all moving in their own characteristic fashion and all nearly touching each other but never colliding. Now create a second crowd, give it yet another range of movements, set it on a course diagonal to the first … and watch as the computer performs a myriad of calculations to produce an emergent pattern from their intersection, one that you could never have quite predicted …
Will virtual dance replace or diminish real dance?
No, I don’t think so. The more we embellish dance with technology, the more we’ll start longing to see the real thing again – real dancers in real time and real space, with no distractions.
But it’s also true that we can never turn back the clock. So isn’t it just as likely we’ll be seeing these unadorned dances with new eyes, new ideas, and new questions?