Shelley and I spent a wonderful week and a half holed up in Chicago making the source materials for Point A to B. This is an atypical project for us, and a very unusual project for me, because rather than some technical high wire stunt involving multiple computers composing images under the constant pressure of real-time, it’s being delivered as a finished animation — a Quicktime movie. This means that I’ve been spending time with a broader range of software much more than I normally do, eager as we are to afford ourselves of the rare opportunity to actually edit, hand keyframe and tweak a finished piece that will just play back the same way each time. Simultaneously I’ve been preparing our codebase for its open-source debut, so I’ve been thinking about my code’s relationship with the other applications on my hard drive.

 

Timelines

I’ve been struck and dismayed by how brittle and cumbersome our animation workflow is now that it’s much more conventional, and how compromised my carefully architected “open” code becomes when surrounded by the big standard software suites. Conventional “creative” software, it turns out, isn’t in a very good place. Worse, it’s all in the same place, and has been for a decade.

AfterEffects, Premiere, Motion, Final Cut, Soundtrack and Logic are all built around timelines; Shake has one, and a keyframe editor; its repackaged remnants, Color, has one. Quicktime Player has an editing timeline (although many miss it in its attempt to look like a car radio). Maya has at least two timelines that I can find and one suspects there are more lurking in its turgid depths, written by an intern and left in a distant corner of the shipping product. 3d Studio Max has 3, I believe. The new version of iTunes got one for editing ringtones. One of the handful of new features in Photoshop CS3 is in fact a timeline of sorts, can it be long until Illustrator and InDesign get theirs? Perhaps they already do. Recently I’ve discovered WireTap Studio, and it has a timeline editor for editing audio that it captures from applications which have their own timelines.

Obviously you can argue this is a significant waste of engineering and QA resources, a waste of UI learning time, and a waste of money in general — $10,000s worth of the same idea. After a final round of consolidation in the market, these products are mainly made by one of the “three A’s”, Apple, Adobe or Autodesk. It doesn’t have to be like this, and if you were to generate screenshots of the scrollbars of each of the above applications you’d find only two species (sadly not one, there’s Apple’s and the undocumented Apple “pro”) both largely provided by the OS. Apple, themselves, are ideally placed to componentize an hierarchical timeline editor into the OS itself just as they have done with the scroll bar (and list view, a tree view and a side scrolling column view &c).

 

Integration as
dissolution

These arguments are surely playing out somewhere, but they largely miss the point. These products, that sometimes ship in the very same box as each other, are converging while simultaneously scrambling to maintain their independence. It’s not that, as a programmer I want a standardized timeline “widget” so that my applications can look as cool as Apple’s, nor do I, as an artist, want all the timelines to look and handle the same. As a programmer and as an artist, I want all the timelines to be the same. I want the same timeline across applications, I want a level of “integration” where the “applications” being “integrated” lose their identities. I need to be able to use MotionBuilder to integrate optical markers with sequences inside Maya, programmatically create Soundtrack markers for our sound designer, track changes in source material from Maya, synchronize video from the MoCap shoot, revise rendering keyframes in my own development environment Field (which, of course, has its own ideas about time editing); this needs to happen on the same timeline and I need to version control the whole thing across each of our machines in Chicago and New York. I can’t do any of this. And there’s no code that I can write that will.

In more technical terms I don’t care that much about the views‘ design, I need live, distributed access to the models, I need my code and all the proprietary codes I use to be able to talk to and write controllers without instantiating or licensing or even being aware of anybody else’s favorite views. No amount of JavaScript / AppleScript / LADSPA / AudioUnits / OSC / SoundFlower and what-have is the answer — they don’t even help in formulating the solution — they are all interstitial technologies that do nothing to dissolve the actual problem. Nor is this a file format issue, this is not something that will be solved with more XML (for example, Collada). This is not a standards problem, or an interchange problem, it’s a problem that stems from the existence of things that treat interchange as special, and single application development as the norm.

I am old enough to remember OLE (which begat ActiveX), and OpenDoc (which begat a hasty Jobsian retreat) and their accompanying canonical vision: a Word document with a real live Excel graph in it. Both ideas went astray — ActiveX revolutionized nothing but the security industry and OpenDoc went the way of the Newton. But it was a exciting time, but mainly in a bad way. Microsoft sought to replace the monolithic Microsoft Word application with two or more monoliths — say, Word and Excel — tied together at the ankle. The resulting three-legged race through your poor machines’ virtual memory was simply too dangerous enter into with data you cared about. Back then, my best friend’s utterly tricked-out PC felt like it was the only machine in the world fast enough to almost load my 100 page high school physics thesis. Two years later the PC room at the Cavendish Laboratory was filled with terrified, desperate students who had just had all of their embedded Excel spreadsheets — model, view, controller and this year’s final grade — turn to black “X”’s by a mistimed Office network upgrade. As it recedes this technology transitions from enabling a densely connected, multi-view, multimedia document creation (what Office was supposed to become) to powering multimedia document consumption in a discrete lumps in a browser window (Internet Explorer). A much simpler problem, a much less significant domain.

The more radical OpenDoc project — which sought to evaporate the monolithic application completely — collapsed, quite possibly under the weight of its own engineering. But it’s not too much of a stretch to think that it threatened the still dominant business model of software companies who are ultimately in the business of making, testing, advertising, supporting, upgrading and selling monoliths. Too bad: in the mid 90s such a disruptive technology could only have come from a hardware manufacturer willing to use unique software API’s to sell tangible objects; in short, it could only have come from Apple. It didn’t. Instead we have a largely empty choice between Motion and AfterEffects, Pages or Word. It’s not about doing more so much as it’s about crashing less.

 

Open source,
Open software

Today, the open source movement ought to have given this story a happy ending. It hasn’t. This ongoing attempt to duplicate Adobe/Autodesk/Apple’s software products button for button, feature for feature is regrettable, but it’s also inevitable once you accept the way the problem has been framed in the first place. GIMP has plugins and scripting as much as Photoshop, Blender as much as Maya, Open Office as much as Microsoft Office, and no more, no deeper, no differently.

This is upside down, and open source software should be different; the sheer thrill of a triumphant ./configure && make && sudo make install (the magic command line incantation that takes the very source that is “open” and turns it into a program that you can use) should be diffused throughout the software product until the very idea of product as separate from source or from use has been disassembled. It would look like Perl’s CPAN / Ruby’s Gems / Python’s PyPi (popular and dependable online repositories for shared, modular code); it would look a little like Eclipse (a plugin-centric development environment for programmers); it would look a little like Mathematica (a mathematicians ‘notebook’); it might look like Field; part Smalltalk, part Lisp Machine, it might feel, but hopefully not look, like Emacs (a superpowered text-editor that thinks it’s an operating system). But it wouldn’t have a name, or the branding of Microsoft Word. It would be hard to have an ad campaign for it. Losely coupling views would be gathered together to hew things that look like timelines and documents and spreadsheets out of fantastically deep shared ground of controllers and standardized models available for download. Code would be mobile, languages used diverse. Grandparents, students and asian animation farms would use different distributions prepackaged by downstream support companies. The language and ideas here are pure open source programming, pure Bazaar to Microsoft’s Cathedral, but they seldom ever reach the surface or open source applications. Richard Stalman talks of protecting the “freedom to tinker”, the fundemental freedom to reconfigure and alter software guaranteed by free software — but why try and make this freedom disappear once the code is compiled?

Here’s the reality: doing anything creative with an application is writing code. Layers in Photoshop / AfterEffects / Motion / Final Cut / Illustrator execute to produce an image; timelines are rules for producing edits; paragraphs and their associated style information compile to produce layout and pagination. Sometimes the domain, representation and “syntax” fit (layers in Photoshop, versus channel operations), sometimes they don’t (what does “convert for smart filters” in Photoshop even mean ?). Most of the time the layers palette is a better interface than a text editor full of code, but many times it isn’t — there should be both. Sometimes the programming pokes through anyway: I’ve debugged Word documents having trouble with their footnotes; I’ve had recourse to expressions in AfterEffects; and the excitement over web 2.0 is fundamentally about programability (the “mashup”) not about network latency reduction strategies. There are good, deep and convenient ways of manipulating code and there are bad, tedious and error prone ways; there there are domain appropriate languages and syntaxes that are just not up to the task. But above all, it’s better to have as many different ways of writing, manipulating, and invoking code as possible. From this perspective the boundaries of applications and the duplication that these boundaries necessitate seem downright petty. My code talks to other people’s code all the time, why can’t the airbrush in Photoshop talk to the Live Paint brush in Maya? Why can’t I provoke that conversation with a well turned page of code? What proprietary secrets are these brushes hiding from each other? From me?

Are we getting there? Applications are incorporating real programming languages. Photoshop is doing Matlab and has had JavaScript for a while, and Autodesk is doing Python now — a move away from the grotesquery that is either MEL or MaxScript that I endorse wholeheartedly — but it’s absurd for me to be as excited by this latter move as I am. It has let me inject code into MotionBuilder that streams joint angles across a network socket to codebase that I control, so that we can use their motion capture skeleton matching algorithms in our live renderer. It works like a charm, but doing this felt like I was exploiting a security hole, stealing information, spying on the insides of an application, doing something that I shouldn’t be doing — something that will break on an upgrade. In telling it, people inside Autodesk have looked at me like I was a little crazy. I’m doing this because I want to use their FK/IK engine, I want to pay them for the privilege of not having to write it myself and I already have it on my hard drive. But we’ve seen artists go so far as to try to make realtime artworks inside Maya just to get access to their mocap plugins (this didn’t end well). Rather than integrating Python into their products Autodesk should be disintegrating their products into a collection of Python libraries (or C, or something, anything). If they did, I’d write maintain and distribute the Java bindings for free.

In this imaginary world I would be free to right my code in, on or with Photoshop and make things that Adobe, Apple and Autodesk can’t imagine. A grand vision, but one out of step with the market. Today I read with disbelief that Apple has proudly announced that their new OS, the eagerly awaited Leopard, will allow you to record menu selections and keypresses and play them back!

“Integration” has a long way to go, and I swear our next animation will be edited in software I wrote.

A few days ago we announced the release of Field and Loops under open source licenses (both the choreography and performance by Merce Cunningham, and our new digital artwork of the same name). Was this altruism on our part? Not completely, there’s a plan: we’re trying to change the eco-system of digital art and performance in our favor.

For a long time now we’ve been frustrated with how digital art is taught and thought about, and I’ve come to believe that part of what’s gone astray is the very thing that we are trying to address in this “preservation project”: that is preservation itself.

 

Scholarship and preservation

Three things are almost constant in our field. Firstly, if you read any criticism about a digital artwork, it’s been written by someone who has been unable to study the object they are writing about in depth, hold it in their hands, take it apart, see how it works. Often they are reduced to guessing, to making things up (while defending making things up as a valid critical approach). Secondly, if you read anything interesting about an artwork, it’s almost impossible to track it down and see it for yourself. Photographs with captions in books are our canon, in a way that’s almost unbearably ironic for a field that often touts its ground-breaking multimedia nature on the very same page. These books are vital, but they are the botany, not the physics of our understanding of this art. And finally, should you locate the artists responsible, there’s a good chance that the artwork will have stopped working or be prohibitively hard to put back together again for installation. At best, all they will have is a wider range of photographs and captions for you to look at.

This isn’t the situation with painting or music; but it is a condition that digital art shares with dance. And these three things severely limit the quality of teaching and criticism in general and the size and complexity of the objects that students and critics can study in particular. Ultimately they damage the art itself. Our large-scale, complex, and fragile artworks don’t fit well within this environment. Something has to change.

We’re not the first to care about the preservation of contemporary art (or dance). On the contrary, there has been considerable discussion, and funding for discussion, over the last decade. It’s amounted to, as far as we can see, very little that’s concrete and even less that’s interesting. It seems driven by either an academic desire to talk about a fascinating and fashionably interdisciplinary subject or an economic desire to transform digital artworks into something that can be bought and sold like a painting. Frustrated by this, we approached the Mellon Foundation, for funding for Loops, with a strong desire to indicate how we thought it should be done by actually doing it.

Digital artworks cannot be bought and sold like a painting — and all the talk and free conference food in the world will not change this. Our new strategy of open-sourcing everything we can find an excuse and the funding to open source, leads then to a different place, frankly, a more moral place. One where the theory (scholarship) and practice (pedagogy) of our field stands to be transformed by the possibility of “close reading” and deep understanding of the artworks themselves; where digital art, and dance scholarship, are no longer starved of actual, useful, stable, share-able examples.

 

Open-source software and open-source Art

That was the practical argument for “giving away” an artwork as open source, but are there deeper connections between the morals of software-making and art-making?

To readers arriving from the software-, rather than the art-, world, the previous section will sound very familiar. This is in fact nothing more than the philosophical goals, and the economic realities, of the open-source movement. The philosophical goals are often obscured today (given the success of the model, some have come to suggest that open source strategies are justifiable more simply, and less politically, in terms of the quality and cost of the software it produces). But, to my eye, the original point of this was both clear and extremely radical: that the attendant, moral price of giving somebody a piece of your code was that you also had to give them the ability to understand it, change it, and share it.

This is a different reading of the philosophical tenants of free software than most, but let’s rewrite this argument for the art-, rather than the software-, world: the price of having strangers come and look at your art in a gallery (or in a theatre, or especially on a street, in public), the price of putting your art in their head, is that you have to give them also the means of understanding, transforming and sharing it as well. Oliver Sacks’ most recent book, Musicophilia, hints at the psychological stakes of this exchange with a stark image, calling music that sticks in your head to the point of irritation an Earworm. Perhaps the cost of being in the Earworm distribution business should be that you have to allow your worms to be dissected, understood and shared. Perhaps, the self-confident Earworm maker might hope to get back more from a now flourishing community of Earwormologists than they lose from their more limited control over their ear-market.

My overview of “free software” here is backwards from the usual presentation of the open software’s core ideas: it’s typically given in terms of preserving the freedom of the recipient (of the software) to “tinker” with what they receive. I believe this (perhaps deliberately) downplays the heavy moral dimension of the life of the software creator in favor of underlining the wonderful freedoms to-be-enjoyed by the software consumer. Fun as “tinkering” is, it just doesn’t capture what makes software special. Software is simultaneously just like a screwdriver (in that if you sell me a screwdriver, you shouldn’t try to tell me what I can and can’t screw with it) but it’s also nothing like a screwdriver (in that your screwdriver doesn’t become a transformative extension of my brain with the moral responsibilities that entails). Software is in a special class of things, a class shared only by a few other things, including, perhaps, art.

 

What is “distribution” ?

But both perspectives can exploit the same strategy, and the third revision of the open source movement’s main tactic, its “core text”, the GNU Public License, was formulated quite recently. This is the very legalistically magic document that aims to maintain these freedoms (and codifies these responsibilities). Simply put: if you distribute software under this license, you must distribute the source code to it, in a way that allows understanding, modification and subsequent redistribution of those modifications under this very license in turn. Version 3 of this document updates the highly successful version 2 to defend this vision against new attacks, imagined and real, conducted by forces in favor of closed-software. (For example, some consumer product manufacturers, most notably Tivo, are dedicated to making hardware that, while having freely modifiable software, from other people, on it, won’t actually run any software that’s been modified, even by those people).

One attack (and defense) that was presumably deemed to be too radical, even for GNU, is being posed by web-based software. Should, say, Google turn out to be based on modifications of open software, should they be bound to release their code to me, an avid google-r, or is there something special about using a web browser that prevents this interaction counting as “software distribution?” If it works at all, my perspective, and my moral argument, still work despite the prophylactic Firefox and cable modem: Google is in the process of transforming not just society and scholarship, but individuals, and yet we are unable to study how it works, talk about it in any deep way, or change our own versions of it. Regardless of its intentions, can this ever really be morally acceptable?

Seeing this issue arising, but perhaps unwilling to demand that everybody play hardball with Google, the Free Software Foundation (the force behind the GNU Public License) worked on an optional addition to its license that would cover this: the confusingly entitled Affero GPL. Works distributed under this license explicitly count web-services as distribution and providing such services automatically triggers the legal responsibility to distribute the underlying code. If Google incorporate this code into a web-service, the whole service becomes “tainted” and requires the distribution of its source-code.

This will, of course, never happen, but the existence of this license shows that the very nature of what “distribution” is is transforming, and up for debate.

 

Distributing Loops?

Which brings me to an utterly hypothetical, but intriguing point back in the (digital) art world: what if performance (or installation) of an artwork counted as distribution? What if works made with tools under GPL-like licenses triggered the code distribution clauses of the GPL as soon as the doors to the gallery opened? What if the audience had to be able demand a CD of the source code for the piece on the way out of the auditorium? Critical thought about digital art would be transformed, new forms of scholarship would appear, the techniques of digital art, poorly taught right now, would practically teach themselves in self-assembling online forums.

Fantasy, perhaps. One thing is for sure, we are sufficiently tickled by this idea that Loops, when it is installed “for real” in a gallery (rather than a press conference), will be accompanied by a stack of DVD-Rs. Like the Loops preservation project itself: a small gesture, but one that we hope becomes exemplary.

And with that, back to cleaning up Loop’s source code, so that I can post it all online.

The issues that I raised in my two previous posts about software — both posts essentially about different views of the “openness” of software — came into sharp focus last month during the shoot for Depth.

We’ve always had an improvisatory approach to workflow — partly out of necessity (there’s seldom any time to properly “rehearse” projects) and partly through choice (pieces are often only found “in the making of them” rather than by careful planning, and conversely rehearsals tend to become pieces). Depth is a series of works that marks a departure for OpenEnded — they are all films of sorts, the first, Housebound, even has a screenplay. That said, they each aim to invent a new way of capturing the world photographically, the series concerns inventing cameras, which, in turn, necessitates inventing workflows.

 

In Stereo

As such, Housebound was recorded in stereo using this elaborate contraption:

A moment of shear gear-headedness: the mount is borrowed from Alain Derobe, a 3d-film specialist in Paris, and uses a half silvered mirror to allow you to effectively get the cameras closer together than their bodies would allow (we continue to be confused by images found online of rigs for sale that have with very wide baselines — are these for landscapes?). The cameras are a pair of Sony XDCAM-EX1s that send HD-SDI out to a pair of BlackMagic Decklink cards in a pair of Mac Pros each with their own three disk Raid 0 array to record the whole affair direct to glorious uncompressed 1080p30, 10-bit 4:2:2. And I do mean glorious, these cameras with this recording technique has an “authority” that to date I’ve only associated with film.

Regardless, the piece calls for the careful, 3d insertion of text fragments into the footage that seem to hover in space, but in real space (that is, they obey the physics of our hand-held camera material).

Attaching text to an object in the video footage is a relatively straightforward point tracking problem — you have the typical redundant non-choice of tools: Motion, After Effects, Nuke, didn’t Maya have a point-tracker somewhere? I had vaguely assumed (that is: was planning to improvise around) that we could make the tracking super-easy by introducing markers into the scene and then removing them in post. I had assumed that this would be straightforward (despite the absence of a background plate given our moving camera) because after all, we know where the marker is in the scene (we’ve tracked it). How I was planning on removing the marker from the scene turned out to be a little less well planned.

 

How to make a healing brush

Adobe Photoshop’s “Healing Brush”, if you haven’t tried it, is a genuine marvel. While almost every other thing that Photoshop does is pretty much obvious to anybody well versed in 2d image manipulation (quick: try to guess the number of unique gaussian blur implementations that Apple alone ship with their OS) this brush that magically removes blemishes while creating exactly the right sense of texture truly does seem to do something magical. And it wasn’t obvious to my eyes or brain just how it was doing it (at least it wasn’t before Siggraph 2003). It’s more than up to the task of removing our tracking markers from a video scene.

Knowing that Adobe had so solidly solved the marker removal problem I synthesized, in the back of my preproduction mind, a completely incorrect assumption: that the most recent version of After Effects would also have this wonderful feature.

It doesn’t. Incredible! Adobe ought to own the wire, scratch, marker removal and intelligent matting problem with this technology, and would do had the code that’s in Photoshop travelled down the hall to After Effects.

This omission is understandable within and only within the “large monolithic product” framework, where each product is a separate sovereign, competing state. But Adobe itself has been amongst the most aggressive in pushing a slightly more enlightened viewpoint: product as plugin-host. After Effects already has the UI for offering the healing brush (it possesses a conventional clone tool), but they remain pathetically trapped in a software framework of their own making. A prediction: just as Yahoo plan to “turn themselves inside out” into a “re-mixable” set of services, Adobe will ultimately be forced to do the same.

But for now neither their After Effects team, nor I, get their wonderful engineering in the form that we actually want it: an API that you can call from code, any old code, your code, anywhere.

What next? What any post-academic would do: read some papers, attempt to find out how they did it, implement it yourself.

We’re in a surprisingly good spot here: Adobe seems to have been permitting their senior hard-math figure Todor Georgiev to publish actual academic papers rather than just patents. Even better (and perhaps precipitating Georgiev’s release) Andrew Blake and friends published it first (Blake is a member of a small circle of computer graphics people whose papers are, strictly, always worth a read).

Still, both papers suffer from a two habits shared by academia and industry alike: Firstly they are a tease: they stop short of telling precisely you how to do it and switch to selling you on just how good their approach is. Georgiev prefaces his paper with an inspiring, and utterly un-SIGGRAPH theory of digital image representation that draws upon math that I recognize only because I once studied General Relativity; Pérez, Gangnet and Blake, in Poisson Image Editing (now there’s a Google-proof title), take you almost all the way there and leave you with the following sentence “results in this paper have been computed using either Gauss-Seidel iteration with successive over-relaxation or V-cycle multigrid”. What they actually mean is that they solved their equation 7 by doing pretty much the first thing that pops into your head if you’ve understood the problem this far. Why not say this? Perhaps it leaves room for the inevitable follow up paper “GPU accelerated Poisson Image Editing” where the intern / talented-undergrad reveals that Gauss-Seidel / SOR for this particular problem is essentially equivalent to the much easier to accelerate and understand “blurring the image a lot” algorithm.

 

How to write about the healing-brush

But the real crime is this: neither paper comes with source code. There’s no good reason for this, and plenty of bad reasons. Space isn’t a good reason — who reads proceedings in paper form? Relevance isn’t one either — this is a self contained problem with well-defined inputs and no architectural issues. In the last academic field I sincerely was a part of — large scale, messy, integrative AI — there really was an argument for not distracting people by sharing your code with them. What you want to convey with a “systems paper” are design decisions, tradeoffs and perhaps a certain philosophy of approach. None of these things are well captured by the language of code, and nobody takes seriously the scientific criterion of reproducibility when it comes to writings about software design. But in this case here, what the authors are at least pretending that they want to convey is exactly what they did in order to get the results they show. In this case, no special infrastructure is required: their algorithms take two images (original image and mask) and produce a third — you can use your favorite c++ image class for all I care — an image is an image. Their code is the ground truth for exactly what they did, any other level of description, for such small, self contained problems, is icing on the cake.

To not share the very code that they have written (and clearly extensively polished) to make their very accomplished demos would seem extremely odd if it weren’t so common-place. Ultimately it’s nothing less than two-faced: it claims to be committed to the exchange of ideas but not too committed. It’s hard to know how we got here — is it the hope of commercialization? fear of academic competition? laziness?

I am bound by no such desires or fears. And to provide a satisfying punch-line to this argument I’m placing a Field based open-source implementation of something very Healing-Brush-ish available for download by anybody (anybody, that is, that’s willing to accept the terms of the license) from the Field development site. It’s probably less than a SIGGRAPH paper column-long. Heck, it’s even GPU accelerated. Here’s it working:

Take it, go do something interesting with it.

 

The academic game

My frustration with academic pseudo-publishing continued for the rest of the week with another surprise. Due to the half silvered mirror in our stereo-rig, our camera pair doesn’t provide the same image when presented with the same scene. The mirror seems to reflect more green than it transmits, and the 300MB/s second pair of video that we obtain need to be carefully color matched with each other.

Unlike “healing brushes” there’s hardly any interesting maths to color matching — just user interface. For this problem, we find ourselves again a little far from the mainstream. But it’s just as much fun to build your own interactive color matching tool in Field:

That wasn’t a problem. But had I thought about this before hand I would surely have realized that this was going to cause problems with my dense optical flow code — algorithms that try to tell you what each bit of one image corresponds to in another. Generally such code depends on a pixel in one image being the same-ish color as a pixel in the other frame, in order to find the correspondences. Not to worry, I would have thought, had I been thinking about it at all: optical flow has been a topic of research in computer vision since David Marr was inventing the field itself in the 70s — there’s plenty of research that I can draw upon.

Well yes and no. This too is a problem with low infrastructural complexity — take two images of a scene separated slightly by space or time and produce another — an image of “arrows” that point from a pixel in one image to a pixel in the other. But this too is a place where hardly anyone is sharing actual honest code. Worse, since this is a far older problem than image in-painting, it has had time to shed its SIGGRAPH “ain’t my movie cool” sparkle and has now become a serious game played by serious players.

But the emphasis is very much on game. There are standard images for you to fight on, and, unbelievably, even a league table. Optical flow algorithms are being evaluated with rules that might be slightly less open than the Netflix prize — freely available training data with hidden test data, online submission, public rankings, anonymous submitters and, in this case, few rules concerning publication. When I checked, topping the charts this conference season are three or four utterly unpublished algorithms by masked authors. There’s a web 2.0 interface so you can browse just how good they are, but you can’t get hold of them yourself. What is the prize I wonder? Tenure? A job at MSR? Weta Digital? Surely not, at this point, a video compression start-up?

Spurious arguments about intellectual property have no place here — you are either committed to sharing ideas or not; you are either working on your dissertation or you are working on your startup. Too many people are essentially getting the intellectual equivalent of patents published by the ACM and IEEE, and these organizations’ willingness to add a quicktime to any paper only encourages this misdirection. Sure, they don’t have the legal weight or the financial future of a patent, but they mark something just as precious for the tenure track faculty or the career researcher: territory.

Looking for a good specimen of typewriting, I found myself rummaging through a box of my old papers. There I recovered a rubber-banded stack of 3×5″ index cards on which I’d once catalogued the vestiges of dreams that I could dredge up the morning after — mostly nightmares, like this fragment of tortured adolescence:

At the time I had a small manual typewriter whose slightly balky keys required that I strike them forcefully enough to propel the letters against the raised ink ribbon, imprinting slightly splattered black marks on the white paper.

The sharp report of my early morning typing was so loud as to jar awake whatever part of me still lingered in sleep. This routine was not only my daily mental exercise but also, to the slight degree befitting the gaunt teenager I was at the time, my physical exercise too. For typing demanded at least a few exertions:

— rolling the stiff index cards into the platen straight.

— striking the keys at an even rhythm that was fast enough to keep pace with my thoughts, but not so fast as to jam the letters.

— tugging the carriage return lever to bring me back to the left margin while pushing me one line space down the page.

Typing is such a loud and forceful action that an iconic image of the early-to-mid 20th century is of a writer at his typewriter pounding out urgent dispatches, often from the front. Writing was a kind of battle with the page, and the writer figured as either an actual or a symbolic war correspondent, his inner tortures sometimes seeming as brutal as any combat swirling around him on the battlefield.

I say “Him” not only because the grammar I grew up with made the third person singular male by default, but also because as a teenager I chose my literary heroes from a pantheon of men. Eliot, for example.

This detail from his typed draft of The Wasteland illustrates the violence of revision:

The typist at teatime evoked by Eliot in these lines, however, is not a man but a woman — and not a writer, but a secretary, on whom the typewriter confers the altogether different social role of female subservience.

Her job is to take down the man’s dictation in shorthand and then type up a fair copy for him afterward, his hands never having touched the machine (though perhaps having touched her, at least implicitly).

No secretary would ever dare eviscerate a page the way the author did; only another writer would (as in this most famous of cases did Ezra Pound to Eliot’s actual draft).

 

Visual vehemence

What’s particularly striking is the look of VISUAL VEHEMENCE that typing gave to raw pages. It was as if with this particular machine a writer felt he could physically pound some sense into his draft, asserting his muscular dominance over words that might otherwise go their own evasive ways.

Seeing a word or phrase gone wrong, he’d correct it with crude and impatient directness — x-ing it over with the x key, covering it up with Wite-Out^®, or scribbling all around it with pencil or pen.

Until the time when a writer’s text was finally typeset and printed, there was no way he could keep his doctoring of the page under wraps, and so the manuscript never lost its sense of immediacy, never seemed completely done. The writer as reviser always threatened a return, the corpse still so warm that it could be shocked back to breath anew.

Sometimes the writer resorted to more violent surgery of his pages. Wielding a razor-blade like a scalpel, he’d cut whole limbs of his draft and then Scotch^®-tape them back into a newly Frankensteined body, stitches showing.

 

Cutting

If all the foregoing seems to push its metaphorical readings too far, they at least report accurately the sense I had of things as an impassioned young man, when writing seemed as if it might be turning into concrete and heroic action the way that painting had in the hands of Jackson Pollack. I took this notion not so much from Kerouac’s celebrated amphetamine-fueled typing of his On the Road manuscript (“spontaneous prose”) as from Burroughs’s razor-blade cut-ups.

I tried forcing parallels between verbal and filmic syntax, as described in an earlier entry, Succession.

For cutting as a physical act engrossed me in those days: I was as immersed in filmmaking as I was in writing. (Also, though this scarcely belongs here but would be dishonest to omit: at a couple of moments of extremity, I turned to self-cutting, but thankfully with fingernails not razor-blades).

The editing of Super-8 film was a literal cut-and-paste procedure, which began with my cutting the acetate film footage into different shots. I’d hang these strips from the little nails I’d hammered in rows near the top of my bedroom walls and then lie on my bed while pondering how to combine them. When I’d chosen two shots to splice together, I’d use a mechanical film splicer to scrape the emulsifier off the edge of one strip and bond it with film cement to the edge of the other (the glue’s acrid smell and its lingering stickiness on my fingertips now comes back to me for the first time in many years).

Since I couldn’t afford work-prints and had to operate directly on my original footage, the physical act of cutting constituted what we’d now call destructive editing: each cut would consume at least one frame of film and could not be undone without additional damage. This meant that making a cut was never a decision I could take lightly nor execute automatically.

While this is not quite so true of cutting up and then pasting different typewritten pieces together, it’s certainly the case that writing and editing in what’s fast receding as the mechanical age was necessarily a slower, more deliberate, and above all more visible process than in the word processing of our digital age.

Since to cut-and-paste on the computer is easy, fast, and invisible, writers now leave no trace of their efforts.

It once occurred to me that this contemporary version of cut-and-paste has many of the same consequences (except pain) as the nip-and-tuck of contemporary facelifts, a thought that led to this text from Other Bodies :

 

Unearthed

 

Inside the Exhibition and Admit the Peacock. 2006, Roundy Wells Press: Massachusetts.

Several years ago my cousin Rebecca Kaiser Gibson sent me two chapbooks of her poetry, which I carried with me on a trip to Austria and read carefully in my cramped barracks-like hotel room at night.

(By day, among other things, I came across an audiotour that sent me walking over the underground slave labor factories of Mauthausen-Gusen, an experience I wrote about elsewhere).

For some reason, I never opened Rebecca’s books at home here in Manhattan until recently, when I looked up the poem that had had lingered so long with me and triggered the thoughts now finding voice in this piece of writing.

The poem is entitled Exhibition: The Books of Kells, and the middle of its fifteen lines, reproduced below, muse on the materiality of this sacred medieval tome, which had once been buried by thieves to hide evidence of their crime:

The closeness of Rebecca’s observations, it seems to me, puts the reader in nearly the same relation to the page as the original scribes, whose thoughts may have wandered along similar lines as they spent their days in the inscription or illumination of the gospel passage entrusted to their care. What boat from what plant on what shore had brought this particular color to their brush?

Wonderings like these are to explain visual things, which is fitting, for the Book of Kells was more to be seen than to be read. As a precious sacred object, its purpose was to inspire churchgoers from afar rather than to be pored over by a pious parishioner in the library.

Or by a monk in his cell. But he had no need for it there, where his practice of slow reading had him humbly absorbing a very short passage of Scripture to the exclusion of all other words (and fleshly temptations) over the long spells of self-inflicted solitary confinement.

The rhythm of the words must have come to rule the rise and fall of his breath, even the pumping of his blood; and as for their meaning, it must have enlarged to fill all the space left vacant in his solitude.

This, I think, is what poetry wants; wants and now rarely gets — the inwardness, slowness, and spaciousness of contemplation.

These are conditions that poetry still commanded in the days when the memorized lines of Shakespeare and Keats could be effortlessly called to mind and to tongue. But now ever less so.

Even these lines of prose want something similar, though they’ll never have it. Here on the screen the eye skids off of in every direction, the hand reaching for this or that other window or hyperlink, the mind ever distracted and distanced.

 

Henry / Husky

The first I learned of my cousin’s poetry was about twenty years earlier when she’d started reading me drafts of the pieces she was writing — almost transcribing — at her father’s bedside. Rebecca called these the Henry poems, Henry being her father, though he went by that name only on such things as his law firm’s letterhead and the briefs he signed.

The world knew him as Husky: the right name for this larger-than-life great-uncle of mine, who at that time was being felled by a cancer that had begun in his lungs and moved to his brain — likely caused by his heavy cigar habit, if not by his wife’s second-hand smoke which joined with his in pervading every nook and cranny of their sizable home.

This property, which stood just over the District line in an old Maryland suburb of Washington, had white colonial columns flanking its front door; a living room dark with heavy drapes and Persian carpet; and a bright blue swimming pool in back that would half-blind me as a boy after too many hours playing Marco Polo in its chlorine.

Facing the house across Bradley Lane was the high fencing of the Chevy Chase Country Club, a place Husky regarded with special scorn, refusing to forget the days when its rules barred all Jews from joining. Husky would time his celebration of American independence with the club’s, which often meant July 3rd rather than the 4th. After a big hotdog-and-hamburger barbecue, family and friends would lean back in their lawn chairs to watch the country club’s fancy fireworks exploding in the wide-open skies over their well-guarded fairways and sand-traps.

Part of our pleasure was in paying no dues for the bigots’ rockets’ red glare.

 

Search

My father has only recently stopped hinting to his children that the Constitution nowhere requires the possession of a law degree for an American to become a Supreme Court justice. Since none of the three of us ever showed the slightest interest in law school (but rather downright contempt, at least on my part), this was his wry way of conveying his idea of a citizen’s highest aspiration.

It was from a similar feeling that he would often relate to us the perhaps fanciful story of Husky’s near-nomination to our highest court. In those days, however, I paid scant attention to such family history, not wanting my future to be projected from their past … which has meant that few of the details I’ve now wanted for this account are in my immediate possession. Several east-to-west-coast phone conversations with my parents have helped, and I’ve also tried my luck on the web.

But there I’ve been surprised to find so little. After filtering out hits for no-relations Henry J. Kaiser the industrialist and his grandson Henry Kaiser the experimental guitarist (whom I once encountered as he prepared for an odd artist-in-residency in Antarctica), I found few links to follow.

The Washington Post’s obituary — overseen, I think, by that paper’s writer and editor Robert Kaiser (Husk’s nephew and my cousin once-removed) — proved to be hidden online behind quite a paywall: a frameable paper reprint was the only option, an exorbitant $74.95. Meanwhile, the New York Times’s archive, though free for the reading, yielded a cursory notice that told me nothing new.

Google Books returned a set of better results: scanned pages from an unlikely source, “The World’s Foremost Amusement Weekly,” The Billboard (a publication that still exists, minus the “The” in its name). Husky appeared in several articles of the late 40s, all concerned with the recently passed and much detested Taft-Hartley labor law. The newspapermen were evidently in Husky’s corner, as revealed for example in this sentence (its phrasing straight out of a post-war film noir):

It was from a brief article on June 18, 1949, that Husky’s own distinctive voice has come back to me most clearly, a gruff quick-witted rumble that sounds across the decades:

This barb was pure Husky. Given his own habit of plain speaking and direct dealing, he must have had trouble stomaching the convolution and cant of his field, not to mention the high-mindedness that often masked its self-serving methods (— the more incomprehensible the statute, the more numerous the billable hours, as I once learned in a different context).

Imagine Husky’s fury over the Tea Party’s union-busting tactics in Wisconsin (headline news as I write this paragraph). It was in liberal Madison in the Great Depression that he earned his law degree.

Husky’s belief, I gather, was that management and labor should just be allowed to duke it out (Husky looked like an ex-boxer) — the opposing parties’ naked interests should be plain for everyone to see, and government bureaucrats should have no business meddling (“mediating”) in the fight other than to referee it.

It was an outrage to labor that the Taft-Hartley legislators had limited the rights of workers to call a strike and walk out on their bosses. Let the two sides be free to go through the rounds, and may the best man win. (The best man, it went without saying, would be the worker.)

 

Phonies

Husky had a long memory for shape-shifting hypocrites. No one aroused his scorn like a phony (his favorite term of disparagement), and he skewered phoniness in whatever form he found it.

Case in point — He was one of few liberals who didn’t fall for John F. (Jack) Kennedy in his so-called Camelot days, for why should he let down his guard for so obvious a phony? He wouldn’t turn a blind eye to the man’s father Joseph Kennedy, that anti-semite and Nazi appeaser who’d channeled his own ambitions through his boys’ political careers and who’d succeeded in buying his second son a presidency.

As for the slightly later legend of Robert (Bobby) Kennedy — embraced by the so-called youth counter-culture and, like his brother, sanctified after his assassination — Husky would have none of him either, unwilling to forget that Bobby had once served his father’s pal Joseph McCarthy with no small show of zeal. Even after resigning his post as a senior staff member of McCarthy’s red-baiting Investigations Subcommittee, Bobby’d never repudiated the demagogue, who also escaped any denunciation by Jack, a Senator at the time, which confirmed at least to Husky the phoniness of both brothers’ liberalism.

 

Strange attire

Husky’s manner of dress bewildered my parents as much as it did me, for with his loud shirts (often bright red) which he wore with collar wide open around his powerful neck, he looked more like a Mafioso than he did the usual bloodless buttoned-down DC lawyer. Looking different was almost certainly his point, though given the widely rumored and too often actual corruption of organized labor by organized crime, it was still a weird choice for him, even if it signified nothing real (his beloved American Federation of Musicians was never suspected of being mobbed up the way the Teamsters and the Longshoreman’s unions were proven to be).

In the late 60s and early 70s, Husky drove a bright red Pontiac convertible (buy American), its top down whenever possible. I remember his second daughter, Tamara (or Tammy, the nickname she still went by then), telling me that Husk had a touching habit of giving rides to any hitchhikers he encountered on his daily commute downtown, getting into good-natured but disputatious conversations with his passengers, who tended to be college students or drop-outs — hippies as they were called early on, then freaks as they came to call themselves a few years later when the Flower Power dreams of peace and love had wilted.

Young fashion reflected this disillusionment, shifting from bright exotic psychedelia to working-class drab: thrift store work-shirt, blue jeans, and boots. This is what I wore on the breaks from prep school that, with our parents posted abroad, my brother and I would often spend at Husky’s; and if his mafioso clothing looked perverse to me, my workman’s outfit must have struck him the same way. Like the middle and upper class youth of my generation who followed this fashion, I’d never done a bit of manual labor in my life, so surely I counted as a phony too, though my uncle for once bit his tongue and didn’t say so outright.

(He did once insist on buying me a tailored pair of trousers, with shiny beige fabric [double-knit?] and permanent creases running down each leg. But this I wore not even once.)

 

Social aging

Social aging is Jean Améry’s term for a cultural obsolescence that he took to be at least as dismal as the body’s inevitable decline. Améry — anagrammed from his given name Mayer to put him at a linguistic remove from anschluss’d Austria, though he never stopped writing in his native German — was an unwilling expert on torture, having been schooled first by the Nazis in its Gestapo interrogation rooms and concentration camps and then, as he attested, by the betrayals of his old age (which he terminated in suicide at age 66).

As you get older, Améry observed, one of the deepest betrayals is that of your own thinking as it starts to date. Not that your ideas necessarily strike you as any less vital than they’ve ever been, but that they no longer mesh with whatever’s on the collective mind of the ever-younger world slipping away from you. When the times move on, they leave you behind in the increasingly irrelevant, or at least disregarded, past.

A telltale symptom of this loss of sync is that new terms and phrases start sounding wrong to your ears and positively ridiculous coming out of your mouth — so that as far as updating yourself goes, in this and in many other ways (clothing, to pick an even more ludicrous example), you’re damned if you do and damned if you don’t. Even if you remain true to what comes naturally to you, you start second-guessing yourself, wondering whether the way you’ve been putting things now strikes others as quaint or stale, either option equally mortifying.

As for your store of accumulated wisdom, said to be the reward of maturity, but no longer so regarded by the young (and certainly not by an earlier version of me where Husky was concerned) — that wisdom is scaffolded by a once-common frame of reference that’s aging at least as fast as you are.

(I imagine that the older Améry must have found this cultural outmoding even more painful than Husky did, for he’d suffered through the inverse experience when young, which must explain his having written so profoundly about each.

(A secular Jew, Améry had been steeped from birth in German culture, but with the rise of Hitler this culture was no longer his but only his Nazis persecutors’ — who robbed him not only of Nietzsche, whose philosophy they cited, but even of Novalis, about whose poetry their periodicals could still publish not-wholly-obtuse appreciations. Even a certain German word — Jud — had become a stamp of death, for he saw how the word became flesh and how the incarnated word finally led to a heap of cadavers.)

 

No fork taken

I mentioned earlier my parents’ perhaps wishful belief that Husky was once nearly tapped for a judgeship on the Supreme Court. I think they might stress this possibility not only because of the great honor it would have brought him, but also — perhaps a subconscious thought — because it might have changed who he ended up becoming.

There are few areas in American life where so-called seniors can still command respect (or, failing that, exert force). With no fixed term or mandatory retirement age, and in confident possession of absolute autonomy, Supreme Court justices can continue working long into their old age — never facing the threat of irrelevance since their considered opinions still weigh as heavily as ever in the scales of justice.

No hope of that happening anymore with Reagan- or Bush-appointed justices, all chosen by ideological litmus tests.

At least during Husky’s years, the tendency of judges was to move to the left in their thinking; conservatives appointed by Nixon or Ford ended up taking much more liberal positions than anyone could have expected, not excluding themselves.

Confined to private life, Husky did the opposite, moving not so much toward the right as away from the left — not in his political ideas, and certainly not in his championing of labor, but in his general outlook on society, which he regarded with mounting bile. And if his scorn for the phoniness of many liberals was very often justified, this had the unfortunate effect of focusing his criticism more sharply on those who betrayed the cause than on those who’d never espoused it to begin with. I wonder whether Husky might even have succumbed unawares to the perverse logic of “the enemy of my enemy is my friend.”

So far as actual rather than figurative friends went, Husky had ever fewer, for he grew estranged from those contemporaries who didn’t share his skepticism about phony liberals but instead stuck cheerfully to the party line (Democratic, of course).

 

Disillusionment

In 1999, ten years after Husky’s death, I found myself standing in the projection booth of the State Theater at Lincoln Center, where we were setting up our projections for the New York premiere of BIPED. Working with me were two others: my colleague Shelley Eshkar, who was configuring our playback computer, and Jack Young, who’d spec’d our equipment and was now setting up the projector.

A fourth man was also with us in the booth, sitting in a corner and reading the sports pages. He had no job to do but was well paid for not doing it — by week’s end, he’d have collected a paycheck far greater than those of the Cunningham dancers, even though their performances were to meet with standing ovations.

Union contract required this shadow’s presence, and while it insured the man’s excellent salary, it was at the cost of his actual worth, by which I mean his dignity.

The perversity of this situation made me think of Husky, who’d had a hand in establishing the principle behind it. For he must have negotiated such things as the house minimums of Broadway, which required that a large number of live musicians be present for every show no matter how many were actually needed to perform the music. Not so long ago, many such superfluous musicians simply had to show up to be paid. They were known as walkers, so named for their daily perambulations from home to theater and immediately back again.

If earlier on I suggested wishful thinking on my parents’s part as they looked back on Husky’s life, the same is likely true of me when I speculate that for a temperament as adverse to bullshit and procedure as Husky’s, the unintended consequences of the inflexible and quickly outmoded union rules he set up, however well-meant originally, must have given rise to a growing unease in his gut that was all the greater for his having (for the sake of his own self-regard) to suppress.

 

Diet and exercise

At home Husky’s diet relied increasingly on midnight raids on the fridge (for club sandwiches bulging with cold-cuts) and on television, which was almost always on. His running commentary was gruffly amusing and often incisive, but answering the fools on TV doesn’t do much to sharpen your wits. Other than the newspaper, I didn’t see him reading much else, and so the deeper arguments to be found in real books stopped challenging his habits of mind.

Just as your arteries tend to harden with age, so too does your thinking. Exercise of mind and body is said to go far in counteracting both effects of aging, but Husky was getting little of either.

 

Pain and soliloquy

As my memory brings Husky’s final scene into sharper view, I see that it was probably not at Husky’s bedside that Rebecca took down the lines for her Henry pieces. Instead she must have sat next to the substantial leather chair in which I always found Husky as he suffered through the protracted days of his dying. So agonizing was the pain of his cancer that it sometimes drove him temporarily out of his mind.

When I visited, I’d already hear his loud groans from the ground floor, so it was with dread that I climbed the stairs to his study, a dark room with a large television and desk. Next to the desk, a second door opened to a little bathroom connecting his study to the master bedroom he shared with his wife, my aunt Paula, to whom I was not related by blood and for whom I was not named.

Paula was an anesthesiologist who as a young lady had braved medical school to become a doctor in an era when women didn’t do that. They were meant to enter medicine only to become nurses, and then to be constantly at a male doctor’s beck and call.

Husky’s groans were not answered with the painkillers one would have expected his doctor, or his wife herself, to prescribe. His untreated agony was agonizing to witness, and I could not find it in my heart to forgive Paula for just standing by, still smoking her cigarettes. She insisted on not having her husband sedated into incoherence, and even mentioned fears of his getting addicted, as if that could possibly matter for this mortally ill man.

Only very recently did I learn more to account for Paula’s strange stance. As my parents tell it, Husky had earlier suffered a kind of psychotic break from his brain cancer, which his doctor had attempted to treat with thorazine only to provoke an even more intensely unpleasant reaction from him. Now Paula resisted giving him all but the most minor sedation, and the man was left on the rack of his body, subject to its unpitying torture.

By then Husky had lost most awareness of others and no longer talked much to us. Instead what poured from his mouth was long stretches from Shakespeare, speeches that he must have learned by heart as a youth and which had now come back to him with an astonishing intensity and eloquence.

If I could pretend to greater erudition than I possessed at the time, I would identify some of the lines that erupted from Husky in his anguish, no doubt opting for Lear. But because I recoiled from all the hamming-up of stage productions — from the feigned emotions overplayed by thespians — I kept myself then and for a long time afterward in ignorance of the Eternal Bard.

Still I was sure that Husky channeled Shakespeare more truly than had any actor declaiming to the far balconies. He may have mumbled and for all I know mixed up many of the lines, but his were true soliloquies — not stage-whispered to the complicit eavesdropping audience, but directed only inside himself, the sound rumbling around in his damaged lungs and the sense flaring up in his failing brain.