Keyframing in Code

Every digital artist is familiar with the keyframe — in Final Cut, AfterEffects, Logic, Soundtrack, Maya, Max, MotionBuilder, what have you. A keyframe has three elements: the data — a (time, value) pair; a curve — that is a path that takes you from some keyframe to another; and the math — some kind of algebraic structure that allows this curve to have meaning (typically "blending between", or "blending on-top-of").

Whereas Final Cut has data, curves and simple maths, Field has code: code that sets numbers, code that draws curves and code that does math. What kinds of keyframing systems can we build using Field that subsume the "simple" but very useful keyframes of standard software packages while retaining the all the things we like about code?

On this page we'll develop the answer to that problem. In terms of "the curve", let it suffice for now to say that we can draw and edit curves using BasicDrawing and intersect them using LineUtils and intersect them with Intersections — enough to build your own curves for keyframes package.

As for data, Field contains a framework for treating code a little bit like data, and allowing one to do math on code.

KeyFrameCreator

Consider the following code:

theVector = Vector3()

k = KeyFrameCreator()

_k = k.begin()
_k.theVector.x = 3
keyframe1 = k.end(_k)
print theVector    # prints (3, 0, 0)

_k = k.begin()
_k.theVector.x = 2
keyframe2 = k.end(_k)
print theVector    # prints (2, 0, 0)

keyframe1.apply()
print theVector    # prints (3, 0, 0)

Our objects keyframe1 and keyframe2 contain the information about the side-effects of assignments made through the intermediate _k objects. You can go as deep as you like (if it make sense to write T.tsl.shaderParameters.color#Vector4(1,1,0,1) it's legal to write _k.T.tsl.shaderParameters.colorVector4(1,1,0,1). And, of course, you have write to multiple, different, global variables within the same keyframe — I'm just keeping the examples short here.

Thus side effects become a little like data, in particular you can manipulate them as data:

(keyframe2*0.6666+keyframe1*0.3333).apply()
print theVector  # prints (2.33333, 0, 0)

This is sufficient to perform a linear blend over keyframes, which is, in turn, sufficient to construct higher order interpolants. (Note. apply() is normalizing its weights)

The keyframe objects in the current version of Field are already fairly intelligent. For example, they understand that direct field access .x=3 and method calls on Vector3's are having side effects on the same thing:

theVector = Vector3()

k = KeyFrameCreator()

_k = k.begin()
_k.theVector.x = 3
_k.theVector.add(Vector3(1,1,0))
keyframe1 = k.end(_k)

_k = k.begin()
_k.theVector.x = 2
_k.theVector.add(Vector3(1,1,0))
keyframe2 = k.end(_k)

(keyframe2*0.6666+keyframe1*0.3333).apply()
print theVector # prints (3.333333, 1.666667, 0.0)

You might need to stare at this example a while to prove to yourself that it really should print 3.333333, 1.666667, 0.0. Field is capturing the side effects of theVector.add(...), because it knows how to (it knows a lot about Vector3's). Field could have taken an alternative route — it could blend the parameters to theVector.add(...) — this is actually easier to code, but less useful. A keyframe is supposed to capture the state of some variables, not how those variables came to that state — a keyframe is a destination, not a journey.

They understand, and can blend, things like VectorN, Quaternions, and so on. You can add objects to this list of small, common, mutatable objects by providing iBlendSupport, iCloneSupport, and iMutationSupport implementations to AssemblingLogging. They can also handle array accesses, and the setting of fields in objects that they don't know anything about. Some things are missing right now — slice notation on vectors for example doesn't work, and special python objects, most notably _self aren't there yet, but arbitrary python objects and global variables should work just fine. File bug reports!

But for method calls on unknown object types (where AssemblingLogging cannot capture the potentially arbitrary side-effects of a method call: remember this is still really Java, not Haskell) the best that we can do is to fall back to the second option — and capture the parameters to the method call, and blend over those. This is often exactly what you want when you have a method call that wipes a mutable variable clean and starts from there (an example from our piece Forest T.tsl.setColorLookupTable(Vector3(1,0,0,1), Vector3(1,1,1,1)) sets up a look-up table)

Finally, we are only one more blend away from being able to write (back to our original example)

theVector.x=0
((keyframe2*2+keyframe1)/3.0).apply(0.5)
print theVector # (1.1666666,0.0,0.0)
((keyframe2*2+keyframe1)/3.0).apply(0.5)
print theVector # (1.75,0.0,0.0)
((keyframe2*2+keyframe1)/3.0).apply(0.5)
print theVector # (2.04166667,0.0,0.0)

That is, .apply(0.5) moves "half way" between wherever all of the elements that the combined keyframe touches current are to where the keyframe tells them they should be — ideal for hacking in a little smoothing.

Laziness

Two routes for extension. Firstly, a little laziness :

def randomNumber():
    return float(Math.random())

theVector = Vector3()

k = KeyFrameCreator()

_k = k.begin()
_k.theVector.x=3
keyframe1 = k.end(_k)

_k = k.begin()
_k.theVector.x = randomNumber
keyframe2 = k.end(_k)


((keyframe2*2+keyframe1)/3.0).apply(1)
print theVector # prints, say, (1.0271876, 0.0, 0.0)
((keyframe2*2+keyframe1)/3.0).apply(1)
print theVector # prints, say, (1.43536, 0.0, 0.0)
((keyframe2*2+keyframe1)/3.0).apply(1)
print theVector # prints, say, (1.513276, 0.0, 0.0)

Illegal assignment (illegal in the sense of Java's type system, you can't assign a function to a float member field) is converted into lazy evaluation. Every .apply() the randomNumber() function is evaluated.

Modified blending

Often a linear blend over the scalars and Vectors &c that make up a keyframe is what you want, but sometimes it's only close to what you want. To "override" the blend coefficients, or the blending altogether :

def middleBlender(sharpness, value, depth=1):
    def b(blendSupport, values, weights, newValue):
        m = depth*(1-Math.pow(Math.abs(max(weights)-0.5)*2, sharpness))
        weights = [1-m, m]
        values = [newValue, value]
        return blendSupport.blend(values, weights)
    return b

keyframe1.addBlender("theVector", middleBlender(1, Vector3(0,0,0), depth=0.5))

This causes theVector, when blended, to "dip" down to the origin when the blend is far away from examples. This kind of perturbation of the straight line is often enough to keep complex sets of parameters out of parts of the blend space that you don't like.

Finally, if these kinds of things interest you, don't miss the 'extended assignment' discussion at the bottom of GeneratorExamples and the Context Object.