Unit Testing (a short introduction)

In case you don't know it Unit Testing is all the rage in serious, professional, buttoned-down software engineering. Depending on who you listen to Unit Testing (part of Test Driven Development, part of Agile programing) is either a good idea, a great idea or the greatest idea in computer programming since the invention of the compiler.

What is it? Unit testing, in short, is the practice of carefully testing the code you are working on by writing yet more code to automatically verify that it's working. In the Unit Testing methodology, you try and test the smallest parts of the code — the 'units' — and you try and do it while you are working on the code itself (rather than leaving the "testing" to a great big messy phase at the end). The idea is that the effort involved in rigorously (and frequently) testing the code you write is more than worth it in the long run. It's easier to write lots of small tests and catch the bugs while they are small then go hunting for a small bug in a big application.

So, as soon as you write a piece of code that says:

def computeMyNumber():
    return 4

You immediately turn around and write something like:

def test_computeMyNumber():
    if (computeMyNumber()!=4):
        print "ERROR: computeMyNumber isn't 4!"

Obviously that's a silly example. Firstly, computeMyNumber is going to be much more complicated than that. But secondly, if your tests were that verbose you'd go mad writing them. So Unit Testing enthusiasts build frameworks and tools to help them write really terse tests and execute them in huge batches. Then, after any and all edits to their codebases, they can just fire off all of their tests with a push of a button. This verifies that they haven't accidentally taken a step backwards rather than forwards.

After a little while working this way you'll discover a new kind of confidence about your code that's hard to describe but its addictiveness is probably the reason for the 1.5 million hits for "unit testing" in Google. It's an industry. It's a philosophy. It might even be a cult.

But the writing-code corner of the digital art world has been largely unaffected by these ideas. Partly because people are generally dealing with small codebases and small programming teams; partly because many of the things that we care about the most are hard to formally specify and automatically test; and partly because none of the tools of the trade really support or encourage such things.

In fact we've (OpenEnded group and friends) been essentially anti-unit testing for years; basically as a matter of gut instinct or neurochemistry or something — we'd rather make a bold unexpected mess and live the fight to survive it than wake up every day and make orderly progress. But recently we've begun to be convinced that we can introduce a little bit of Unit Testing into our creative non-linearity.

Unit Testing in Field

Field introduces the worlds smallest, least cultish unit testing framework (maybe the first in a digital art environment ?). It aims for minimal over almost everything else. That said, we're pretty excited about how such a small footprint feature is changing how we think about code.

Let's start with any old box of code in Field:

There's nothing special about the code that's in there:

def sum1toN(n):
   return sum(range(1, n+1))
 
def sumMultiples(limit, a):
   return sum1toN((limit - 1) / a) * a
 

I just copied it off of wikipedia. It really could be anything — a call out to some Java; something that uses a Processing library; something that uses the Max/MSP plugin. It could be a function defined written in Field someplace else.

But it's the code that we're going to test for the purposes of this example.

I wrote sum1toN with the intention that it sums up the numbers between (and including) 1 and N. If I wanted to test this I'd go:

print sum1ToN(1)

I'd keep the cursor on this line and jab command-return to execute it. If all goes well two things will happen. Firstly, I'll get the answer '1' printed to the bottom of the text editor window — great, the sum of the numbers between 1 and 1 inclusive is 1. Secondly, I'll get a little grey area off to in the "ruler" area of the text editor:

If you don't know what that grey area is, it's the execution ruler. It pops up a mark for any range of code that you execute in the text editor. To execute something again, you can just 'option click' on the grey area and off it goes. The areas are saved with your document and, best they can, track the contents of the editor. You can even assign keyboard shortcuts to these areas.

Field's unit testing feature hooks into this spot. Right clicking on the area gives us a new option:

(Once you get into this you can just press ^U instead of command-return, right-click, select item.)

This gets you a new unit test:

Field has automatically executed the text and captured the last line that it 'printed'. It's taken that line and placed it in a comment.

Let's make some more:

Professional, tie-wearing unit testing frameworks that get books written about them have all kinds of different ways of 'asserting' the working-ness of code. We have exactly one — the print statement.

Lastly, we complete our work here and realize that we've actually built a solution to the first problem in Project Euler):

print sumMultiples(1000, 3) + sumMultiples(1000, 5) - sumMultiples(1000, 15)
#->233168

print sum(x for x in range(1, 1000) if (x%3)==0 or(x%5)==0)
#->233168

print (sum(x for x in range(1, 1000) if (x%3)==0 or(x%5)==0)) == (sumMultiples(1000, 3) + sumMultiples(1000, 5) - sumMultiples(1000, 15))
#->True

Work done we go away for a week, or longer. We make some art. We forget all about this code.

One day we return and we notice that our sum1toN function is way slower than it needs to be. We remember Gauss's "trick" for summing all the numbers between 1 and N. And we rewrite the definition of sum1toN based on our hazy memory of the formula:

def sum1toN(n):
     return n*(n-1)/2

Because we've change some code we'd like to rerun all of our pertinent unit tests. In Field we can run a unit test in one of three ways. We can just press ^u again when our cursor is in the right place. Or we can right click on the big 'u' in the margin:

Of course, when we run it, we are confronted by the truth:

Indeed, if we select "run all unit tests":

Gulp. You'll note that the the box itself now has a very harsh red badge of unit test failing shame written on it. After all, there's no point writing and running these test if you can't see the ones that fail. You can even run unit tests without looking at the editor — right click a the visual element containing tests to get a menu option.

Ultimately we realize that we misremembered Gauss's formula and fix it to read:

def sum1toN(n):
     return n*(n+1)/2

Rerunning the tests gets us back to pale green.

One final note, should we decide that we like our new definition of Guass's formula better for some reason (maybe it makes for a great image somewhere else) we can select "run and & revise" from the menu instead — this revises the unit test with some new output.

Conclusions

Note that, simple as it is, the unit testing feature of Field has quite a bit going for it. In our toy example above we know that the error is in sum1toN (rather than sumMultiples or some other piece of code miles away that uses sum1toN because we have a test for it. But we can immediately see failed output (and start thinking about why it's 0 not 1). Unlike, say, a Java or C based framework we never leave the interpreted language world. When a test fails we are immediately in the context of that failure — essentially we are already in the debugger, ready to inspect the bit that failed. The delay between locating the failure and jumping into the problem is very short. And unlike other frameworks the tests themselves are usually located right next to the piece of code that they are testing — indeed documenting. Outside of the ^U world the tests are print statements and comments; generally they can just hang out.