Simple Optical Flow

SimpleHead is a simple program that analyzes movement and a few other things in video (offline) or from a camera (real-time). It runs on Macs, and Field can use either its analysis product files or talk to it live. Installation instructions are at the bottom of the page.

Realtime use

Field can initiate a SimpleHead with the following code:

var SimpleHead = Java.type("trace.video.SimpleHead")
SimpleHead.start("/Users/marc/temp/head/")

As you can see, Field needs a path to wherever you’ve put the binary that you’ve downloaded. The path is the folder.

Run that (once) and you’ll see something like this:

Those two windows can be safely hidden.

Having started a SimpleHead you can now ask for parts of its analysis:

SimpleHead.now contains the current set of values for the last frame that was analyzed. These include:

vx and vy — ‘velocities’. Average amounts of motion (in ‘pixels’) in the x and the y direction. This is correctly oriented for Field (positive vx is right, positive vy is down).
brightness. Average brightness (note that almost all webcams auto expose, so this is only useful over the short-term).
red, green, blue. Average red, green and blue.
meanv_x and meanv_y are the ‘position’ of where the motion is occurring. That is, the average ‘x’ and ‘y’ position weighted by the amount of motion there. This only makes any sense at all if there’s a relatively compact object that’s doing most of the moving.
meanb_x and meanb_y are the ‘position’ of where the brightness is occurring. That is, the average ‘x’ and ‘y’ position weighted by the amount of brightness there. This only makes any sense at all if there’s a relatively compact object that’s brighter than the background.

Offline use

_.stage.withImageDirectory now looks for .simpleHead files in parallel with it. Given that you can ask a layer for a .head and get the same kind of object as SimpleHead.now returns for the frame (or mix of frames) that they layer is currently showing. For example, this code here, which combines some standard code for playing out a video on a Stage with some code that draws a line in the direction of motion:

var t = 0

while(true)
{
	
	// play out a video stretched over the whole stage
	
	var layer = _.stage.withImageSequence("/Users/marc/Documents/Videos/Pexels Videos 1793414.mp4.dir/")

	
	var f = new FLine().rect(0,0,100,100)

	f.color=vec(1,1,1,1)

	layer.bakeTexture(f)

	f.filled=true
	layer.lines.f = f 
	
	t = t + 1

	layer.time = 0.2+t/10000.0


	// draw a line from the middle of stage out in the direction of
	// the average flow

	// we need another layer (because we don't want it textured)
	var arrowLayer = _.stage.withName("arrowLayer")
	
	// a line from 50,50 to 50,50 + average flow
	var arrow = new FLine()
	arrow.moveTo(50,50)
	arrow.lineTo(50 + layer.head.vx*10, 50 + layer.head.vy*10)

	// a bright red line
	arrow.color = vec(1,0,0,1)
	
	// thats 1 pixel thick
	arrow.thicken = 1
	
	arrowLayer.lines.arrow = arrow
	
	
	_.stage.frame()
}

Yields:

Installation [Mac Only]

If you want to process your own files, or run SimpleHead in real-time, there’s a three stage process.

Install Homebrew

First you’ll need to install some command line tools called Homebrew. You can read all about that here. Copy and paste this into a new terminal window:

/usr/bin/ruby -e "$(curl -fsSL https://raw.githubusercontent.com/Homebrew/install/master/install)"

Next enter your password when prompted. That should install Homebrew and the Apple Developer Tools. Don’t, in general, trust websites that download random things to your machines and ask for your password…

Install ffmpeg

Now that you have Homebrew installed you can install other things easily. We need ffmpeg which is a command line program for manipulating video.

brew install ffmpeg

That was easy!

Install SimpleHead

Download SimpleHead. If you are using Safari, get it out of the all-too-special Downloads folder and put it somewhere else (like your desktop).

Offline / analyze a movie

The magic command in the terminal for analyzing a movie, turning it into a .simpleHead file, looks like this:

/Users/loganmediacenter/Desktop/head/simpleHead myAwesomeMovie.mp4 > myAwesomeMovie.simpleHead

You’ll have to stare at that carefully though (like anything else from the command line). There’s three parts — the command simpleHead, your movie file and then the name of your simpleHead file that you want to create, which should be the same name as your .dir directory for Field, but with .simpleHead rather than .dir. One way to make sure you spell everything right is to exploit the fact that you can drag things from the finder into Terminal and have them written out for you. So, drag the simpleHead executable into Terminal, drag your mp4, write the > and so on.