For sketch 3 we’re distributing some code that analyses videos, webcams and other camera sources and produces a range of analysis products in a range of formats and protocols. This page will go over how to install it, how to run it, and (hopefully) give those of you without any command line experience just enough knowledge to survive the terminal. The goal here is twofold:
get state-of-the art, historically important image and video analysis codes into your hands, onto computers that you can use.
make the codes simple and small so that you can look at them and figure out some things about how they work.
Since this code consists of nice little pieces of a programing language called python, this package is called pythonhead. A lot of this stuff is (supposed) to be kept strictly to ‘the priesthood’ — let’s try to break that down a bit.
Installation, part 1 — Python
First we’re going to make sure that you have a recent installation of python on your machine. Python is a popular, and very well designed, programming language that lets you write code that runs on both Windows and Mac (and Linux).
If you are on a UChicago Lab mac, for example, it’s likely that python3 is already a thing. Skip to part 2 below
You might have a version of Python already installed. To check, type python3 on a Mac (in a Terminal) or python on a Windows machine (in a cmd window). If you do, skip the installation instructions and proceed to part 2.
Go download and install the latest and greatest version of Python 3 from here. It’s probably something like version 3.7.3 or so. Once you are done, you should be able to open up a command prompt (by running cmd from the start menu) and type python without getting an error.
Install Homebrewfrom here — you’ll need the root password of the machine you are on (for the Logan center lab machines just ask). Then ask Homebrew, in a terminal, to go install the latest version of python for you:
Installation, part 2 — Dependancies
Now ask Python to go install almost everything that we’ll be needing for our computer vision work:
(for windows you’ll write pip rather than pip3)
Installation, part 3 — pythonhead
Finally, we’ll install our code, which is a tiny little thing called pythonhead sitting atop of all of this other code! It’s available here. Decompress that and put it somewhere that’s easy to get to (like your desktop).
To run this code you are going to have to issue a command like:
(for windows you’ll write python rather than python3)
Which might give you something like:
But it very much probably wont! It will much more likely complain that it can’t find process.py. Why?
To survive in the command line we need to be very precise about what we want to do and much of that precision comes about from specifying what files we want to do things to. We want to “run” a file called process.py which is part of that .zip file you decompressed, so we need to specify where that file is precisely. It might be something like:
If you that file ended up in a folder called pythonhead on your desktop and you are called marc on a mac. On Windows it would be c:\Users\marc\Desktop\pythonhead\process.py (note the ‘slashes’ occasionally go the other way on Windows). Likely, you are called something else, so what you need to type will vary. That goes for every command on this page.
Four tips to make this easier.
You can drag files from the Finder / Explorer into the terminal / cmd prompt to have the computer spell out what they are actually called.
If you are half way through typing a filename you can press tab to finish it (if there’s only one file or folder that starts that way) or press tab twice to see alternatives. If it doesn’t autocomplete the way you expect (or at all) consider that you might have a typo.
the up and down arrows move through things you have already typed so you don’t have to keep typing them over and over.
every terminal or cmd prompt has a notion of a ‘current directory’ which is where it looks for files if you don’t type their complete ‘name’. You can ‘change directory’ by typing cd name/of/directory (in both Windows and Mac). This can save (a lot) on typing.
Really running pythonhead this time.
Now that we know how to run this code, what to do with it? Pay attention to the --help description. It tells you the kinds of things you can say after the python3 process.py. For example:
[remember, for windows it’s python not python3]
Will run the analysis on a movie called ‘great_movie.mov’ inside my Movies directory (see tip 1 above) while doing the optical flow analysis and the line detector. If everything works then you’ll see something that looks like:
Accompanied by some windows:
All those numbers scrolling by (which are mostly the results of do_lines) are the data. While we might enjoy being awash in data, we’d really like to save it. We can do this using some more syntax in the terminal:
That will create a file on your (my?) desktop called ourGreateData.json. We’ve stuck .json on the end because that’s the kind of file it is. Although the nice thing about .json files is that they are “human readable” (as well as machine readable). It starts something like this:
It’s well worth hacking your way through the file to find out what it contains. Things to try:
The various do_xxx options turn on various analyses.
--compute_contours will add to the --do_background option the ‘geometry’ of the foreground / background interface.
skipping the --video option means that we’ll try to use a built-in camera instead (and, if you have two such ‘webcams’, you can write --video 0 and --video 1 to tell them apart)
--output_directory /name/of/directory will cause the creation of that directory (and some sub-directories) that save the input and output of all of the analysis. This can be handy if you just want to use the video ‘products’
If everything is running a bit too slow, try passing in a --shrink parameter. It defaults to --shrink 0.5, so try --shrink 0.25? Remember: due to the debayering that cameras do, half those pixels are fake anyway.
Transforming the data further
Along with process.py there are two other python files that might interest you. The first tocsv.py converts json to CSV (stands for comma separated values — i.e a file with rows of numbers separated by commas). You can load these files into After Effects and lots of other things (for example, Excel, R and so on).
You can run tocsv.py on an existing json file using another piece of terminal / cmd syntax. For Mac:
Field alpha.27 contains support for running and connecting to the pythonhead process directly. Alas, digging into theses datastructures that pythonhead produces isn’t the most beginner-friendly thing to think about (think about this: a set of ‘tracks’ for a frame is a list-of-lists-of-lists. Yikes! Unpacking that a little: the set of ‘tracks’ for a frame is a list (over all of the ongoing tracks) of tracks which consist of a list of points (going backwards throught time) where each point is a list of two numbers (the x and y position of the point). We made it!).
Here’s some example code, a ‘cookbook’ if you will, for a few of the things that pythonhead produces; for actually figuring out how to draw lines in Field, you’ll want to start with the documentation. There’s a command in each snippet of code that points Field to pythonhead — you’ll want to change this line to point to where you put pythonhead, which is unlikely to be exactly where I put it. So, if you put it on the desktop on a mac it will be something like:
On the UChicago CSIL Lab Macs its:
On windows it’s a little more involved:
The simplest set of numbers to grab from pythonhead is the bulk statistics over the motion.
This is just drawing a line from the position of the “average moving thing” (mean_motion_px, py) in the direction of motion (mean_motion_x,y):
Its very wobbly, let’s smooth it:
The lines of a frame is a list (all the lines) of list (the start and the end of the line) of lists (with two numbers, x and y). We can plot them like this
We can then start to refine this data and change the way that it’s drawn:
Extending this further and further we could have:
Contours and Tracks
Contours and tracks use almost exactly the same code to draw them (they are both lists of lists of lists); unlike the line detector they can consist of a variable number of points (that middle list doesn’t always have just 2 things in it).
So we can do code like this:
Changing PythonHead.now.contours to PythonHead.now.tracks, and the launch parameters from --do_background to do_tracks and the same code will draw the motion trails.
Things to remember
Machines, operating systems and cameras vary — there is almost certainly an instruction on this page that needs refining. Pay attention to things that look like error messages, and ask for help in the slack channel when things to awry, if in doubt, ask questions with screenshots.
Once everything’s working, ask for changes — new analyses, new transformations.