Alrighty Then!

A friend called me and asked how i would measure and present the heights archieved at a snowboard highjump contest. He promised loosely to make contact with someone organizing such events. After thinking about it for 10 minutes, i couldn’t resist and had to throw together a prototype.

First i gave google a chance to show me what has been done so far. Found a good hint for “difference” blend mode to get changes in the image. Found nothing really good for motion and object detection.

Approach #1

I went off with approach #1: I determined the changes in the image by blending the two consecutive frames with “difference” blend mode. Then computed the change-rate average, minimum and maximum on 10×10 pixel areas and stored the areas that were more than a certain delta above the average change-rate. Now computed the averaged position and variance of the stored areas. The average position gives a good estimate on a single moving object, the variance is a measure of the “sharpness” of the recognized motion. All of this worked ok, but was way to slow to be near realtime.

Approach #2

I monitored (and averaged) the camera’s activityLevel. If it went more than a certain delta above the average activityLevel i started to save away every camera image. If activity went below the limit for more than some frames, i stopped “recording” and triggered post-processing (as developed in approach #1) for every pair of frames recorded. The image sizes had to go down to 1/4th (1/16th area) of the original 320×240 images to give a reasonable responsiveness. Calculating the difference between consecutive images results in 2 ghost-images of the visible motion: one from where the moving object was in frame one and a second one where it is in frame two. Also, when a thrown object (or a snowboarder) reaches its highest point, it “freezes” for 2-3 frames so that no motion can be detected at the point we need to measure. This led to …

Approach #3

Now using BitmapData’s merge and a ringbuffer of BitmapData’s i could iteratively calculate a averaged image of the live video (in realtime! @ 25fps). Postprocessing changed to calculate the image-changes from this average-image to the recorded frames: the ghost-images went away. Then the delta-images where converted to grayscale (using the matrix-parameter of BitmapData’s draw) saving some time when averaging.

In the online prototype you have live video on the left side, the averaged image on the right. The recorded film-strip with every frame’s delta-image at the bottom. The delta-images have red pixels drawn where motion is detected and rectangles at the centroid. The replay-panel switches to the frame where a object is detected “highest” for 4 seconds.

Where to go from here:

• make the replay-window show a slow-motion of the recorded frames with the measured height shown in the movie
• optimize motion detection to be able to detect multiple objects
• optimize parameter auto-adjust

Summary:

With Flex’ functions for scaling, color-conversion and blending i was able to put together an impressive application (at least for me) in reasonable time and with good performance. The only thing that really is missing is a performance optimized way to get a simple custom function (average, minimum, maximum …) executed on whole image areas.