Tracking Visual Motion

Video sequences are interesting because they show scenes and objects in motion. The preceding chapter introduced the tools for reading, processing, and saving videos. In this chapter, we will look at different algorithms that track the visible motion in a sequence of images. This visible or apparent motion can be caused by objects that move in different directions and at various speeds, or by the motion of the camera (or a combination of both).

Tracking apparent motion is of utmost importance for many applications. It allows you to follow specific objects while they are moving in order to estimate their speed and determine where they are going. It also permits you to stabilize videos taken from handheld cameras by removing or reducing the amplitude of camera jitters. Motion estimation is also used in video coding to compress a video sequence in order to...

Tracing feature points in a video

We learned in previous chapters that analyzing an image through some of its most distinctive points can lead to effective and efficient computer vision algorithms. This is also true for image sequences, in which the motion of some interest points can be used to understand how the different elements of a captured scene move. In this recipe, you will learn how to perform a temporal analysis of a sequence by tracking feature points as they move from frame to frame.

How to do it...

We will now start with the tracing of feature points in a video, frame by frame. Let's take a look at the following steps:

1. To start the tracking process, the first thing to do is to detect the feature points...

Estimating the optical flow

When a scene is observed by a camera, the observed brightness pattern is projected on the image sensor and thus forms an image. In a video sequence, we are often interested in capturing the motion pattern, that is, the projection of the 3D motion of the different scene elements on an image plane. This image of projected 3D motion vectors is called the motion field. However, it is not possible to directly measure the 3D motion of scene points from a camera sensor. All we observe is a brightness pattern that is in motion from frame to frame. This apparent motion of the brightness pattern is called the optical flow. You might think that the motion field and optical flow should be equal, but this is not always true. An obvious case can be the observation of a uniform object; for example, if a camera moves in front of a white wall, then no optical flow is...

Tracking an object in a video

In the previous two recipes, we learned how to track the motion of points and pixels in an image sequence. In many applications, however, the usual requirement is to track a specific moving object in a video. An object of interest is first identified, and then it must be followed over a long sequence. This is challenging because, as it evolves in the scene, the image of this object will undergo many changes in appearance due to viewpoint and illumination variations, non-rigid motion, occlusion, and more.

This recipe presents some of the object-tracking algorithms implemented in the OpenCV library. These implementations are based on a common framework, which facilitates the substitution of one method by another. Contributors have also made a number of new methods available. Note that we have already presented a solution to the object-tracking problem...