In this chapter, we are going to learn about tracking an object in a live video. We will discuss the different characteristics that can be used to track an object. We will also learn about the different methods and techniques for object tracking.
By the end of this chapter, you will know:
How to use frame differencing
How to use colorspaces to track colored objects
How to build an interactive object tracker
How to build a feature tracker
How to build a video surveillance system
This is, possibly, the simplest technique we can use to see what parts of the video are moving. When we consider a live video stream, the difference between successive frames gives us a lot of information. The concept is fairly straightforward! We just take the difference between successive frames and display the differences.
If I move my laptop rapidly from left to right, we will see something like this:
If I rapidly move the TV remote in my hand, it will look something like this:
As you can see from the previous images, only the moving parts in the video get highlighted. This gives us a good starting point to see what areas are moving in the video. Here is the code to do this:
import cv2
# Compute the frame difference
def frame_diff(prev_frame, cur_frame, next_frame):
# Absolute difference between current frame and next frame
diff_frames1 = cv2.absdiff(next_frame, cur_frame)
# Absolute difference between current frame and # previous frame
diff_frames2 ...Frame differencing gives us some useful information, but we cannot use it to build anything meaningful. In order to build a good object tracker, we need to understand what characteristics can be used to make our tracking robust and accurate. So, let's take a step in that direction and see how we can use colorspaces to come up with a good tracker. As we have discussed in previous chapters, HSVcolorspace is very informative when it comes to human perception. We can convert an image to the HSV space, and then use colorspacethresholding to track a given object.
Consider the following frame in the video:
If you run it through the colorspace filter and track the object, you will see something like this:
As we can see here, our tracker recognizes a particular object in the video, based on the color characteristics. In order to use this tracker, we need to know the color distribution of our target object. Following is the code:
import cv2 import numpy as np # Capture the input...
Colorspace based tracker gives us the freedom to track a colored object, but we are also constrained to a predefined color. What if we just want to pick an object at random? How do we build an object tracker that can learn the characteristics of the selected object and just track it automatically? This is where the CAMShift algorithm, which stands for Continuously Adaptive Meanshift, comes into the picture. It's basically an improved version of the Meanshift algorithm.
The concept of Meanshift is actually nice and simple. Let's say we select a region of interest and we want our object tracker to track that object. In that region, we select a bunch of points based on the color histogram and compute the centroid. If the centroid lies at the center of this region, we know that the object hasn't moved. But if the centroid is not at the center of this region, then we know that the object is moving in some direction. The movement of the centroid controls the...
Feature based tracking refers to tracking individual feature points across successive frames in the video. We use a technique called optical flow to track these features. Optical flow is one of the most popular techniques in computer vision. We choose a bunch of feature points and track them through the video stream.
When we detect the feature points, we compute the displacement vectors and show the motion of those keypoints between consecutive frames. These vectors are called motion vectors. There are many ways to do this, but the Lucas-Kanade method is perhaps the most popular of all these techniques. You can refer to their original paper at http://cseweb.ucsd.edu/classes/sp02/cse252/lucaskanade81.pdf. We start the process by extracting the feature points. For each feature point, we create 3x3 patches with the feature point in the center. The assumption here is that all the points within each patch will have a similar motion. We can adjust the size of this window...
Background subtraction is very useful in video surveillance. Basically, background subtraction technique performs really well for cases where we have to detect moving objects in a static scene. As the name indicates, this algorithm works by detecting the background and subtracting it from the current frame to obtain the foreground, that is, moving objects. In order to detect moving objects, we need to build a model of the background first. This is not the same as frame differencing because we are actually modeling the background and using this model to detect moving objects. So, this performs much better than the simple frame differencing technique. This technique tries to detect static parts in the scene and then include it in the background model. So, it's an adaptive technique that can adjust according to the scene.
Let's consider the following image:
Now, as we gather more frames in this scene, every part of the image will gradually become a part of the background...
In this chapter, we learned about object tracking. We learned how to get motion information using frame differencing, and how it can be limiting when we want to track different types of objects. We learned about colorspacethresholding and how it can be used to track colored objects. We discussed clustering techniques for object tracking and how we can build an interactive object tracker using the CAMShift algorithm. We discussed how to track features in a video and how we can use optical flow to achieve the same. We learned about background subtraction and how it can be used for video surveillance.
In the next chapter, we are going to discuss object recognition, and how we can build a visual search engine.
© 2015 Packt Publishing Limited All Rights Reserved
