Apart from everything we have seen up until now throughout this book, there is another side to the computer vision story, and that is dealing with videos, cameras, and essentially real-time processing of the input frames. It is one of the most popular computer vision topics and for good reason, since it can power live machines or devices that monitor their surroundings for objects of interest, movements, patterns, colors, and so on. All of the algorithms and classes that we have learned about, especially in Chapter 6, Image Processing in OpenCV and Chapter 7, Features and Descriptors, were meant to work with a single image, and for this same reason they can be easily applied to individual video frames in the exact same way. We only need to make sure individual frames are correctly read (for instance using the cv::VideoCapture class) into cv::Mat class instances and then passed into those functions as individual images. But when dealing with videos, and by videos...
As was mentioned in the introductory part of this chapter, there are a concepts in computer vision that are especially important when dealing with video processing and the algorithms we'll talk about later on in this chapter. One of those concepts is histograms. Since understanding histograms is essential to understanding most of the video analysis topics, we'll go through quite a bit of information about them in this section, before moving on to the next topics. A histogram is often referred to as a way of representing the distribution of data. It is a very simple and complete description, but let's also describe what it means in terms of computer vision. In computer vision, a histogram is a graphical representation of the distribution of pixel values in an image. For example, in a grayscale image, a histogram will be a graph representing the number of pixels that contain each possible intensity in the grayscale (a value between 0 and 255). In an RGB color image...
Apart from the visual information in a histogram, there is a more important use for it. This is called back-projection of a histogram, which can be used to modify an image using its histogram, or as we'll see later on in this chapter, to locate objects of interest inside an image. Let's break it down further. As we learned in the previous section, a histogram is the distribution of pixel data over the image, so if we somehow modify the resulting histogram and then re-apply it to the source image (as if it was a lookup table for pixel values), the resulting image would be considered the back-projection image. It is important to note that a back-projection image is always a single-channel image in which the value of each pixel is fetched from its corresponding bin in the histogram.
Let's see this as another example. First of all, here is how a back-projection is calculated in OpenCV:
calcBackProject(&image,
1,
channels,
histogram...What we learned until now in this chapter, apart from the use cases that we already saw, was meant to prepare us for correctly using the and CamShift algorithms, since they extensively benefit from histograms and back-projection images. But what are the and CAMShift algorithms?
Let's start with the MeanShift and then move on to CamShift, which is basically the enhanced version of the same algorithm. So, a very practical definition for MeanShift (as it is stated in the current OpenCV documentation) is the following:
Finds an object on a back projection image
That's quite a simple yet practical definition of the MeanShift algorithm, and we are going to stick to that more or less when we work with it. However, it's worth noting the underlying algorithm, since it helps with using it easily and much more efficiently. To start describing how MeanShift works, first, we need to think about the white pixels in a back-projection image (or binary images in general) as scattered...
Background/foreground detection, or segmentation, which is often also to as background subtraction for quite good reasons, is the method of differentiating between the moving or changing regions in an image (foreground), as opposed to the regions that are more or less constant or static (background). This method is also very effective in detecting motions in an image. OpenCV includes a number of different methods for background subtraction, with two of them being available in the OpenCV installation by default, namely BackgroundSubtractorKNN and BackgroundSubtractorMOG2. Similar to the feature detector classes we learned about in Chapter 7, Features and Descriptors, these classes also originate from the cv::Algorithm class, and they are both used quite easily and similarly since they differ not in the usage or the result, but in the implementation of the classes.
BackgroundSubtractorMOG2 can be used to detect the background/foreground by using the Gaussian...
Writing computer vision applications that perform real-time image processing is one of the hot topics of today, and OpenCV contains many classes and functions to help with simplifying the development of such applications. In this chapter, we tried to cover some of the most important classes and functions provided by OpenCV for real-time processing of videos and images. We learned about the MeanShift, CamShift, and background subtraction algorithms in OpenCV, which are packed into fast and efficient classes which are, at the same time, very easy to use, provided that you are familiar with the basic concepts used in most of them, such as histograms and back-projection images. That is why we started by learning all about histograms, how they are calculated, visualized, and compared with each other. We also learned how back-projection images are calculated and used as a lookup table to update images. We used the same also in the MeanShift/CamShift algorithms to track objects of specific...
© 2018 Packt Publishing Limited All Rights ReservedAuthor (1)
