Processing Video Sequences

Video signals constitute a rich source of visual information. They are made of a sequence of images, called frames, that are taken at regular time intervals (specified as the frame rate, generally expressed in frames per second) and show a scene in motion. With the advent of powerful computers, it is now possible to perform advanced visual analysis on video sequences—sometimes at rates close to, or even faster than, the actual video frame rate. This chapter will show you how to read, process, and store video sequences.

We will see that, once the individual frames of a video sequence have been extracted, the different image processing functions presented in this book can be applied to each of them. In addition, we will also look at a few algorithms that perform a temporal analysis of the video sequence, compare adjacent frames to track objects...

Reading video sequences

In order to process a video sequence, we need to be able to read each of its frames. OpenCV has put in place an easy-to-use framework that can help us perform frame extraction from video files or even from USB or IP cameras. This recipe shows you how to use it.

How to do it...

Basically, all you need to do in order to read the frames of a video sequence is create an instance of the cv::VideoCapture class. You then create a loop that will extract and read each video frame. Here, we are going to explain how to code the main function to show each frame. The following steps will help us to read video sequences:

1. Open a video capture with the following code:

int main() 
{ 
  // Open the video file 
  cv...

Processing video frames

In this recipe, our objective is to apply some processing function to each of the frames of a video sequence. We will do this by encapsulating the OpenCV video capture framework into our own class. Among other things, this class will allow us to specify a function that will be called each time a new frame is extracted.

How to do it...

What we want is to be able to specify a processing function (a callback function) that will be called for each frame of a video sequence. This function can be defined as receiving a cv::Mat instance and outputting a processed frame as follows:

1. Therefore, in our framework, the processing function must have the following signature to be a valid callback:

void processFrame...

Writing video sequences

In the previous recipes, we learned how to read a video file and extract its frames. This recipe will show you how to write frames and, therefore, create a video file. This will allow us to complete the typical video-processing chain—reading an input video stream, processing its frames, and then storing the results in a new video file.

How to do it...

Let's perform the following steps:

1. Writing video files in OpenCV is done using the cv::VideoWriter class. An instance is constructed by specifying the filename, the frame rate at which the generated video should play, the size of each frame, and whether or not the video will be created in color:

writer.open(outputFile, // filename 
    codec...

Extracting the foreground objects in a video

When a fixed camera observes a scene, the background remains mostly unchanged. In this case, the interesting elements are the moving objects that evolve inside this scene. In order to extract these foreground objects, we need to build a model of the background, and then compare this model with a current frame in order to detect any foreground objects. This is what we will do in this recipe. Foreground extraction is a fundamental step in intelligent surveillance applications.

If we had an image of the background of the scene (that is, a frame that contains no foreground objects) at our disposal, then it would be easy to extract the foreground of a current frame through a simple image difference:

  // compute difference between current image and background 
  cv::absdiff(backgroundImage,currentImage,foreground); 

Each pixel for which...