Chapter 5. Image Derivatives and Edge Detection

In the very first chapter, we made a distinction between the terms image processing and computer vision. We made it clear that the outputs of the operations that fall under the umbrella of image processing are images themselves. So, an image-processing operation will take an image as input and produce yet another image as its output. Essentially, all the techniques that we have covered so far are image-processing operations: grayscale transformations, image filtering, and image thresholding. Computer vision tasks are slightly more interesting. The inputs for computer vision algorithms are again, images. But, the outputs are what we call symbols. These symbols represent some form of semantic information that the algorithm has derived from the image. The kind of semantic inference that is done by vision algorithms is quite close to what human beings would do. An example of a computer vision task in an image would be to separate the foreground...

Image derivatives

Those of you who may have taken up a basic calculus course in your high school will know the definition of a derivative as it applies to functions. For the sake of others, we repeat the same here, albeit very briefly. Mathematically, a derivative is represented as follows:

Note

Note that we are not trying to be mathematically fastidious, our only intention here is to give you an intuition into the concept of a derivative and how it applies to images. So, this chapter won't go into the intricacies behind the mathematics that is involved. Rather, our focus will be on how the principles of derivatives of functions are transferred to the domain of images.

What the preceding formula essentially tells you is that the derivative of a function f(x) at any point is the ratio of the change in the output to the input, as the input is varied by an infinitesimal amount in and around that point. Imagine that you have a 2D plot of the graph of the function f(x). You sample the function...

Image derivatives in two dimensions

Now that we are comfortable with the theory behind image derivatives in one dimension, we will see how the concept can be scaled to two dimensions. After all, when we start our OpenCV implementation from the next section onward, we will deal with images: two-dimensional data.

From what we have gathered in this chapter until now, derivatives essentially measure how our function changes. When we are dealing with one-dimensional data, there is only one possible direction along which this change can take place. When we talk about two-dimensional functions, we have two variables that we can vary independently, typically x and y. So, changing one of them (while keeping the other constant) allows us to measure the change in our function value (the thing we call derivative) along two directions as opposed to one. Therefore, it is prudent to assume that for two-dimensional functions (such as images), two different derivatives are computed. One derivative measures...

Visualizing image derivatives with OpenCV

In this section, our focus will be on the implementation of the concepts for image derivatives that we have covered in the last couple of sections. In terms of code, this section doesn't really present anything radically different from what we have worked on so far. In the last section, we presented two-dimensional image derivatives within the framework of image filters. All that remains to be done for calculating the derivative is that we implement the filters. And we already know from Chapter 2, Image Filtering that OpenCV's filter2D() function allows us to do exactly that. So without any further ado, let's delve into our code:

#include <iostream> 
#include <opencv2/core/core.hpp> 
#include <opencv2/highgui/highgui.hpp> 
#include <opencv2/imgproc/imgproc.hpp> 
 
using namespace std; 
using namespace cv; 

If you recall our discourse on the filter2D() function, one of the foremost things that...

The Sobel derivative filter

We have just implemented and applied our own derivative filter to an image. Since computing the derivative is quite a fundamental operation in computer vision and image processing, we typically do not want to burden the programmer with the task of having to populate the filter all by himself (as we just did). It is no surprise that OpenCV provides you with a function that can return the final output of the image derivative operation. Also, by varying the parameters of the function call, you can compute both the x and y derivatives. The function's name is Sobel(), named after Irwin Sobel, who came up with the design of this filter along with Gary Feldman in 1968.

What is so special about this filter? Well, the design is slightly different from what we implemented just now. For example, the x-derivative filter is shown as follows:

If you look carefully, you will notice a subtle difference in the Sobel filter. The middle row has been multiplied by two. What this...

From derivatives to edges

This chapter covers both image derivatives and edge detection. So far, we are done with the first half of the chapter, that is, image derivatives. The remainder of the chapter will be based on edge detection algorithms. Before we embark on an explanation of the various edge detection algorithms out there and the nuances of implementing them using OpenCV/C++, let's take a moment to get a feeling for how these two topics are related. This would not only help you appreciate why these two topics have been put together in the same chapter, but will also make the transition from derivatives to edge detection seamless.

For a moment, let's forget about computer vision or OpenCV and think as a layman. Now if I ask you, what do you understand by edges in images, what would your answer be? Well, to put it simply, we refer to the boundaries of objects as edges. Most of the natural images that one might expect to come across would consist of a finite number of objects (some of...

The Sobel detector - a basic framework for edge detection

In the last section, we touched upon the foundation on which the frameworks for edge detection are built on. Let's partake in a deeper analysis of the same here.

The very first step in a common edge detection framework that you might come across is the computation of derivatives. The derivatives along both the x and the y direction are computed separately and stored in, say, two different matrices. So, for every pixel (x, y) in the input image, we essentially have the gradient in the x and the y direction:  and  (these gradient values will be stored in the locations that correspond to pixel (x, y) in both the matrices). From these two values of gradients, we compute what is known as the gradient magnitude at (x, y). The gradient magnitude is given by the following formula:

If you recall, this is the same as the magnitude of a two-dimensional vector that has components  and . In fact, derivatives (or gradients) in multiple dimensions...

The Canny edge detector

Having learned about and also implemented the Sobel edge detector in the previous section, we now turn our attention to yet another edge detection algorithm, namely the Canny edge detector. It's named after its inventor John F. Canny who came up with the algorithm in 1986. The algorithm is much more involved than the Sobel edge detector and is considered to be superior to the latter.

The basic guiding principles powering the Canny detector remain the same. This means that we will still use the gradient values as indicators of whether the pixel belongs to a potential edge region or not. However, there are certain additional steps that are performed by the detector to improve the quality of detected edges. We provide a brief explanation of the same. Note that we won't be getting into the intricate mathematical details behind Canny. Rather, we would only be sharing the intuitions that motivate the additional steps that the algorithm performs. In addition to improving...

Image noise and edge detection

We are almost at the end of our chapter on edge detection. Before we close off the topic, I want to stress a practical aspect concerning edge detection algorithms. While talking about image filtering in Chapter 2, Image Filtering we discussed image noise. We said that noise in an image is not that uncommon and can occur due to a variety of factors. In this section, we're going to look at image noise with reference to edge detection.

So far, we have learned that edge detection algorithms rely on detecting abrupt changes in pixel intensity values. Now try to think of the effect that noise has on the pixel values in an image. It can do two things:

1. It can change the intensity of the group of pixels in an otherwise uniform area of the original image in such a manner that it becomes considerably different from its surroundings, thereby getting classified as an edge.

2. It can alter the intensity values of the pixels that actually belong to edges or regions near the edges...

Laplacian - yet another edge detection technique

In this section, we are going to look at yet another technique that is available for edge detection: Laplacian. Simply put, Laplacian is nothing but the second derivative of the image, where the second derivative refers to the derivative of the derivative. Mathematically, this is represented as follows:

When we were looking for edges using the first derivative, we saw that regions that are potentially edge regions have a sufficiently high magnitude of the derivative (gradient). As it turns out, in the same edge regions, the second derivative is zero. This phenomenon is used as a criterion to detect edges using the Laplacian operator.

The Laplacian() function in OpenCV implements the Laplacian operator that we just discussed. In fact, a single call to Laplacian() will handle both the dimensions, x and y. Internally, it calls the Sobel() function to calculate gradients. A code snippet showing the implementation of Laplacian() is as follows:

...

Blur detection using OpenCV

Let's take a look at one of the applications of the Laplacian operator: detecting the amount of blur in images. Often, the pictures that we take in our day-to-day lives using digital cameras, DSLRs, and so on. turn out to be not that clear, sharp, and well-focused. This can arise due to a variety of factors ranging from the motion of the subject that is being captured to the sudden movement of the capturing device just before the picture was taken. The problem that we are going to solve is that given an image, can you detect whether it is blurry or not?

The approach that we are going to take here is to use the Laplacian operator to quantify the amount of blur that is present in the image. As we'll soon see, the higher the value of our metric, the less blurry our image would be.

Now, how do we arrive at such a metric? As it turns out, it has been proven (through peer-reviewed research that we are not going to get into) that the variance of Laplacian gives a sufficiently...

Summary

This brings us to the end of our discourse on image derivatives and edge detection. We started off by discussing the concept of the derivatives of functions. Similar to some other mathematical concepts that we have covered (Gaussian functions), we saw that discrete approximation of the continuous derivatives can be applied to images. Image derivatives were a precursor to edge detection frameworks. We introduced a couple of different frameworks, namely Sobel and Canny. Toward the end of the chapter, we saw yet another technique that helps detect edge-like regions in images: the Laplacian (or the second derivative) operator. Apart from edge detection, Laplacian lends its utility to other related, practical use cases, such as quantifying the amount of blur in images.

As we progress through the book, you would notice a clear shift in our focus towards discussing processes that identify themselves as being core computer vision algorithms. You will realize, and perhaps you have started...