OpenCV: Image Processing using Morphological Filters

Exclusive offer: get 50% off this eBook here
OpenCV 2 Computer Vision Application Programming Cookbook

OpenCV 2 Computer Vision Application Programming Cookbook — Save 50%

Over 50 recipes to master this library of programming functions for real-time computer vision

£16.99    £8.50
by Robert Laganière | May 2011 | Cookbooks Open Source

OpenCV (Open Source Computer Vision) is an open source library containing more than 500 optimized algorithms for image and video analysis. Since its introduction in 1999, it has been largely adopted as the primary development tool by the community of researchers and developers in computer vision. OpenCV was originally developed at Intel by a team led by Gary Bradski as an initiative to advance research in vision and promote the development of rich, vision-based CPU-intensive applications.

In this article by Robert Laganière, author of OpenCV 2 Computer Vision Application Programming Cookbook, we will cover:

  • Eroding and dilating images using morphological filters
  • Opening and closing images using morphological filters
  • Detecting edges and corners using morphological filters

 

OpenCV 2 Computer Vision Application Programming Cookbook

OpenCV 2 Computer Vision Application Programming Cookbook

Over 50 recipes to master this library of programming functions for real-time computer vision

        Read more about this book      

(For more resources related to the article, see here.)

Introduction

Morphological filtering is a theory developed in the 1960s for the analysis and processing of discrete images. It defines a series of operators which transform an image by probing it with a predefined shape element. The way this shape element intersects the neighborhood of a pixel determines the result of the operation. This article presents the most important morphological operators. It also explores the problem of image segmentation using algorithms working on the image morphology.

Eroding and dilating images using morphological filters

Erosion and dilation are the most fundamental morphological operators. Therefore, we will present them in this first recipe.

The fundamental instrument in mathematical morphology is the structuring element. A structuring element is simply defined as a configuration of pixels (a shape) on which an origin is defined (also called anchor point). Applying a morphological filter consists of probing each pixel of the image using this structuring element. When the origin of the structuring element is aligned with a given pixel, its intersection with the image defines a set of pixels on which a particular morphological operation is applied. In principle, the structuring element can be of any shape, but most often, a simple shape such as a square, circle, or diamond with the origin at the center is used (mainly for efficiency reasons).

Getting ready

As morphological filters usually work on binary images, we will use a binary image produced through thresholding. However, since in morphology, the convention is to have foreground objects represented by high (white) pixel values and background by low (black) pixel values, we have negated the image.

How to do it...

Erosion and dilation are implemented in OpenCV as simple functions which are cv::erode and cv::dilate. Their use is straightforward:

// Read input image
cv::Mat image= cv::imread("binary.bmp");
// Erode the image
cv::Mat eroded; // the destination image
cv::erode(image,eroded,cv::Mat());
// Display the eroded image
cv::namedWindow("Eroded Image");");
cv::imshow("Eroded Image",eroded);
// Dilate the image
cv::Mat dilated; // the destination image
cv::dilate(image,dilated,cv::Mat());
// Display the dilated image
cv::namedWindow("Dilated Image");
cv::imshow("Dilated Image",dilated);

The two images produced by these function calls are seen in the following screenshot. Erosion is shown first:

Followed by the dilation result:

How it works...

As with all other morphological filters, the two filters of this recipe operate on the set of pixels (or neighborhood) around each pixel, as defined by the structuring element. Recall that when applied to a given pixel, the anchor point of the structuring element is aligned with this pixel location, and all pixels intersecting the structuring element are included in the current set. Erosion replaces the current pixel with the minimum pixel value found in the defined pixel set. Dilation is the complementary operator, and it replaces the current pixel with the maximum pixel value found in the defined pixel set. Since the input binary image contains only black (0) and white (255) pixels, each pixel is replaced by either a white or black pixel.

A good way to picture the effect of these two operators is to think in terms of background (black) and foreground (white) objects. With erosion, if the structuring element when placed at a given pixel location touches the background (that is, one of the pixels in the intersecting set is black), then this pixel will be sent to background. While in the case of dilation, if the structuring element on a background pixel touches a foreground object, then this pixel will be assigned a white value. This explains why in the eroded image, the size of the objects has been reduced. Observe how some of the very small objects (that can be considered as "noisy" background pixels) have also been completely eliminated. Similarly, the dilated objects are now larger and some of the "holes" inside of them have been filled.

By default, OpenCV uses a 3x3 square structuring element. This default structuring element is obtained when an empty matrix (that is cv::Mat()) is specified as the third argument in the function call, as it was done in the preceding example. You can also specify a structuring element of the size (and shape) you want by providing a matrix in which the non-zero element defines the structuring element. In the following example, a 7x7 structuring element is applied:

cv::Mat element(7,7,CV_8U,cv::Scalar(1));
cv::erode(image,eroded,element);

The effect is obviously much more destructive in this case as seen here:

Another way to obtain the same result is to repetitively apply the same structuring element on an image. The two functions have an optional parameter to specify the number of repetitions:

// Erode the image 3 times.
cv::erode(image,eroded,cv::Mat(),cv::Point(-1,-1),3);

The origin argument cv::Point(-1,-1) means that the origin is at the center of the matrix (default), and it can be defined anywhere on the structuring element. The image obtained will be identical to the one we obtained with the 7x7 structuring element. Indeed, eroding an image twice is like eroding an image with a structuring element dilated with itself. This also applies to dilation.

Finally, since the notion of background/foreground is arbitrary, we can make the following observation (which is a fundamental property of the erosion/dilation operators). Eroding foreground objects with a structuring element can be seen as a dilation of the background part of the image. Or more formally:

  • The erosion of an image is equivalent to the complement of the dilation of the complement image.
  • The dilation of an image is equivalent to the complement of the erosion of the complement image.

There's more...

It is important to note that even if we applied our morphological filters on binary images here, these can also be applied on gray-level images with the same definitions.

Also note that the OpenCV morphological functions support in-place processing. This means you can use the input image as the destination image. So you can write:

cv::erode(image,image,cv::Mat());

OpenCV creates the required temporary image for you for this to work properly.

OpenCV 2 Computer Vision Application Programming Cookbook Over 50 recipes to master this library of programming functions for real-time computer vision
Published: May 2011
eBook Price: £16.99
Book Price: £27.99
See more
Select your format and quantity:
        Read more about this book      

(For more resources related to the article, see here.)

Opening and closing images using morphological filters

The previous recipe introduced the two fundamental morphological operators: dilation and erosion. From these, other operators can be defined. The next two recipes will present some of them. The opening and closing operators are presented in this recipe.

How to do it...

In order to apply higher-level morphological filters, you need to use the cv::morphologyEx function with the appropriate function code. For example, the following call will apply the closing operator:

cv::Mat element5(5,5,CV_8U,cv::Scalar(1));
cv::Mat closed;
cv::morphologyEx(image,closed,cv::MORPH_CLOSE,element5)

Note that here we use a 5x5 structuring element to make the effect of the filter more apparent. If we input the binary image of the preceding recipe, we obtain:

Similarly, applying the morphological opening operator will result in the following image:

This one being obtained from the following code:

cv::Mat opened;
cv::morphologyEx(image,opened,cv::MORPH_OPEN,element5);

How it works...

The opening and closing filters are simply defined in terms of the basic erosion and dilation operations:

  • Closing is defined as the erosion of the dilation of an image.
  • Opening is defined as the dilation of the erosion of an image.

Consequently, one could compute the closing of an image using the following calls:

// dilate original image
cv::dilate(image,result,cv::Mat());
// in-place erosion of the dilated image
cv::erode(result,result,cv::Mat());

The opening would be obtained by inverting these two function calls.

While examining the result of the closing filter, it can be seen that the small holes of the white foreground objects have been filled. The filter also connects together several of the adjacent objects. Basically, any holes or gaps too small to completely contain the structuring element will be eliminated by the filter.

Reciprocally, the opening filter eliminated several of the small objects in the scene. All of the ones that were too small to contain the structuring element have been removed.

These filters are often used in object detection. The closing filter connects together objects erroneously fragmented into smaller pieces, while the opening filter removes the small blobs introduced by image noise. Therefore, it is advantageous to use them in sequence. If our test binary image is successively closed and opened, we obtain an image showing only the main objects in the scene, as shown below. You can also apply the opening filter before closing if you wish to prioritize noise filtering, but this can be at the price of eliminating some fragmented objects.

It should be noted that applying the same opening (and similarly the closing) operator on an image several times has no effect. Indeed, with the holes having been filled by the first opening, an additional application of this same filter will not produce any other changes to the image. In mathematical terms, these operators are said to be idempotent.

Detecting edges and corners using morphological filters

Morphological filters can also be used to detect specific features in an image. In this recipe, we will learn how to detect lines and corners in a gray-level image.

Getting started

In this recipe, the following image will be used:

How to do it...

Let's define a class named MorphoFeatures which will allow us to detect image features:

class MorphoFeatures {
private:
// threshold to produce binary image
int threshold;
// structuring elements used in corner detection
cv::Mat cross;
cv::Mat diamond;
cv::Mat square;
cv::Mat x;

Detecting lines is quite easy using the appropriate filter of the cv::morphologyEx function:

cv::Mat getEdges(const cv::Mat &image) {
// Get the gradient image
cv::Mat result;
cv::morphologyEx(image,result,
cv::MORPH_GRADIENT,cv::Mat());
// Apply threshold to obtain a binary image
applyThreshold(result);
return result;
}

The binary edge image is obtained through a simple private method of the class:

void applyThreshold(cv::Mat& result) {
// Apply threshold on result
if (threshold>0)
cv::threshold(result, result,
threshold, 255, cv::THRESH_BINARY);
}

Using this class in a main function, you then obtain the edge image as follows:

// Create the morphological features instance
MorphoFeatures morpho;
morpho.setThreshold(40);
// Get the edges
cv::Mat edges;
edges= morpho.getEdges(image);

The result is the following image:

The detection of corners using morphological corners is a bit more complex since it is not directly implemented in OpenCV. This is a good example of the use of non-square structuring elements. Indeed, it requires the definition of four different structuring elements shaped as a square, diamond, cross, and an X-shape. This is done in the constructor (all of these structuring elements having a fixed 5x5 dimension for simplicity):

MorphoFeatures() : threshold(-1),
cross(5,5,CV_8U,cv::Scalar(0)),
diamond(5,5,CV_8U,cv::Scalar(1)),
square(5,5,CV_8U,cv::Scalar(1)),
x(5,5,CV_8U,cv::Scalar(0)){

// Creating the cross-shaped structuring element
for (int i=0; i<5; i++) {
cross.at<uchar>(2,i)= 1;
cross.at<uchar>(i,2)= 1;
}
// Creating the diamond-shaped structuring element
diamond.at<uchar>(0,0)= 0;
diamond.at<uchar>(0,1)= 0;
diamond.at<uchar>(1,0)= 0;
diamond.at<uchar>(4,4)= 0;
diamond.at<uchar>(3,4)= 0;
diamond.at<uchar>(4,3)= 0;
diamond.at<uchar>(4,0)= 0;
diamond.at<uchar>(4,1)= 0;
diamond.at<uchar>(3,0)= 0;
diamond.at<uchar>(0,4)= 0;
diamond.at<uchar>(0,3)= 0;
diamond.at<uchar>(1,4)= 0;
// Creating the x-shaped structuring element
for (int i=0; i<5; i++) {
x.at<uchar>(i,i)= 1;
x.at<uchar>(4-i,i)= 1;
}
}

In the detection of corner features, all of these structuring elements are applied in cascade to obtain the resulting corner map:

cv::Mat getCorners(const cv::Mat &image) {
cv::Mat result;

// Dilate with a cross
cv::dilate(image,result,cross);

// Erode with a diamond
cv::erode(result,result,diamond);

cv::Mat result2;
// Dilate with a X
cv::dilate(image,result2,x);

// Erode with a square
cv::erode(result2,result2,square);

// Corners are obtained by differencing
// the two closed images
cv::absdiff(result2,result,result);

// Apply threshold to obtain a binary image
applyThreshold(result);

return result;
}

In order to better visualize the result of the detection, the following method draws a circle on the image at each detected point on the binary map:

void drawOnImage(const cv::Mat& binary,
cv::Mat& image) {
cv::Mat_<uchar>::const_iterator it=
binary.begin<uchar>();
cv::Mat_<uchar>::const_iterator itend=
binary.end<uchar>();
// for each pixel
for (int i=0; it!= itend; ++it,++i) {
if (!*it)
cv::circle(image,
cv::Point(i%image.step,i/image.step),
5,cv::Scalar(255,0,0));
}
}

Corners are then detected on an image by using the following code:

// Get the corners
cv::Mat corners;
corners= morpho.getCorners(image);
// Display the corner on the image
morpho.drawOnImage(corners,image);
cv::namedWindow("Corners on Image");
cv::imshow("Corners on Image",image);

The image of detected corners is then, as follows.

How it works...

A good way to help understand the effect of morphological operators on a gray-level image is to consider an image as a topological relief in which gray-levels correspond to elevation (or altitude). Under this perspective, bright regions correspond to mountains, while the darker areas form the valleys of the terrain. Also, since edges correspond to a rapid transition between darker and brighter pixels, these can be pictured as abrupt cliffs. If an erosion operator is applied on such a terrain, the net result will be to replace each pixel by the lowest value in a certain neighborhood, thus reducing its height. As a result, cliffs will be "eroded" as the valleys expand. Dilation has the exact opposite effect, that is, cliffs will gain terrain over the valleys. However, in both cases, the plateaux (that is, area of constant intensity) will remain relatively unchanged.

The above observations lead to a simple way of detecting the edges (or cliffs) of an image. This could be done by computing the difference between the dilated image and the eroded image. Since these two transformed images differ mostly at the edge locations, the image edges will be emphasized by the differentiation. This is exactly what the cv::morphologyEx function is doing when the cv::MORPH_GRADIENT argument is inputted. Obviously, the larger the structuring element is, the thicker the detected edges will be. This edge detection operator is also called the Beucher gradient. Note that similar results could also be obtained by simply subtracting the original image from the dilated one, or the eroded image from the original. The resulting edges would simply be thinner.

Corner detection is a bit more complex since it uses four different structuring elements. This operator is not implemented in OpenCV but we present it here to demonstrate how structuring elements of various shapes can be defined and combined. The idea is to close the image by dilating and eroding it with two different structuring elements. These elements are chosen such that they leave straight edges unchanged, but because of their respective effect, edges at corner points will be affected. Let's use the simple following image made of a single white square to better understand the effect of this asymmetrical closing operation:

The first square is the original image. When dilated with a cross-shaped structuring element, the square edges are expanded, except at the corner points where the cross shape does not hit the square. This is the result illustrated by the middle square. This dilated image is then eroded by a structuring element that, this time, has a diamond shape. This erosion brings back most edges at their original position, but pushes the corners even further since they were not dilated. The left square is then obtained, which, as it can be seen, has lost its sharp corners. The same procedure is repeated with an X-shaped and a square-shaped structuring element. These two elements are the rotated version of the previous ones and will consequently capture the corners at a 45-degree orientation. Finally, differencing the two results will extract the corner features.

Summary

This article explored the concept of mathematical morphology. We took a look at eroding and dilating images, opening and closing images, and detecting edges and corners using morphological filters.


Further resources related to this subject:


OpenCV 2 Computer Vision Application Programming Cookbook Over 50 recipes to master this library of programming functions for real-time computer vision
Published: May 2011
eBook Price: £16.99
Book Price: £27.99
See more
Select your format and quantity:

About the Author :


Robert Laganière

Robert Laganière is a professor at the University of Ottawa, Canada. He received his Ph.D. degree from INRS-Telecommunications in Montreal in 1996. Dr. Laganière is a researcher in computer vision with an interest in video analysis, intelligent visual surveillance, and imagebased modeling. He is a co-founding member of the VIVA research lab. He is also a Chief Scientist at iWatchLife.com, a company offering a cloud-based solution for remote monitoring. Dr. Laganière is the co-author of Object-oriented Software Engineering published by McGraw
Hill in 2001.

Visit the author's website at http://www.laganiere.name.

Books From Packt


Blender 2.5 HOTSHOT
Blender 2.5 HOTSHOT

GIMP 2.6 cookbook
GIMP 2.6 cookbook

OpenSceneGraph 3.0: Beginner's Guide
OpenSceneGraph 3.0: Beginner's Guide

Away3D 3.6 Cookbook
Away3D 3.6 Cookbook

Cocos2d for iPhone 0.99 Beginner's Guide
Cocos2d for iPhone 0.99 Beginner's Guide

Panda3D 1.7 Game Developer's Cookbook
Panda3D 1.7 Game Developer's Cookbook

Python 2.6 Graphics Cookbook
Python 2.6 Graphics Cookbook

OpenGL 4.0 Shading Language Cookbook: RAW
OpenGL 4.0 Shading Language Cookbook: RAW


Code Download and Errata
Packt Anytime, Anywhere
Register Books
Print Upgrades
eBook Downloads
Video Support
Contact Us
Awards Voting Nominations Previous Winners
Judges Open Source CMS Hall Of Fame CMS Most Promising Open Source Project Open Source E-Commerce Applications Open Source JavaScript Library Open Source Graphics Software
Resources
Open Source CMS Hall Of Fame CMS Most Promising Open Source Project Open Source E-Commerce Applications Open Source JavaScript Library Open Source Graphics Software