OpenCV 2 Computer Vision Application Programming Cookbook

5 (3 reviews total)
By Robert Laganiere
  • Instant online access to over 8,000+ books and videos
  • Constantly updated with 100+ new titles each month
  • Breadth and depth in over 1,000+ technologies
  1. Playing with Images

About this book

In today's digital world, images are everywhere, and with the advent of powerful and affordable computing devices, it has become possible to create sophisticated applications manipulating images and videos. Adding special effects, enhancing image features, performing object recognition, and reconstructing 3D information are tasks that can be programmed easily with the OpenCV library, which is a widely used open source library that offers a rich set of advanced computer vision algorithms.

OpenCV 2 Computer Vision Application Programming Cookbook will introduce you to numerous computer vision algorithms included in the OpenCV library. You will learn how to read, write, create and manipulate images. You will explore different techniques commonly used in image analysis and how they can be effectively implemented in C++. The book provides a complete introduction to the OpenCV library and explains how to build your first computer vision program. You will be presented with a variety of computer vision algorithms and be exposed to important concepts in image analysis that will enable you to build your own computer vision applications.

The book helps you to get started with the library, showing you how to install and deploy the OpenCV library to write effective computer vision applications following good programming practices. The techniques to process an image and its pixels using the data structures offered by the library are explained in detail. You will learn how to build and manipulate an image histogram; how to detect lines and contours. You will be introduced to the concept of mathematical morphology and image filtering. The detection and use of interest points in computer vision is presented with applications for image matching and object recognition. Techniques to achieve camera calibration and 3D reconstruction are presented.

OpenCV 2 Computer Vision Application Programming Cookbook is your guide to the development of computer vision applications. It is a comprehensive reference that exposes you to computer vision concepts illustrated with extensive examples.

Publication date:
May 2011
Publisher
Packt
Pages
304
ISBN
9781849513241

 

Chapter 1. Playing with Images

In this chapter, we will cover:

  • Installing the OpenCV Library

  • Creating an OpenCV project with MS Visual C++

  • Creating an OpenCV project with Qt

  • Loading, displaying, and saving images

  • Creating a GUI application using Qt

 

Introduction


This chapter will teach you the basic elements of OpenCV and will show you how to accomplish the most fundamental tasks: reading, displaying, and saving images. Before you can start with OpenCV, you need to install the library. This is a simple process that is explained in the first recipe of this chapter.

You also need a good development environment (IDE) to run your OpenCV applications. We propose two alternatives here. The first is to use the well-known Microsoft Visual Studio platform. The second option is to use an open source tool for C++ project development called Qt. Two recipes will show you how to set up a project with these two tools, but you can also use other C++ IDEs. In fact, in this cookbook, the tasks will be presented in a way that is independent of any particular environment and operating system, so you are free to use the one of your choice. However, be aware that you need to use the compiled version of the OpenCV library that is appropriate to the compiler and operating system you are using. If you obtain strange behaviors, or if your application crashes without apparent reasons, that could be a symptom of incompatibilities.

 

Installing the OpenCV library


OpenCV is an open source library for developing computer vision applications. It can be used in both academic and commercial applications under a BSD license that allows you to freely use, distribute, and adapt it. This recipe will show you how to install the library on your machine.

Getting ready

When you visit the OpenCV official website at http://opencv.willowgarage.com/wiki/, you will find the latest release of the library, the online documentation, and many other useful resources about OpenCV.

How to do it...

From the OpenCV website, go to the Download page that corresponds to the platform of your choice (Linux/Unix/Mac or Windows). From there you will be able to download the OpenCV package. You will then uncompress it, normally under a directory with a name corresponding to the library version (for example, OpenCV2.2). Once this is done, you will find a collection of directories, notably the doc directory containing the OpenCV documentation, the include directory containing all of the include files, the modules directory which contains all of the source files (yes, it is open source), and the samples directory containing many small examples to help you to get started.

If you are working under Windows with Visual Studio, you also have the option to download the executable installation package corresponding to your IDE and Windows platform. Executing this setup program will not only install the source library, but also all of the precompiled binaries you will need to build your applications. In that case, you are ready to start using OpenCV. If not, you need to take few additional steps.

In order to use OpenCV under the environment of your choice, you need to generate the library binary files using the appropriate C++ compiler. To build OpenCV, you need to use the CMake tool available at http://www.cmake.org/. CMake is another open source software tool designed to control the compilation process of a software system using platform-independent configuration files. You therefore need to download and install CMake. You can then run it using the command line, but it is easier to use CMake with its Graphical User Interface (GUI). In this latter case, all you need to do is to specify the folder containing the OpenCV library and the one that will contain the binaries. You then click on Configure in order to select the compiler of your choice (here we chose Visual Studio 2010), and you click on Configure again, as seen in the following screenshot:

You are now ready to generate your makefiles and workspace files by clicking on the Generate button. These files will allow you to compile the library. This is the last step of the installation process.

Compiling the library will make it ready to use for your development environment. If you selected an IDE like Visual Studio, then all you need to do is to open the top-level solution file that CMake has created for you. You then issue the Build Solution command. In Unix environments, you will use the generated makefiles by running your make utility command.

If everything went well, you should now have your compiled and ready-to-use OpenCV library in the specified directory. This directory will contain, in addition to the directories we already mentioned, a bin directory containing the compiled library. You can move everything to your preferred location (for example, c:\OpenCV2.2) and add the bin directory to your system path (under Windows, this is done by opening your Control Panel. You start the System utility and under the Advanced tab, you will find the Environment Variables button).

How it works...

Since version 2.2, the OpenCV library is divided into several modules. These modules are built in library files located in the lib directory. They are:

  • The opencv_core module that contains the core functionalities of the library, in particular, the basic data structures and arithmetic functions.

  • The opencv_imgproc module that contains the main image processing functions.

  • The opencv_highgui module that contains the image and video reading and writing functions, along with other user interface functions.

  • The opencv_features2d module that contains the feature point detectors and descriptors and the feature point matching framework.

  • The opencv_calib3d module that contains the camera calibration, two-view geometry estimation, and stereo functions.

  • The opencv_video module that contains the motion estimation, feature tracking, and foreground extraction functions and classes.

  • The opencv_objdetect module containing the object detection functions such as the face and people detectors.

The library also includes other utility modules containing machine learning functions (opencv_ml), computational geometry algorithms (opencv_flann), contributed code (opencv_contrib), obsolete code (opencv_legacy), and gpu accelerated code (opencv_gpu).

All of these modules have a header file associated with them (located in include directory). Typical OpenCV C++ code will therefore start by including the required modules. For example (and this is the suggested declaration style):

#include <opencv2/core/core.hpp>
#include <opencv2/imgproc/imgproc.hpp>
#include <opencv2/highgui/highgui.hpp>

If you see OpenCV code starting with:

 #include "cv.h"

it is because it uses the old style, before the library was restructured into modules.

There's more...

You can also access the latest code being developed from the OpenCV SVN server located at:

https://code.ros.org/svn/opencv/trunk/opencv/

You will find that there are a large number of examples that can help you learn how to use the library and give you many development tips.

 

Creating an OpenCV project with MS Visual C++


Using MS Visual C++, you can easily create OpenCV applications for Windows. You can build simple console application or you can create more sophisticated applications with a nice graphical user interface (GUI). Since it is the easiest option, we will create a simple console application here. We'll use Visual Studio 2010, however, the same principles also apply to any other versions of the Microsoft IDE since the menus and options are very similar in the different versions.

When you run Visual Studio for the first time, you can set it up in a way such that C++ becomes your default development environment. This way, when you will launch the IDE, it will be in Visual C++ mode.

We assume that you have installed OpenCV under the C:\OpenCV2.2 directory as explained in the previous recipe.

Getting ready

When working with Visual Studio, it is important to understand the difference between a solution and a project. Basically, a solution is made of several projects (each project is a distinct software module, for example, a program and a library). This way the projects of your solution can share files and libraries. Usually, you create one master directory for your solution that contains all of your projects directories. But you can also group the solution and a project into one single directory. This is what you will most often do for a one-project solution. As you become more familiar with VC++, and build more complex applications, you should take advantage of the multi-project solution structure.

Also, when you compile and execute your Visual C++ projects, you can do it under two different configurations: Debug and Release. The Debug mode is there to help you create and debug your application. It is a more protected environment, for example, it will tell you if your application contains memory leaks or it will check at runtime if you are using certain functions properly. However, it generates slower executable files. This is why, once your application has been tested and is ready to be used, you build it under the Release mode. This will produce the executable that you will distribute to the users of your application. Note that it may happen that you have code running perfectly well in debug mode but has problems in release mode. You then need to do more testing in order to identify the potential sources of errors. Debug and Release modes are not unique to Visual C++, most IDEs also support these two modes of compilation.

How to do it...

We are now ready to create our first project. This is done by using the File|New Project | Project... menu option. You can create different project types here. Let's start with the simplest option which is to select a Win32 Console Application, seen in the following screenshot:

You need to specify where your want to create your project, and what name you want to give to your project. There is also an option for creating or not creating a directory for the solution (the bottom-right checkbox). If you check this option, an additional directory will be created (with the name you specify) that will contain your solution directory. If you simply leave this option unchecked, a solution file (extension .sln) will still be created, but this one will be contained within the same (single) project directory. Click on OK and then Next to go to the Application Settings window of the Win32 Application Wizard. As seen in the following screenshot, a number of options are offered there. We will simply create an empty project.

Note that we also have unchecked the Precompiled header option which is an MS Visual Studio-specific feature to make the compilation process faster. Since we want to stay within the ANSI C++ standard, we will not use this option. If you click on Finish, your project will be created. It is empty for now, but we will add a main file to it soon.

But first, to be able to compile and run your future OpenCV application, you need to tell Visual C++ where to find the OpenCV libraries and include files. Since you will probably create several OpenCV projects in the future, the best option is to create a Property Sheet that you will be able to reuse from project to project. This is done through the Property Manager. If it is not already visible in your current IDE, you can access it from the View menu.

In Visual C++ 2010, a property sheet is an XML file that describes your project settings. We will now create a new one by right-clicking on the Debug | Win32 node of the project, and by selecting the Add New Project Property Sheet option (as seen in the following screenshot):

The new property sheet is then added once we click on Add. We now need to edit it. Simply double-click on the property sheet's name and select VC++ Directories, as seen here:

Edit the Include Directories textfield and add the path to the include files of your OpenCV library:

Do the same thing with the Library Directories. This time you add the path to your OpenCV library files:

It is important to note that we used the explicit path to the OpenCV library in our property sheet. It is generally a better practice to use an environment variable to designate the library location. This way, if you switch to another version of the library, you simply change the definition of this variable so that it points to the library's new location. Also, in the case of a team project, the different users might have installed the library at different locations. Using an environment variable would avoid needing to edit the property sheet for each user. Consequently, if you define the environment variable OPENCV2_DIR to be c:\OpenCV2.2, then the two OpenCV directories will be specified as $(OPENCV_DIR)\include and $(OPENCV_DIR)\lib in the property sheet.

The next step is to specify the OpenCV library files which need to be linked with your code in order to produce an executable application. Depending on the application, you may need different OpenCV modules. Since we want to reuse this property sheet in all of our projects, we will simply add the library modules we need to run the applications of this book. Go to the Input item of the Linker node, as seen in the following screenshot:

Edit the Additional Dependencies textfield and add the following list of library modules:

Note that we specified the libraries with names ending with the letter "d". These are the binaries for the Debug mode. You will need to create another (almost identical) property sheet for the Release mode. You follow the same procedure, but you add it under the Release | Win32 node. This time, the library names are specified without appending a "d" at the end.

We are now ready to create, compile, and run our first application. We add a new source file by using the Solution Explorer, and right-clicking the Source Files node. You select Add New Item... which gives you the opportunity to specify main.cpp as the name of this C++ file:

You can also use the File|New|File... menu option to do this. Now, let's build a simple application that will display an image named img.jpg located under the default directory.

Once you have recopied the code in the preceding figure (it will be explained later), you can compile it and run it using the Start green arrow in the Toolbar at the top of your screen. You will see your image displayed for five seconds. An example is seen here:

If it is the case, then you have completed your first successful OpenCV application! If the program fails when executed, it is probably because it cannot find the image file. See the following section to find out how to put it in the correct directory.

How it works...

When you click on the Start Debugging button (or push F5), your project will be compiled and then executed. You can also just compile the project by selecting Build Solution (F7) under the Build menu. The first time you compile your project, a Debug directory will be created. This will contain the executable file (extension .exe). Similarly, you can also create the release version by simply selecting the Release configuration option using the drop-down menu, to the right of the green arrow button (or using the Configuration Manager... option under the Build menu). A Release directory will then be created.

When you execute a project using the Start button of Visual Studio, the default directory will always be the one that contains your solution file. However, if you choose to execute your application outside of your IDE (that is, from Windows Explorer) by double-clicking on your .exe file (normally the Release directory), then the default directory will become the one that contains the executable file. Therefore, make sure your image file is located in the appropriate directory before you execute this application.

See also

The Loading, displaying, and saving images recipe later in this chapter that explains the OpenCV source code we have used in this task.

 

Creating an OpenCV project with Qt


Qt is a complete Integrated Development Environment (IDE) for C++ application that was originally developed by Trolltech, a Norwegian software company which was acquired in 2008 by Nokia. It is offered under the LPGL open source license as well as under a commercial (and paying) license for the development of proprietary projects. It is composed of two separate elements: a cross-platform IDE, called Qt Creator, and a set of Qt class libraries and development tools. Using the Qt Software Development Kit (SDK) to develop C++ applications has many benefits:

  • It is an open source initiative, developed by the Qt community, that gives you access to the source code of the different Qt components.

  • It is cross-platform, meaning that you can develop applications that can run on different operating systems such as Windows, Linux, Mac OS X, and so on.

  • It includes a complete and cross-platform GUI library that follows an effective object-oriented and event-driven model.

  • Qt also includes several cross-platform libraries to develop multimedia, graphics, database, multithreading, web application, and many other interesting building blocks useful for designing advanced applications.

Getting ready

Qt can be downloaded from http://qt.nokia.com. It is free if you select the LPGL license. You should download the complete SDK. However, make sure to select the Qt libraries package that is appropriate for your platform. Obviously, since we are dealing with open source software, it is always possible to re-compile the library under the platform of your choice.

Here, we use Qt Creator version 1.2.1 with Qt version 4.6.3. Note that under the Projects tab of Qt Creator, it is possible to manage the different Qt versions that you might have installed. This ensures you can always compile your projects with the appropriate Qt version.

How to do it...

When you start Qt, it will ask you if you wish to create a new project or if you want to open a recent one. You can also create a new project by going under the File menu and selecting the New... option. To replicate what we did in the previous recipe, we will select the Qt4 Console Application as seen in the following screenshot:

You then specify a name and a project location as seen here:

The following screen will ask you to select the modules you want to include in your project. Just keep the one selected by default and click on Next, and then Finish. An empty console application is then created as seen here:

The code generated by Qt creates a QCoreApplication object and calls its exec() method. This is only required when your application needs an event handler to process the user interactions with a GUI. In our simple open and display image example, this is not needed. We can simply replace the generated code by the one we use in the previous task. The simple open and display image program would then read as follows:

In order to be able to compile this program, the OpenCV library files and header files location need to be specified. With Qt, this information is given in a project file (with extension .pro) which is a simple text file describing the project parameters. You can edit this project file in Qt Creator by selecting the corresponding project file as seen in the following screenshot:

The information required to build an OpenCV application is provided by appending the following lines at the end of the project file:

INCLUDEPATH += C:\OpenCV2.2\include\

LIBS += -LC:\OpenCV2.2\lib \
-lopencv_core220 \
-lopencv_highgui220 \
-lopencv_imgproc220 \
-lopencv_features2d220 \
-lopencv_calib3d220

Tip

Downloading the example code

You can download the example code files for all Packt books you have purchased from your account at http://www.PacktPub.com. If you purchased this book elsewhere, you can visit http://www.PacktPub.com/support and register to have the files e-mailed directly to you.

The program is now ready to be compiled and executed. This is accomplished by clicking the bottom-left green arrow (or by pressing Ctrl+R). There is also a Debug and a Release mode that you set up using the Build Settings of the Projects tab.

How it works...

A Qt project is described by a project file. This is a text file that declares a list of variables containing the relevant information required to build the project. This file is in fact processed by a software tool called qmake which Qt invokes when a compilation is requested. Each variable defined in a project file is associated with a list of values. The main variables that are recognized by qmake in Qt are as follows:

  • TEMPLATE: Defines the type of project (applications, library, and so on).

  • CONFIG : Specifies different options that the compiler should use when building the project.

  • HEADERS : Lists the header files of the project.

  • SOURCES : Lists the source files (.cpp) of the project.

  • QT : Declares the required Qt extension modules and libraries. By default, the core and the GUI modules are included. If you want to exclude one of them, you use the -= notation.

  • INCLUDEPATH : Specifies the header file directories that should be searched.

  • LIBS : Contains the list of library files that should be linked with the project. You use the flag –L for directory paths and the flag –l for library names.

Several other variables are defined but the ones listed here are most commonly used.

There's more...

Many additional features can be used in qmake project files. For example, scopes can be defined to add declarations that apply to a specific platform:

win32 {
   # declarations for Windows 32 platforms only
}
unix {
   # declarations for Unix 32 platforms only
}

You can also use the pkg-config utility package. It is an open source tool that helps to use the correct compiler options and library files. When you install OpenCV with CMake, the unix-install contains an opencv.pc file that is read by pkg-config to determine the compilation parameters. A multi-platform qmake project file could then look as follows:

unix { 

    CONFIG += link_pkgconfig
    PKGCONFIG += opencv
}

Win32 {
   
INCLUDEPATH += C:\OpenCV2.2\include\

LIBS += -LC:\OpenCV2.2\lib \
-lopencv_core220 \
   -lopencv_highgui220 \
   -lopencv_imgproc220 \
   -lopencv_features2d220 \
   -lopencv_calib3d220
}

See also

The next recipe, Loading, displaying, and saving images explains the OpenCV source code we used in this task.

Consult the website http://qt.nokia.com for the complete documentation about Qt, Qt Creator, and all of the Qt extension modules.

 

Loading, displaying, and saving images


The two preceding recipes taught you how to create a simple OpenCV project but we have not explained the OpenCV code that was used. This task will show you how to perform the most fundamental operations needed in the development of an OpenCV application. These include loading an input image from file, displaying an image on a window, and storing an output image on disk.

Getting ready

Using either MS Visual Studio or Qt, create a new console application with a main function ready to be filled. See the first two recipes on how to proceed.

How to do it...

The first thing to do is to declare a variable that will hold the image. Under OpenCV 2, you define an object of class cv::Mat.

cv::Mat image;

This definition creates an image of size 0 by 0. This can be confirmed by calling the cv::Mat method size() that allows you to read the current size of this image. It returns a structure containing the height and width of the image:

std::cout << "size: " << image.size().height << " , " 
          << image.size().width << std::endl;

Next a simple call to the reading function will read an image from file, decode it, and allocate the memory:

image=  cv::imread("img.jpg");

You are now ready to use this image. However, you should first check if the image has been correctly read (an error will occur if the file is not found or if the file is corrupted, or is not a recognizable format). The validity of the image is tested by:

if (!image.data) { 
   // no image has been created... 
}

The member variable data is in fact a pointer to the allocated memory block that will contain the image data. It is simply set to 0 when no image has been read. The first thing you might want to do with this image is to display it. You do it using the highgui module provided by OpenCV. You start by declaring the window on which you want to display images, and then you specify the image to be shown on this special window:

cv::namedWindow("Original Image"); // define the window
cv::imshow("Original Image", image); // show the image

Now, you would normally apply some processing to the image. OpenCV offers a wide selection of processing functions, and several of them are explored in this book. Let's start with a very simple one that will simply flip the image horizontally. Several image transformations in OpenCV can be performed in-place, meaning that the transformation is applied directly on the input image (no new image being created). This is the case of the flipping method. However, we can always create another matrix to hold the output result and that is what we will do:

cv::Mat result;
cv::flip(image,result,1); // positive for horizontal
                          // 0 for vertical,                     
                          // negative for both

And the result is displayed on another window:

cv::namedWindow("Output Image");
cv::imshow("Output Image", result);

Since it is a console window that will terminate at the end of the main function, we add an extra highgui method to wait for a user key before ending the program:

cv::waitKey(0);

You can then see both the input and output images displayed on two distinct windows. Finally, you will probably want to save the processed image on your disk. This is done using the following highgui function:

cv::imwrite("output.bmp", result);

The file extension determines which codec will be used to save the image.

How it works...

All classes and functions defined in the C++ API of OpenCV are defined within the name space cv. You have two options to access them. First, precede the main function definition by the following declaration:

using namespace cv;

Alternatively, prefix all OpenCV class and function names by the namespace specification that is cv::, as we did in this recipe.

The class cv::Mat is the data structure used to hold your images (and obviously other matrix data). By default, they have a zero size but you can also specify an initial size:

cv::Mat ima(240,320,CV_8U,cv::Scalar(100));

In this case, you also need to specify the type of each matrix element, here CV_8U which corresponds to 1-byte pixel images. Letter U means unsigned. You can also declare signed numbers by using the letter S. For a color image, you would specify three channels (CV_8UC3). You can also declare integers (signed or unsigned) of size 16 and 32 (for example, CV_16SC3). You also have access to 32-bit and 64-bit floating point numbers (for example, CV_32F).

When the cv::Mat object goes out of scope, the memory allocated is automatically released. This is very convenient because you avoid having problems with memory leaks. Moreover, the cv::Mat class implements reference counting and shallow copy such that when an image is assigned to another one, the image data (that is the pixels) is not copied, and both images will point to the same memory block. This also applies to images passed by value or returned by value. A reference count is kept such that the memory will be released only when all of the references to the image will be destructed. If you wish to create an image that will contain a new copy of the original image, you will use the method copyTo(). You can test this behavior by declaring a few extra images in the example of this project, as follows:

cv::Mat image2, image3;
image2= result; // the two images refer to the same data
result.copyTo(image3); // a new copy is created

Now if you again flip the output image and display the two additional images, you will see that image2 is also affected by the transformation (because it points to the same image data than result image) while image3 remains unchanged as it holds a copy of the image. This allocation model for cv::Mat objects also means that you can safely write functions (or class methods) that return an image:

cv::Mat function() {

   // create image
   cv::Mat ima(240,320,CV_8U,cv::Scalar(100));
   // return it
   return ima;
}

If we call this function from our main function:

   // get a gray-level image
   cv::Mat gray= function();

The gray variable will now hold the image created by the function without extra memory allocation. Indeed, only a shallow copy of the image will be transferred from the returned cv::Mat instance to the gray image. When the ima local variable goes out of scope, this variable is de-allocated, but since the associated reference counter indicates that its internal image data is being referred by another instance (that is the gray variable) its memory block is not released.

However, in the case of classes, you should be careful and not return image class attributes. Here is an example of an error-prone implementation:

class Test {

   // image attribute
   cv::Mat ima;

  public:

     // constructor creating a gray-level image
     Test() : ima(240,320,CV_8U,cv::Scalar(100)) {}

     // method return a class attribute, not a good idea...
     cv::Mat method() { return ima; }
};

Here, if a function calls the method of this class, it obtains a shallow copy of the image attributes. If later, this copy is modified, the class attribute will also be modified which can affect the subsequent behavior of the class (and vice versa). To avoid these kinds of errors, you should instead return a copy of the attribute.

There's more...

With version 2 of the OpenCV, a new C++ interface has been introduced. Previously, C-like functions and structures were used (and can still be used). In particular, images were manipulated using the IplImage structure. This structure was inherited from the IPL library (that is the Intel Image Processing Library) now integrated with the IPP library (the Intel Integrated Performance Primitive library). If you use code and libraries that were created with the old C interface, you might need to manipulate those IplImage structures. Fortunately, there is a convenient way to convert an IplImage into a cv::Mat object.

IplImage* iplImage = cvLoadImage("c:\\img.jpg");
cv::Mat image4(iplImage,false);

The function cvLoadImage is the C-interface function to load images. The second parameter in the constructor of the cv::Mat object indicates that the data will not be copied (set it to true if you want a new copy, while false is the default value so it could have been omitted), that is both IplImage and image4 will share the same image data. You need to be careful here to not create dangling pointers. For this reason, it is safer to encapsulate the IplImage pointer into the reference-counting pointer class provided by OpenCV 2:

cv::Ptr<IplImage> iplImage = cvLoadImage("c:\\img.jpg");

Otherwise, if you need to deallocate the memory pointed by your IplImage structure, you need to do it explicitly:

 cvReleaseImage(&iplImage);

Remember, you should avoid using this deprecate data structure. Instead, always use cv::Mat.

 

Creating a GUI application using Qt


Qt offers a rich library to build a sophisticated GUI with a professional look. Using Qt Creator, the process of GUI creation is made easy. This recipe will show you how to build an OpenCV application with Qt that a user can control using a GUI.

Getting ready

Start Qt Creator which we will use to create our GUI application. It is also possible to create a GUI without this tool, but the use of a visual IDE in which the widgets can simply be dragged and dropped is the easiest way to build a nice looking GUI.

How to do it...

Select Create New Project... and choose Qt GUI Application as seen in the following screenshot:

Give a name and a location to your project. If you then click on Next, you will see that the QtGUI Module is checked. Since we do not need other modules, you can click on Finish at this point. This will create your new project. In addition to the usual project file (.pro) and the main.cpp file, you see two mainwindow files defining the class that contains your GUI window. You will also find a file having the extension .ui, which is the one that describes the UI layout. In fact, if you double-click on it, you will see the current user interface as seen here:

You can drag-and-drop different widgets on it. Drop two Push Buttons as we did in the preceding example. You can resize them and resize the window to make it nice. You should also rename the button labels. Just click on the text and insert the name of your choice.

Let's now add a signal method in order to handle the click button event. Right-click on the first button and select Go to slot... in the contextual menu. The list of possible signals is then displayed as seen in the following screenshot:

Simply select the clicked() signal. This is the one that handles the button pushed events. By doing this, you will be brought to the mainwindow.cpp file. You will see that a new method has been added. This is the slot method that is called when the click() signal is received:

#include "mainwindow.h"
#include "ui_mainwindow.h"
MainWindow::MainWindow(QWidget *parent)
: QMainWindow(parent), ui(new Ui::MainWindow)
{
   ui->setupUi(this);
}
MainWindow::~MainWindow()
{
   delete ui;
}
void MainWindow::on_pushButton_clicked()
{
}

In order to be able to display and then process the image, we need to define a cv::Mat class member variable. This is done in the header file of the MainWindow class. This header now reads as follows:

#ifndef MAINWINDOW_H
#define MAINWINDOW_H

#include <QtGui/QMainWindow>
#include <QFileDialog>
#include <opencv2/core/core.hpp>
#include <opencv2/highgui/highgui.hpp>

namespace Ui
{
   class MainWindow;
}
class MainWindow : public QMainWindow
{
   Q_OBJECT
public:
   MainWindow(QWidget *parent = 0);
   ~MainWindow();
private:
   Ui::MainWindow *ui;
   cv::Mat image; // the image variable
private slots:
   void on_pushButton_clicked();
};

#endif // MAINWINDOW_H

Note that we have also included the core.hpp and the highgui.hpp header files. As we learned in a preceding recipe, we must not forget to edit the project file in order to append the OpenCV library information.

OpenCV code can then be added. The first button opens the source image. This is done by adding the following code to the corresponding slot method:

void MainWindow::on_pushButton_clicked()
{
   QString fileName = QFileDialog::getOpenFileName(this,
        tr("Open Image"), ".", 
      tr("Image Files (*.png *.jpg *.jpeg *.bmp)"));

   image= cv::imread(fileName.toAscii().data());
   cv::namedWindow("Original Image");
   cv::imshow("Original Image", image);
}

You then create a new slot by right-clicking on the second button. This second slot will perform some processing on the selected input image. The following code will simply flip the image:

void MainWindow::on_pushButton_2_clicked()
{
   cv::flip(image,image,1);
   cv::namedWindow("Output Image");
   cv::imshow("Output Image", image);
}

You can now compile and run this program and your 2-button GUI will allow you to select an image and process it.

The input and output images are being displayed on the two highgui windows that we defined.

How it works...

Under the GUI programming framework of Qt, objects communicate using signals and slots. Whenever a widget changes state, or an event occurs, a signal is emitted. This signal has a pre-defined signature, and if another object wants to receive this signal it must define a slot with the same signature. A slot is therefore a special class method that is automatically called when the signal to which it is connected is emitted.

Signals and slots are defined as class methods but must be declared under the Qt access which specifies slots and signals. This is what Qt Creator did when you added a slot to your button, that is:

private slots:
  void on_pushButton_clicked();

Signals and slots are loosely coupled, that is, a signal does not know anything about the objects having slots connected to it, and a slot does not know if a signal is connected to it or not. Also, many slots can be connected to one signal and a slot can receive signals from many objects. The only requirement is that the signatures of the signal and the slot methods must match.

All classes that inherit from the QObject class can contain signals and slots. These will most often be subclasses of a widget class (subclass of QWidget) but any other class can define slots and signals. The signal-and-slot concept is, in fact, a very powerful class communication mechanism. It is however specific to the Qt framework.

In Qt, the main window is an instance of the class, MainWindow. You have access to it through the member variable ui that is declared within the MainWindow class definition. In addition, each widget of the GUI is also an object. When the GUI is created, a pointer to each of the widget instances you have added to the main window is associated with the ui variable. Therefore, you can access the properties and methods of each widget in your program. For example, if you want the Process button to be disabled until an input image is selected, all you need to do is call the following method when the GUI is initialized (that is in the MainWindow constructor):

ui->pushButton_2->setEnabled(false);

The pointer variable pushbutton_2 corresponds here to the Process button. You then enable the button when an image is successfully loaded (in the Open Image button):

if (image.data) {
   ui->pushButton_2->setEnabled(true);
}

It is also worth noting that under Qt, the layout of your GUI is completely described in a XML file. This is the file with the .ui extension. If you go in your project directory and open the .ui file with a text editor, you will be able to read the XML content of this file. Several XML tags are defined. In the case of the example application presented in this recipe, you will find two widget class tags defined as QPushButton. A name is associated with these widget classes' tags that corresponds to the name of the pointer variable attached to the ui object. Each of these defines a geometry property that describes their position and size. Many other property tags are also defined. Qt Creator has a property tab that shows the value of the properties of each widget. Consequently, even if Qt Creator is the best tool to create your GUI, you can also edit the .ui XML file to create and modify your GUI.

There's more...

Displaying an image directly on the GUI is relatively easy with Qt. All you need to do is add a label object to your window. You then assign an image to this label in order to display this image. Remember, you have access to the label instance via the corresponding pointer attribute of the ui pointer (ui->label in our example). But this image must be of type QImage, the Qt data structure to handle images. The conversion is relatively straightforward except that the order of the three color channels needs to be inverted (from BGR in our cv::Mat to RGB in QImage). We can use the cv::cvtColor function for this. The Process button of our simple GUI application can then be changed to:

void MainWindow::on_pushButton_2_clicked()
{
   cv::flip(image,image,1); // process the image

   // change color channel ordering
   cv::cvtColor(image,image,CV_BGR2RGB); 

   // Qt image
   QImage img= QImage((const unsigned char*)(image.data), 
      image.cols,image.rows,QImage::Format_RGB888);

   // display on label
   ui->label->setPixmap(QPixmap::fromImage(img)); 
   // resize the label to fit the image
   ui->label->resize(ui->label->pixmap()->size());
}

As a result, the output image is now displayed directly on the GUI as seen here:

See also

Consult the online Qt documentation located at http://doc.trolltech.com for more information on the Qt GUI module and on the signals and slots mechanism.

About the Author

  • Robert Laganiere

    Robert Laganiere is a professor at the University of Ottawa, Canada. He is also a faculty member of the VIVA research lab and is the coauthor of several scientific publications and patents in content-based video analysis, visual surveillance, driver-assistance, object detection, and tracking. He cofounded Visual Cortek, a video analytics start-up, which was later acquired by iWatchLife. He is also a consultant in computer vision and has assumed the role of chief scientist in a number of start-ups companies, including Cognivue Corp, iWatchLife, and Tempo Analytics. Robert has a Bachelor of Electrical Engineering degree from Ecole Polytechnique in Montreal (1987), and M.Sc. and Ph.D. degrees from INRS-Telecommunications, Montreal (1996).

    Browse publications by this author

Latest Reviews

(3 reviews total)
The purchasing is perfect.
Good book, many pictures, all basic image processing techniques and algorithms explained very well
Excellent
Book Title
Access this book, plus 8,000 other titles for FREE
Access now