Computer Vision with OpenCV 3 and Qt5

These are the modules promised by Qt to be available on all supported platforms. They are basically the foundation of Qt and they contain the majority of the classes that are used by almost all Qt applications. Qt Essential modules include all of the general-purpose modules and classes. Pay real attention to the words general-purpose, since it's exactly what these modules are used for. The following is a brief list for a quick study of the modules and for later reference purposes:

For information, refer to http://doc.qt.io/qt-5/qtmodules.html.

Note that it's not possible and perhaps not a good idea to cover all modules and all classes in this book, and, for the most part we will stick to whatever module and class we require; however, by the end of the book you'll feel comfortable enough exploring all the numerous and powerful modules and classes within Qt by yourself. You'll learn how to include a module and class in your projects in the upcoming chapters, so for now, let's not bother with too much detail and just focus on having a picture of what Qt really is and what it contains in our minds.

These modules may or may not be available on all platforms. This means they are used for developing specific capabilities as opposed to the general-purpose nature of Qt Essentials. A few examples of these type of modules are Qt 3D, Qt Print Support, Qt WebEngine, Qt Bluetooth, and many more. You can always refer to the Qt documentation for a complete list of these modules, and in fact, they are too many to be just listed here. For the most part, you can get a brief idea of what a module is used for by just looking.

For information on this you can refer to http://doc.qt.io/qt-5/qtmodules.html.

These modules offer additional capabilities and are offered with a commercial license from Qt. Yes, you guessed it right, these are the modules that are only available in a paid version of Qt and are not offered in the open source and free version of the Qt, but they're mostly aimed to help with very specific tasks that we don't need at all for the purpose of this book. You can get a list using the Qt documentation pages.

For information on this you can refer to http://doc.qt.io/qt-5/qtmodules.html.

Exactly as their name implies, these modules are the ones that are offered in a state that is not guaranteed to work for all cases; they may or may not contain bugs or other issues, and they are simply still in development and are offered as a preview for test and feedback purposes. As soon as a module is developed and becomes mature enough, it becomes available in other categories mentioned before, and it's taken out of the technology preview category. An example of these types of modules, at the moment of writing this book, is Qt Speech, which is a module that is aimed to add support for text-to-speech in Qt applications. It's always a good idea to keep an eye on these modules if you are looking forward to becoming a fully-fledged Qt developer.

For more information on this, you can refer to http://doc.qt.io/qt-5/qtmodules.html.

When we are talking about developing applications, the platform can have many different meanings, including the OS type, OS version, Compiler type, Compiler version, and Architecture of the processor (32-bit, 64-bit, Arm, and more). Qt supports many (if not all) of the well-known platforms and is quick enough to catch up with new platforms when they are released. The following is the list of platforms supported by Qt at the moment of writing this book (Qt 5.9). Note that you will probably not use all of the platforms mentioned here, but it gives you a sense of how powerful and cross-platform Qt really is:

Reference: http://doc.qt.io/qt-5/supported-platforms.html

As you'll see in the next sections, we'll use the Microsoft Visual C++ 2015 (or from here on, simply MSVC 2015) compiler on Windows since both Qt and OpenCV (which you'll learn about later) highly support it. We'll also use GCC on Linux and Clang on macOS operating systems. All of which are tools that are either free and open source, or they are provided by the operating system provider. Although our main development system will be Windows, we'll cover Linux and macOS operating systems whenever there is a difference between Windows and other versions. So, the default screenshots throughout the book will be that of Windows, with Linux and macOS screenshots provided wherever there's any serious difference between them and not just simply a slight difference between paths, coloring of buttons, and so on.

Creator is the name of the IDE (Integrated Development Environment) used to develop applications. It's also the IDE that we will use throughout this book to create and build our projects. It is worth noting that Qt applications can be created using any other IDE (such as Visual Studio or Xcode), and Qt Creator is not a requirement to build Qt applications, but it's a lightweight and powerful IDE that comes with the Qt Framework installer by default. So, the biggest advantage it has is easy integration with the Qt framework.

The following is a screenshot of the Qt Creator, which shows the IDE in code editing mode. Details on how to use the Qt Creator will be covered in the next chapter, although we'll give it a try for a couple of tests later on in this chapter too, without going into too much detail about it:

Here are some examples of OpenCV main modules. Note that they're just a few (and probably the most widely used) of modules within OpenCV, and covering all of them is out of the scope of this book, but it makes sense to have an idea of what OpenCV contains, just like what we saw Qt earlier in this chapter. Here they are:

As it was mentioned before, Extra modules are mostly wrappers for third-party libraries, that means they only contain interfaces or methods needed to integrate those modules. An example Extra module would be the text module. This module contains interfaces to use Text in images or OCR (Optical Character Recognition), and you'll also need those third-party modules for this work, and they are not covered as a part of this book, but you can always check the OpenCV documentation for an list of Extra modules how they are used.

For more on this you can refer to http://docs.opencv.org/master/index.html.

To be able to install and use Qt, we to first create a Qt account. Although it is not mandatory to do this, it still is highly recommended since you can get access to everything Qt related to this single, unified and free account. For any recent version of Qt that you want to install, you will need your Qt Account credentials, which you will only have if you have created a Qt account. To do this, first, you to go to the Qt website using your favorite browser. Here is the link:

https://login.qt.io/login

 Here is the screenshot of it:

Here, you have to use your email address using the Create Qt Account page just under the Sign in button. The process is almost identical to any similar account creation on the web. You might be asked to enter captcha images to prove you are not a robot or click on an activation link in your email. After going through the procedures required by Qt, you'll have your very own Qt Account user, which is your email, and password. Make a note of that since you'll be needing it later on. We'll refer to it as your Qt Account Credentials from here on.

This is the point where we start downloading the required tools for Qt development. However, where to start? Qt maintains all officially released versions through the Qt Downloads web page. Here's a link: https://download.qt.io/official_releases/.

If you open your browser and navigate to the preceding webpage, you'll be presented with a very simple web page (similar to your file explorer program), and from there, you to choose the right file yourself:

Qt releases all its official tools in here and, as you'll see, the Last modified column will be changing all the time. Some entries not so often, some more often. For now, we will not go into details of what each of these folders contains and what they are used for, but as you'll see later in this book, almost all of the tools required by us are in a single installer file and it's under the qt folder. So, by clicking on each entry, navigate to the following folder: qt/5.9/5.9.1/

You'll notice the same is added to the web address in your browser: https://download.qt.io/official_releases/qt/5.9/5.9.1/

You should note that there might be a newer version at the time when you visit this page, or this version may simply not be available anymore, so you need to start from the Qt Download page mentioned before, and work your way into the latest Qt version folder. Or, you can use the archive link in the  Downloads main page (https://download.qt.io/archive/) to always access previous versions of Qt.

These are pre-built Qt libraries and contain the complete Qt SDK for each of the mentioned operating systems. This means you don't need to build Qt libraries by yourself to be able to use them. Here's what these installation files include in general and the tools we'll use:

You need to start the installation by executing the files you downloaded. If you are on a Windows or macOS operating system, then you just need to run the downloaded file. However, if you are using Linux, then you may need to make your downloaded .run file executable first before being able to actually run it. The following command can be executed on Linux to make your installer file executable:

chmod +x qt-opensource-linux-x64-5.9.1.run

Or, you can simply right-click on the .run file and make it executable using the properties dialog:

Note that you still need a working internet connection even though nothing will be downloaded and it's just to confirm your Qt Account Credentials. Running the installer will present you with the following series of dialogs that you need to get through. Wherever instructions on the dialog boxes are enough, just make sure you read them and provide what is required and press Next, Agree, or similar buttons to proceed forward. As you see in the following screenshots, you need to provide your Qt Account Credentials in order to proceed with the installation. These dialogs are identical on all operating systems:

The rest of the dialog boxes are not shown here, but they are pretty much self-explanatory, and if you have ever installed any app on any computer, then you have definitely seen similar dialogs and they need no introduction in general.

macOS users

When installing Qt for macOS, you will be with the following dialog (or a quite similar one depending on the version of macOS you are using) box if you don't have XCode installed on your Mac:

Unfortunately, it's not enough to follow the Install button, which takes a lot less time than installing Xcode, even though it might look like the obvious choice. You still need to make sure you get Xcode installed on your Mac either by pressing the Get Xcode button, directly getting it from App Store, or you'll face the following while installing Qt:

Use App Store to install the latest version of Xcode (at the moment of writing this book, Xcode 8.3.3 is available) then continue with Qt installation.

On the Select Components dialog, make sure you select at least macOS version. You won't need the rest of the components but installing them won't harm, other than the fact that it might take a lot of space on your computer.

sudo apt-get install build-essential libgl1-mesa-dev

You can now safely run Qt Creator and wonderful applications with it. For now, let's just make sure our Qt installation is working correctly. Don't bother with the details now, since we'll be covering it all during the course of the book, and especially don't worry if you think you don't understand what's really going on behind the scenes. Just run Qt Creator and press the big New Project button seen, as follows:

In the window that appears next, choose Application, Qt Widgets Application, and then click on Choose, as shown in the following screenshot:

In the next window, you need to provide a name and folder (where your test project will be created) and then click Next to proceed forward. Make sure to check the Use as default project location checkbox if you want to have a dedicated folder for your Qt projects. You only need to do this once and afterward all of your projects will be created in that folder. For now, let's just put a name and path since we're only going to test our Qt installation, and click Next. You will see something similar to what is shown in the following screenshot:

In the next window, you need to select a so-called Kit to build your application with. Choose the one that has a name starting with Desktop Qt 5.9.1 and click Next. Depending on what components you selected during the installation of Qt, you may have more than one choice here, and depending on the operating system and compilers installed on your system, you may have more than one Kit with a name that starts with Desktop, so make sure you select the compilers we'll use in this book, which would be the following:

After you have selected the correct Kit according to the ones mentioned earlier, you can click on Next to proceed forward:

You don't really need to bother with the next two windows that appear and just clicking on Next should be enough for our test of Qt installation. The first window makes it easier to a new class and the second one allows you to select a version control tool and track changes in your code:

After you click on the Finish button on the last window, you will be taken to the Edit mode within Qt Creator. We'll cover different aspects of the Qt Creator in the next chapter, so for now, just click on the Run button (or press Ctrl +60;R) to start compiling your test (and empty) application, as shown here:

Depending on the speed of your computer, it will take some time for the build process to be finished. After a few moments, you should see your test (and first) Qt application running. It's just an empty application similar to what is seen in the following screenshot, and the purpose of doing this was to make sure our Qt installation is working as we want it to. Obviously, your empty Qt application may look a little different than this on different operating systems, and different visual options may affect the whole coloring or the way windows are displayed. Nevertheless, your newly built application should look exactly the same (or quite similar) to the window seen here:

In case your application doesn't show up, make sure to go over the instructions once again. Also, make sure you don't have any conflicting installations of Qt or other settings that may interfere with Qt installation. Always refer to documentation pages and the Qt community for answers to unexpected behavior of Qt Creator or other Qt tools. Being an open source project for such a long time, Qt has grown a huge and loyal set of users who are eager to share their knowledge on the internet and answer issues faced by fellow Qt users. So, it's a good idea to keep an eye on the Qt community since you already have a unified Qt that you can use to access Qt forums. This is the same user and password that you created to proceed with the Qt installation process.

Most of the open source frameworks and libraries on the internet, or at least the that want to remain IDE neutral (this means, a project that can be configured and built using any IDE and a project that does not depend on a specific IDE to be able to work), use CMake or similar so-called make systems. I guess this also answers questions such as Why do I need CMake at all?, Why can't they just give the libraries and be done with it? or any other questions like these. So, we need CMake to be able to configure and build OpenCV using sources. CMake is an open source and cross-platform application that allows for configuring and building open source projects (or apps, libraries, or so on), and you can download and use it on all operating systems that were mentioned in the previous sections. At the moment of writing this book, CMake version 3.9.1 can be downloaded from the website download page (https://cmake.org/download/).

Make sure you download and install it on your computer before proceeding forward. There's nothing special about the CMake installation that needs to be taken note of, except for the fact that you should make sure you install the GUI version since that's what we'll be using in the next section and it's the default option in the link provided earlier.

maintains its official and stable releases under the  page at their website (http://opencv.org/releases.html):

This is where you can always find the most recent release of OpenCV sources, documentation, and pre-built binaries for Windows, Android, and iOS. As new versions are released, they are added at the top of the page. At the moment of writing this book, version 3.3.0 is the most recent release of OpenCV and that's the version we'll use. So, without further ado, you should go ahead and download the Sources by clicking on the Sources link for version 3.3.0. Download the source zip file to a folder of your choice, extract it, and make a note of the extracted path since we'll use it in a few moments.

Now that we have all required tools and files to build OpenCV, we can start the by running the CMake GUI application. If you've installed CMake correctly, then you should be able to run it from your desktop, start menu, or dock, depending on your operating system.

sudp apt-get install libgtk2.0-dev and pkg-config  

After you run the CMake GUI application, you need to set the following two folders:

After these two folders are set, you can proceed forward by clicking on the Configure button, as shown in the following screenshot:

Clicking on the Configure button will start the configuration process. You may be asked to create the build folder if it does not already exist, to which you need to answer by clicking on the Yes button. Don't worry if you still feel like you're just repeating what's in the book. This will all sink in as you move forward with the book and the instructions. For now, let's just focus on getting OpenCV built and installed on your computer. Consider this an installation that is not as easy as just clicking on a few Next buttons, and as soon as you start using OpenCV, it will all make sense. So, in the window that will appear next, select the correct generator and click Finish. See the following instructions for the correct generator type on each operating system:

You'll see a short process in CMake, and when it's finished, you'll be able to set various parameters to configure your OpenCV build. There are many parameters to configure, so we'll just bother with the ones that directly affect us.

Make sure you check the checkbox next to the BUILD_opencv_world option. This will allow building all OpenCV modules into a single library. So, if you're on Windows, you'll have only a single DLL file that includes all OpenCV functions. As you'll see later on, this has the advantage of bothering with only a single DLL file when you want to deploy your computer vision applications. Of course, the obvious downside to this is that your application installer size will be a little bit larger. But again, ease of deployment,  will prove to be much more useful later on.

You need to click on the Configure button again after changing the build parameters. Wait for the reconfiguration to finish and finally click on the Generate button. This will make your OpenCV build ready to compile. For the next part, you'll need to execute somewhat different commands if you're using Windows, macOS, or Linux operating systems. So, here they are:

Windows users: Go to the OpenCV build folder that you set earlier in CMake (in our case it was c:\dev\opencv\build). There should be a Visual Studio 2015 Solution (that is, the type of MSVC projects) that you can easily execute and build OpenCV with. You can also immediately click on the Open Project button, which is right next to the Generate button on CMake. You can also just run Visual Studio 2015 and open the Solution file you just created for OpenCV.

After Visual Studio is opened, you need to select Batch Build from the Visual Studio main menu. It's right under Build:

Make sure checkboxes in the Build column are enabled for ALL_BUILD and INSTALL, as shown in the following screenshot:

For macOS and Linux users: Run a terminal instance and execute the following commands after you switch to the Binaries folder you chose in CMake. To switch to a specific folder, you need to use the cd command. After you are in the OpenCV build folder (it should be the folder in your home that you chose when you opened CMake), you need to execute the following command. You'll be asked to provide the administrative password, just provide your password and press Enter to proceed forward with building OpenCV:

    sudo make

This will trigger the build process, and depending on your speed this can take quite some time. Wait until building all libraries are finished and you see the progress bar reach 100%.

After the long wait, there's only one more command left to execute for macOS and Linux users. If you are on Windows, you can close Visual Studio IDE and proceed to the next step.

This last command for non-Windows users will make sure OpenCV is installed on your computer and is completely ready to use. You're fine to proceed forward if you didn't miss any commands in this section. You have an OpenCV framework ready to build computer vision applications with.

Remember we mentioned that is a framework and you will learn how to use it with Qt? Well, Qt offers a very easy-to-use method to include any third-party library, such as OpenCV, in your Qt projects. To be able to use OpenCV in Qt, you need to use a special kind of file which is called a PRI file. PRI files are files that are used to add third-party modules and include them in your Qt projects. Note that you only need to do this once, and for the rest of the book you will use this file in all your projects, so it's a very crucial (yet very easy) part of the Qt configuration.

Start by creating a text file in a folder of your choice. I recommend using the same folder you used for OpenCV build because that can help make sure you have all your OpenCV related files in a single folder. However, technically speaking, this file can be located anywhere on your computer. Rename the file to opencv.pri and open it using any text editor and write the following inside this PRI file:

INCLUDEPATH += c:/dev/opencv/build/install/include 
Debug: { 
LIBS += -lc:/dev/opencv/build/install/x86/vc14/lib/opencv_world330d 
} 
Release: { 
LIBS += -lc:/dev/opencv/build/install/x86/vc14/lib/opencv_world330 
} 

 INCLUDEPATH += /usr/local/include 
 LIBS += -L/usr/local/lib \ 
    -lopencv_world 

There's one more thing left for Windows users, and that is adding OpenCV DLLs folder to the PATH environment variable. Simply open the System properties window and add a new entry in your PATH. They are usually separated by ; so just add a new one after that. Note that this path is related to the Windows operating system only, and where it can find your DLL files for OpenCV for an easier build process; users of Linux and macOS don't need to do anything regarding this.

The worst is past us, and we are now ready to dig into the world of Computer Vision and start building exciting applications using Qt and OpenCV. Although this one last step is called Testing OpenCV, it's actually the first Qt+OpenCV application that you'll write, as simple as it may seem at first. Our aim in this section is not to bother with any details of how things are working and what goes on behind the scenes, but just to make sure we have configured everything correctly and avoid wasting time with configuration-related issues later in the book. If you have followed everything as it was described and executed all instructions in the correct order, then by now, you shouldn't be worrying about anything, but it's always best to verify, and that is what we'll do now.

So, we'll verify our installation with a very simple application that reads an image file from the hard disk and just displays it. Again, don't bother with any code related details since we'll cover it all in the upcoming chapters and just focus on the task at hand, that is testing our OpenCV installation. Start by running Qt Creator and creating a new Console Application. You have already done a very similar task before when you were testing your Qt installation. You need to follow exactly the same instructions, except instead of Qt Widgets, you have to make sure you select Qt Console Application. Repeat all of the similar steps like before, until you end up in Qt Creator edit mode. If you are asked about build system just select qmake, which should be selected by default, so you just need to move forward. Make sure to give a name such as QtCvTest to your project. This time, instead of clicking on the Run button, double-click on your project's PRO file, which you can find in the explorer at the left side of the Qt Creator screen, and add the following line at the end of your project's PRO file:

    include(c:/dev/opencv/opencv.pri) 

Note that this, in fact, is a type of hard-coding that should always be avoided, and as we'll see in the later chapters, we will write more sophisticated PRO files that work on all operating systems without changing a single line; however, since we're just testing our OpenCV installation, it is OK for now to go ahead with a little bit of hard-coding to simplify things a little bit and not overwhelm you with more configuration details.

So, back to what we were doing, the moment you save your PRO file by pressing Ctrl + S, you will notice a quick process and update in the project explorer and the opencv.pri file will appear in the explorer. You can always change the contents of opencv.pri from here, but you will probably never need to do it. Ignore the Comment-Like lines and make sure your PRO file is similar to what I have here:

 QT += core 
 QT -= gui 
 CONFIG += c++11 
 TARGET = QtCvTest 
 CONFIG += console 
 CONFIG -= app_bundle 
 TEMPLATE = app 
 SOURCES += main.cpp 
 DEFINES += QT_DEPRECATED_WARNINGS 
 include(c:/dev/opencv/opencv.pri) 

This simple line of code in our project's PRO file is basically the result of all of our efforts in this chapter. Now, we are able to add OpenCV to our Qt projects by simply including this simple piece of code in every Computer Vision project that we want to build using Qt and OpenCV.

In the upcoming chapters, we'll learn about PRO files in Qt and everything about the preceding code; however, for now let's just move on knowing that this file is responsible for our project's configurations. So, the last line, which is pretty much self-explanatory, simply means we want to add include headers and libraries to our Qt project.

Now, you can actually write some OpenCV code. Open up your main.cpp file and change the contents of it to make it similar to this:

 #include <QCoreApplication> 
 #include "opencv2/opencv.hpp" 
 int main(int argc, char *argv[]) 
 { 
    QCoreApplication a(argc, argv); 
    using namespace cv; 
    Mat image = imread("c:/dev/test.jpg"); 
    imshow("Output", image); 
    return a.exec(); 
 } 

By default, your main.cpp file should already have most of what is in the preceding code, but you'll notice the include line at the top and the three lines responsible for reading and displaying a test image from my computer. You can replace the path to any other image (just make sure you stick to JPG or PNG files for now), and it's very important to make sure the image file exists and it's accessible, otherwise, our test might fail, even though our installation is still correct. The whole code is almost self-explanatory but once again, you shouldn't bother with the codes for now since we're just testing our build, so just press the Run button to have your image file displayed. You should see something similar to the following screenshot on your computer: