Mastering OpenCV 4 with Python

4.8 (5 reviews total)
By Alberto Fernández Villán
  • Instant online access to over 8,000+ books and videos
  • Constantly updated with 100+ new titles each month
  • Breadth and depth in over 1,000+ technologies
  1. Setting Up OpenCV

About this book

OpenCV is considered to be one of the best open source computer vision and machine learning software libraries. It helps developers build complete projects in relation to image processing, motion detection, or image segmentation, among many others. OpenCV for Python enables you to run computer vision algorithms smoothly in real time, combining the best of the OpenCV C++ API and the Python language.

In this book, you'll get started by setting up OpenCV and delving into the key concepts of computer vision. You'll then proceed to study more advanced concepts and discover the full potential of OpenCV. The book will also introduce you to the creation of advanced applications using Python and OpenCV, enabling you to develop applications that include facial recognition, target tracking, or augmented reality. Next, you'll learn machine learning techniques and concepts, understand how to apply them in real-world examples, and also explore their benefits, including real-time data production and faster data processing. You'll also discover how to translate the functionality provided by OpenCV into optimized application code projects using Python bindings. Toward the concluding chapters, you'll explore the application of artificial intelligence and deep learning techniques using the popular Python libraries TensorFlow, and Keras.

By the end of this book, you'll be able to develop advanced computer vision applications to meet your customers' demands.

Publication date:
March 2019
Publisher
Packt
Pages
532
ISBN
9781789344912

 

Chapter 1. Setting Up OpenCV

Mastering OpenCV 4 with Python will give you the knowledge to build projects involving Open Source Computer Vision Library (OpenCV) and Python. These two technologies (the first one is a programming language, while the second one is a computer vision and machine learning library) will be introduced. Also, you will learn why the combination of OpenCV and Python has the potential to build every kind of computer application. Finally, an introduction about the main concepts related to the content of this book will be provided. 

In this chapter, you will be given step-by-step instructions to install everything you need to start programming with Python and OpenCV. This first chapter is quite long, but do not worry, because it is divided into easily assimilated sections, starting with general terminology and concepts, which assumes that the reader is new to this information. At the end of this chapter, you will be able to build your first project involving Python and OpenCV. 

The following topics will be covered in this chapter:

  • A theoretical introduction to the OpenCV library
  • Installing Python OpenCV and other packages
  • Running samples, documentation, help, and updates
  • Python and OpenCV project structure
  • First Python and OpenCV project
 

Technical requirements


This chapter and subsequent chapters are focused on Python (a programming language) and OpenCV (a computer vision library) concepts in connection with computer vision, machine learning, and deep learning techniques (among others). Therefore, Python (https://www.python.org/) and OpenCV (https://opencv.org/) should be installed on your computer. Moreover, some Python packages related to scientific computing and data science should also be installed (for example, NumPy (http://www.numpy.org/) or Matplotlib (https://matplotlib.org/)). 

Additionally, it is recommended that you install an integrated development environment (IDE) software package because it facilitates computer programmers with software development. In this sense, a Python-specific IDE is recommended. The de facto Python IDE is PyCharm, which can be downloaded from https://www.jetbrains.com/pycharm/. 

Finally, in order to facilitate GitHub activities (for example, cloning a repository), you should install a Git client. In this sense, GitHub provides desktop clients that include the most common repository actions. For an introduction to Git commands, check out https://education.github.com/git-cheat-sheet-education.pdf, where commonly used Git command-line instructions are summarized. Additionally, instructions for installing a Git client on your operating system are included. 

The GitHub repository for this book, which contains all the supporting project files necessary to work through the book from the first chapter to the last, can be accessed at https://github.com/PacktPublishing/Mastering-OpenCV-4-with-Python.

Finally, it should be noted that the README file of the GitHub repository for Mastering OpenCV with Python includes the following, which is also attached here for the sake of completeness:

  • Code testing specifications
  • Hardware specifications
  • Related books and products

Code testing specifications

Mastering OpenCV 4 with Python requires some installed packages, which you can see here:

  • Chapter 1, Setting Up OpenCV: opencv-contrib-python
  • Chapter 2, Image Basics in OpenCV: opencv-contrib-python and matplotlib
  • Chapter 3, Handling Files and Images: opencv-contrib-python and matplotlib
  • Chapter 4, Constructing Basic Shapes in OpenCV: opencv-contrib-python and matplotlib
  • Chapter 5, Image Processing Techniques: opencv-contrib-python and matplotlib
  • Chapter 6, Constructing and Building Histograms: opencv-contrib-python and matplotlib
  • Chapter 7, Thresholding Techniques: opencv-contrib-python, matplotlib, scikit-image, and scipy
  • Chapter 8, Contours Detection, Filtering, and Drawing: opencv-contrib-python and matplotlib
  • Chapter 9, Augmented Reality: opencv-contrib-python and matplotlib
  • Chapter 10, Machine Learning with OpenCV: opencv-contrib-python and matplotlib
  • Chapter 11, Face Detection, Tracking, and Recognition: opencv-contrib-python, matplotlib, dlib, face-recognition, cvlib, requests, progressbar, keras, and tensorflow
  • Chapter 12, Introduction to Deep Learning: opencv-contrib-python, matplotlib, tensorflow, and keras
  • Chapter 13, Mobile and Web Computer Vision with Python and OpenCV: opencv-contrib-python, matplotlib, flask, tensorflow, keras, requests, and pillow

Note

Make sure that the version numbers of your installed packages are equal to, or greater than, versions specified here to ensure that the code examples run correctly.

If you want to install the exact versions this book was tested on, include the version when installing from pip, which is indicated as follows.

Run the following command to install the both main and contrib modules:

  • Install opencv-contrib-python:
pip install opencv-contrib-python==4.0.0.21

 

 

 

 

 

 

It should be noted that OpenCV requires numpy. numpy-1.16.1 has been installed when installing opencv-contrib-python==4.0.0.21.

Run the following command to install Matplotlib library:

  • Install matplotlib:
pip install matplotlib==3.0.2

It should be noted that matplotlib requires kiwisolver, pyparsing, six, cycler, and python-dateutil.

cycler-0.10.0, kiwisolver-1.0.1, pyparsing-2.3.1, python-dateutil-2.8.0, and six-1.12.0 have been installed when installing matplotlib==3.0.2.

Run the following command to install library which contains collections of algorithm for image processing:

  • Install scikit-image:
pip install scikit-image==0.14.2

It should be noted that scikit-image requires cloudpickle, decorator, networkx, numpy, toolz, dask, pillow, PyWavelets, and six.

PyWavelets-1.0.1, cloudpickle-0.8.0, dask-1.1.1, decorator-4.3.2, networkx-2.2, numpy-1.16.1, pillow-5.4.1, six-1.12.0, and toolz-0.9.0 have been installed when installing scikit-image==0.14.2.

If you need SciPy, you can install it with the following command:

  • Install scipy:
pip install scipy==1.2.1

It should be noted that scipy requires numpy.

numpy-1.16.1 has been installed when installing scipy==1.2.1.

Run the following command to install dlib library: 

  • Install dlib:
pip install dlib==19.8.1

To install the face recognition library, run the following command:

  • Install face-recognition:
pip install face-recognition==1.2.3

It should be noted that face-recognition requires dlib, Click, numpy, face-recognition-models, and pillow.

dlib-19.8.1, Click-7.0, face-recognition-models-0.3.0, and pillow-5.4.1 have been installed when installing face-recognition==1.2.3.

Run the following command to install open source computer vision library:

  • Install cvlib:
pip install cvlib==0.1.8

To install requests library run the following command:

  • Install requests:
pip install requests==2.21.0

It should be noted that requests requires urllib3, chardet, certifi, and idna.

urllib3-1.24.1, chardet-3.0.4, certifi-2018.11.29, and idna-2.8 have been installed when installing requests==2.21.0.

Run the following command to install text progress bar library:

  • Install progressbar:
pip install progressbar==2.5

Run the following command to install Keras library for deep learning:

  • Install keras:
pip install keras==2.2.4

It should be noted that keras requires numpy, six, h5py, keras-applications, scipy, keras-preprocessing, and pyyaml.

h5py-2.9.0, keras-applications-1.0.7, keras-preprocessing-1.0.9, numpy-1.16.1 pyyaml-3.13, and scipy-1.2.1 six-1.12.0 have been installed when installing keras==2.2.4.

Run the following command to install TensorFlow library: 

  • Install tensorflow:
pip install tensorflow==1.12.0

It should be noted that TensorFlow requires termcolor, numpy, wheel, gast, six, setuptools, protobuf, markdown, grpcio, werkzeug, tensorboard, absl-py, h5py, keras-applications, keras-preprocessing, and astor.

termcolor-1.1.0, numpy-1.16.1, wheel-0.33.1, gast-0.2.2, six-1.12.0, setuptools-40.8.0, protobuf-3.6.1, markdown-3.0.1, grpcio-1.18.0, werkzeug-0.14.1, tensorboard-1.12.2, absl-py-0.7.0, h5py-2.9.0, keras-applications-1.0.7, keras-preprocessing-1.0.9, and astor-0.7.1 have been installed when installing tensorflow==1.12.0.

Run the following command to install Flask library:

  • Install flask:
pip install flask==1.0.2

It should be noted that flask requires Werkzeug, click, itsdangerous, and MarkupSafe Jinja2.

Jinja2-2.10, MarkupSafe-1.1.1, Werkzeug-0.14.1, click-7.0, and itsdangerous-1.1.0 have been installed when installing flask==1.0.2.

Hardware specifications

The hardware specifications are as follows:

  • 32-bit or 64-bit architecture
  • 2+ GHz CPU
  • 4 GB RAM
  • At least 10 GB of hard disk space available
 

Understanding Python


Python is an interpreted high-level and general-purpose programming language with a dynamic type system and automatic memory management. The official home of the Python programming language is https://www.python.org/. The popularity of Python has risen steadily over the past decade. This is because Python is a very important programming language in some of today's most exciting and challenging technologies. Artificial intelligence (AI), machine learning, neural networks, deep learning, Internet of Things (IoT), and robotics (among others) rely on Python. 

Here are some advantages of Python:

  • Python is considered a perfect language for scientific computing, mainly for four reasons:
    • It is very easy to understand.
    • It has support (via packages) for scientific computing.
    • It removes many of the complexities other programming languages have.
    • It has a simple and consistent syntax. 
  • Python stimulates rapid prototyping because it helps in easy writing and execution of code. Indeed, Python can implement the same logic with as little as one-fifth of the code as compared to other programming languages. 
  • Python has a lot of prebuilt libraries (NumPy, SciPy, scikit-learn) for every need of your AI project. Python benefits from a rich ecosystem of libraries for scientific computing.
  • It is an independent platform, which allows developers to save time in testing on different platforms.
  • Python offers some tools, such as Jupyter Notebook, that can be used to share scripts in an easy and comfortable way. This is perfect in scientific computing because it stimulates collaboration in an interactive computational environment.

Introducing OpenCV

OpenCV is a C++ programming library, with real-time capabilities. As it is written in optimized C/C++, the library can profit from multi-core processing. A theoretical introduction about the OpenCV library is carried out in the next section.

In connection with the OpenCV library, here are some reasons for its popularity:

  • Open source computer vision library
  • OpenCV (BSD license—https://en.wikipedia.org/wiki/BSD_licenses) is free
  • Specific library for image processing
  • It has more than 2,500 optimized algorithms, including state-of-the-art computer vision algorithms
  • Machine learning and deep learning support
  • The library is optimized for performance
  • There is a big community of developers using and supporting OpenCV
  • It has C++, Python, Java, and MATLAB interfaces
  • The library supports Windows, Linux, Android, and macOS
  • Fast and regular updates (official releases now occur every six months)

Contextualizing the reader

In order to contextualize the reader, it is necessary to establish and set the bases in relation to the main concepts concerning the theme of this book. The last few years have seen considerable interest in AI and machine learning, specifically in the area of deep learning. These terms are used interchangeably and very often confused with each other. For the sake of completeness and clarification, these terms are briefly described next.

AI refers to a set of technologies that enable machines – computers or robotic systems – to process information in the same way humans would.

The term AI is commonly used as an umbrella for a machine technology in order to provide intelligence covering a wide range of methods and algorithms. Machine Learning is the process of programming computers to learn from historical data to make predictions on new data. Machine learning is a sub-discipline of AI and refers to statistical techniques that machines use on the basis of learned interrelationships. On the basis of data gathered or collected, algorithms are independently learned by computers. These algorithms and methods include support vector machine, decision tree, random forest, logistic regression, Bayesian networks, and neural networks.

 

 

Neural Networks are computer models for machine learning that are based on the structure and functioning of the biological brain. An artificial neuron processes a plurality of input signals and, in turn, when the sum of the input signals exceeds a certain threshold value, signals to further adjacent neurons will be sent. Deep Learning is a subset of machine learning that operates on large volumes of unstructured data, such as human speech, text, and images. A deep learning model is an artificial neural network that comprises multiple layers of mathematical computation on data, where results from one layer are fed as input into the next layer in order to classify the input data and/or make a prediction.

Therefore, these concepts are interdependent in a hierarchical way, AI being the broadest term and deep learning the most specific. This structure can be seen in the next diagram:

Computer vision is an interdisciplinary field of Artificial Intelligence that aims to give computers and other devices with computing capabilities a high-level understanding from both digital images and videos, including functionality for acquiring, processing, and analyzing digital images. This is why computer vision is, partly, another sub-area of Artificial Intelligence, heavily relying on machine learning and deep learning algorithms to build computer vision applications. Additionally, Computer vision is composed of several technologies working together—Computer graphics, Image processing, Signal processing, Sensor technology, Mathematics, or even Physics.

Therefore, the previous diagram can be completed to introduce the computer vision discipline:

 

A theoretical introduction to the OpenCV library


OpenCV is a programming library with real-time computer vision capabilities and it is free for both academic and commercial use (BSD license). In this section, an introduction about the OpenCV library will be given, including its main modules and other useful information in connection with the library.

OpenCV modules

OpenCV (since version 2) is divided into several modules, where each module can be understood, in general, as being dedicated to one group of computer vision problems. This division can be seen in the next diagram, where the main modules are shown:

OpenCV modules are shortly described here:

  • core: Core functionality. Core functionality is a module defining basic data structures and also basic functions used by all other modules in the library.
  • imgproc: Image processing. An image-processing module that includes image filtering, geometrical image transformations, color space conversion, and histograms.
  • imgcodecs: Image codecs. Image file reading and writing.
  • videoio: Video I/O. An interface to video capturing and video codecs.
  • highgui: High-level GUI. An interface to UI capabilities. It provides an interface to easily do the following:
    • Create and manipulate windows that can display/show images
    • Add trackbars to the windows, keyboard commands, and handle mouse events 
  • video: Video analysis. A video-analysis module including background subtraction, motion estimation, and object-tracking algorithms.
  • calib3d: Camera calibration and 3D reconstruction. Camera calibration and 3D reconstruction covering basic multiple-view geometry algorithms, stereo correspondence algorithms, object pose estimation, both single and stereo camera calibration, and also 3D reconstruction.
  • features2d: 2D features framework. This module includes feature detectors, descriptors, and descriptor matchers.
  • objdetect: Object detection. Detection of objects and instances of predefined classes (for example, faces, eyes, people, and cars).
  • dnn: Deep neural network (DNN) module. This module contains the following:
    • API for new layers creation
    • Set of built useful layers
    • API to construct and modify neural networks from layers
    • Functionality for loading serialized networks models from different deep learning frameworks
  • ml: Machine learning. The Machine Learning Library (MLL) is a set of classes and methods that can be used for classification, regression, and clustering purposes.
  • flann: Clustering and search in multi-dimensional spaces. Fast Library for Approximate Nearest Neighbors (FLANN) is a collection of algorithms that are highly suited for fast nearest-neighbor searches.
  • photo: Computational photography. This module provides some functions for computational photography.
  • stitching: Images stitching. This module implements a stitching pipeline that performs automatic panoramic image stitching.
  • shape: Shape distance and matching. Shape distance and matching module that can be used for shape matching, retrieval, or comparison.
  • superres: Super-resolution. This module contains a set of classes and methods that can be used for resolution enhancement. 
  • videostab: Video stabilization. This module contains a set of classes and methods for video stabilization.
  • viz: 3D visualizer. This module is used to display widgets that provide several methods to interact with scenes and widgets.

OpenCV users

Regardless of whether you are a professional software developer ora novice programmer, the OpenCV library will be interesting for graduate students, researchers, and computer programmers in image-processing and computer vision areas. The library has gained popularity among scientists and academics because many state-of-the-art computer vision algorithms are provided by this library.

 

 

Additionally, it is often used as a teaching tool for both computer vision and machine learning. It should be taken into account that OpenCV is robust enough to support real-world applications. That is why OpenCV can be used for non-commercial and commercial products. For example, it is used by companies such as Google, Microsoft, Intel, IBM, Sony, and Honda. Research institutes in leading universities, such as MIT, CMU, or Stanford, provide support for the library. OpenCV has been adopted all around the world. It has more than 14 million downloads and more than 47,000 people in its community.

OpenCV applications

OpenCV is being used for a very wide range of applications:

  • 2D and 3D feature toolkits
  • Street view image stitching
  • Egomotion estimation
  • Facial-recognition system
  • Gesture recognition
  • Human-computer interaction
  • Mobile robotics
  • Motion understanding
  • Object identification
  • Automated inspection and surveillance
  • Segmentation and recognition
  • Stereopsis stereo vision – depth perception from two cameras
  • Medical image analysis
  • Structure from motion
  • Motion tracking
  • Augmented reality
  • Video/image search and retrieval
  • Robot and driverless car navigation and control
  • Driver drowsiness and distraction detection

Why citing OpenCV in your research work

If you are using OpenCV in your research, it is recommended you cite the OpenCV library. This way, other researchers can better understand your proposed algorithms and reproduce your results for better credibility. Additionally, OpenCV will increase repercussion, resulting in a better computer vision library. The BibTex entry for citing OpenCV is shown in the following code:

@article{opencv_library,
 author = {Bradski, G.},
 citeulike-article-id = {2236121},
 journal = {Dr. Dobb's Journal of Software Tools},
 keywords = {bibtex-import},
 posted-at = {2008-01-15 19:21:54},
 priority = {4},
 title = {{The OpenCV Library}},
 year = {2000}
}
 

Installing OpenCV, Python, and other packages


OpenCV, Python, and AI-related packages can be installed on most operating systems. We will see how to install these packages by means of different approaches.

Note

Make sure you check out the different installation options before choosing the one that best suits your needs.

Additionally, at the end of this chapter, an introduction to Jupyter Notebook is given due to the popularity of these documents, which can be run to perform data analysis.

Installing Python, OpenCV, and other packages globally

In this section, you will see how to install Python, OpenCV, and any other package globally. Specific instructions are given for both Linux and Windows operating systems.

Installing Python

We are going to see how to install Python globally on both the Linux and Windows operating systems.

Installing Python on Linux

On Debian derivatives such as Ubuntu, use APT to install Python. Afterwards, it is recommended to upgrade the pip version. pip (https://pip.pypa.io/en/stable/) is the PyPA (https://packaging.python.org/guides/tool-recommendations/) recommended tool for installing Python packages:

$ sudo apt-get install python3.7

$ sudo pip install --upgrade pip

To verify that Python has been installed correctly, open a Command Prompt or shell and run the following command:

$ python3 --version
 Python 3.7.0

Installing Python on Windows

Go to https://www.python.org/downloads/. The default Python Windows installer is 32 bits. Start the installer. Select Customize installation:

 On the next screen, all the optional features should be checked:

Finally, on the next screen, make sure to check Add Python to environment variables and Precompile standard library. Optionally, you can customize the location of the installation, for example, C:\Python37:

Press the Install button and, in a few minutes, the installation should be ready. On the last page of the installer, you should also press Disable path length limit:

To check whether Python has been installed properly, press and hold the Shift key and right-click with your mouse somewhere on your desktop. Select Open command window here. Alternatively, on Windows 10, use the lower-left search box to search for cmd. Now, write python in the command window and press the Enter key. You should see something like this:

You should also upgrade pip:

$ python -m pip install --upgrade pip

Installing OpenCV

Now, we are going to install OpenCV on both the Linux and Windows operating systems. First, we are going to see how to install OpenCV on Linux, and then how to install OpenCV on Windows.

Installing OpenCV on Linux

Ensure you have installed NumPy. To install NumPy, enter the following:

$ pip3 install numpy

Then install OpenCV:

$ pip3 install opencv-contrib-python

Additionally, we can install Matplotlib, which is a Python plotting library that produces quality figures:

$ pip3 install matplotlib

Installing OpenCV on Windows

Ensure you have installed NumPy. To install NumPy, enter the following:

$ pip install numpy

Then install OpenCV:

$ pip install opencv-contrib-python

Additionally, we can install Matplotlib:

$ pip install matplotlib

 

 

Testing the installation

One way to test the installation is to execute an OpenCV Python script. In order to do it, you should have two files, logo.png and test_opencv_installation.py, in a specific folder: 

Open a cmd and go to the path where these two files are. Next, we can check the installation by typing the following: 

python test_opencv_installation.py

You should see both the OpenCV RGB logo and the OpenCV grayscale logo:

In that case, the installation has been successful.

 

Installing Python, OpenCV, and other packages with virtualenv


virtualenv (https://pypi.org/project/virtualenv/) is a very popular tool that creates isolated Python environments for Python libraries. virtualenv allows multiple Python projects that have different (and sometimes conflicting) requirements. In a technical way, virtualenv works by installing some files under a directory (for example, env/).

Additionally, virtualenv modifies the PATH environment variable to prefix it with a custom bin directory (for example, env/bin/). Additionally, an exact copy of the Python or Python3 binary is placed in this directory. Once this virtual environment is activated, you can install packages in the virtual environment using pip. virtualenv is also recommended by the PyPA (https://packaging.python.org/guides/tool-recommendations/). Therefore, we will see how to install OpenCV or any other packages using virtual environments.

Usually, pip and virtualenv are the only two packages you need to install globally. This is because, once you have installed both packages, you can do all your work inside a virtual environment. In fact, virtualenv is really all you need, because this package provides a copy of pip, which gets copied into every new environment you create.

Now, we will see how to install, activate, use, and deactivate virtual environments. Specific commands are given now for both Linux and Windows operating systems. We are not going to add a specific section for each of the operating systems, because the process is very similar in each one. Let's start installing virtualenv:

$ pip install virtualenv

Inside this directory (env), some files and folders are created with all you need to run your python applications. For example, the new python executable will be located at /env/scripts/python.exe. The next step is to create a new virtual environment. First, change the directory into the root of the project directory. The second step is to use the virtualenv command-line tool to create the environment:

$ virtualenv env

Here, env is the name of the directory you want to create your virtual environment inside. It is a common convention to call the directory you want to create your virtual environment inside env, and to put it inside your project directory. This way, if you keep your code at ~/code/myproject/, the environment will be at ~/code/myproject/env/.

The next step is to activate the env environment that you have just created using the command-line tool to execute the activate script, which is in the following location:

  • ~/code/myprojectname/env/bin/activate (Linux)
  • ~/code/myprojectname/env/Scripts/activate (Windows)

For example, under Windows, you should type the following:

$ ~/code/myprojectname/env/Scripts/activate
 (env) $

 

 

 

 

Now you can install the required packages only for this activated environment. For example, if you want to install Django, which is a free and open source web framework, written in Python, you should type this:

(env)$ pip install Django

Note

Remember that this package will only be installed for the myprojectname project.

You can also deactivate the environment by executing the following:

$ deactivate
$

You should see that you have returned to your normal prompt, indicating that you are no longer in any virtualenv. Finally, if you want to delete your environment, just type the following:

$ rmvirtualenv test
 

Python IDEs to create virtual environments with virtualenv


In the next section, we are going to create virtual environments with PyCharm, which is a Python IDE. But before doing that, we are going to discuss IDEs. An IDE is a software application that facilitates computer programmers with software development. IDEs present a single program where all the development is done. In connection with Python IDEs, two approaches can be found: 

  •  General editors and IDEs with Python support
  •  Python-specific editors and IDEs

In the first category (general IDEs), some examples should be highlighted: 

  • Eclipse + PyDev 
  • Visual Studio + Python Tools for Visual Studio
  • Atom + Python extension 

In the second category, here are some Python-specific IDEs:

  • PyCharm: One of the best full-featured, dedicated IDEs for Python. PyCharm installs quickly and easily on Windows, macOS, and Linux platforms. It is the de facto Python IDE environment.
  • Spyder: Spyder, which comes with the Anaconda package manager distribution, is an open source Python IDE that is highly suited for data science workflows. 
  • Thonny: Thonny is intended to be an IDE for beginners. It is available for all major platforms (Windows, macOS, Linux), with installation instructions on the site. 

In this case, we are going to install PyCharm (the de facto Python IDE environment) Community Edition. Afterwards, we are going to see how to create virtual environments using this IDE. PyCharm can be downloaded from https://www.jetbrains.com/pycharm/. PyCharm can be installed on Windows, macOS, and Linux:

After the installation of PyCharm, we are ready to use it. Using PyCharm, we can create virtual environments in a very simple and intuitive way.

Note

PyCharm makes it possible to use the virtualenv tool to create a project-specific isolated virtual environment. Additionally, the virtualenv tool comes bundled with PyCharm, so the user does not need to install it.

After opening Pycharm, you can click Create New Project. If you want to create a new environment, you should click on Project Interpreter: New Virtualenv environment. Then click on New environment using Virtualenv. This can be seen in the next screenshot:

You should note that the virtual environment is named (by default in PyCharm) venv and located under the project folder. In this case, the project is named test-env-pycharm and the virtual environment, venv, is located at test-env-pycharm/venv. Additionally, you can see that the venv name can be changed according to your preferences.

When you click on the Create button, PyCharm loads the project and creates the virtual environment. You should see something like this:

After the project is created, you are ready to install a package with just a few clicks. Click on File, then click on Settings... (Ctrl + Alt + S). A new window will appear, showing something like this:

Now, click on Project: and select Project Interpreter. On the right-hand side of this screen, the installed packages are shown in connection with the selected Project Interpreter. You can change it on top of this screen. After selecting the appropriate interpreter (and, hence, the environment for your project), you can install a new package. To do so, you can search in the upper-left input box. In the next screenshot, you can see an example of searching for the numpy package:

 

 

You can install the package (latest version by default) by clicking on Install Package. You can also specify a concrete version, as can be seen in the previous screenshot:

After the installation of this package, we can see that we now have three installed packages on our virtual environment. Additionally, it is very easy to change between environments. You should go to Run/Debug Configurations and click on Python interpreter to change between environments. This feature can be seen in the next screenshot:

 

 

Finally, you may have noticed that, in the first step, of creating a virtual environment with PyCharm, options other than virtualenv are possible. PyCharm gives you the ability to create virtual environments using Virtualenv, Pipenv, and Conda:

We previously introduced Virtualenv and how to work with this tool for creating isolated Python environments for Python libraries. 

Pyenv (https://github.com/pyenv/pyenv) is used to isolate Python versions. For example, you may want to test your code against Python 2.6, 2.7, 3.3, 3.4, and 3.5, so you will need a way to switch between them.

Conda (https://conda.io/docs/) is an open source package management and environment management system (provides virtual environment capabilities) that runs on Windows, macOS, and Linux. Conda is included in all versions of Anaconda and Miniconda. 

 

 

Note

Since working with Anaconda/Miniconda and Conda may be of interest to readers, a quick introduction is given in the next subsection, but it is not necessary to run the code examples included in this book.

 

Anaconda/Miniconda distributions and conda package–and environment-management system 


Conda (https://conda.io/docs/) is an open source package-management and environment-management system (provides virtual environment capabilities) that runs on many operating systems (for example, Windows, macOS, and Linux). Conda installs, runs, and updates packages and their dependencies. Conda can create, save, load, and switch between environments. 

Note

As conda is included in all versions of Anaconda and Miniconda, you should have already installed Anaconda or Miniconda.

Anaconda is a downloadable, free, open source, high-performance Python and R distribution. Anaconda comes with conda, conda build, Python, and more than 100 open source scientific packages and their dependencies. Using the conda install command, you can easily install popular open source packages for data science from the Anaconda repository. Miniconda is a small version of Anaconda, which includes only conda, Python, the packages they depend on, and a small number of other useful packages.

Installing Anaconda or Miniconda is easy. For the sake of simplicity, we are focusing on Anaconda. To install Anaconda, check the Acadonda installer for your operating system (https://www.anaconda.com/download/). Anaconda 5.2 can be installed in both Python 3.6 and Python 2.7 versions on Windows, macOS, and Linux:

After you have finished installing, in order to test the installation, in Terminal or Anaconda Prompt, run the following command:

$ conda list

For a successful installation, a list of installed packages appears. As mentioned, Anaconda (and Miniconda) comes with conda, which is a simple package manager similar to apt-get on Linux. In this way, we can install new packages in Terminal using the following command:

$ conda install packagename

Here, packagename is the actual name of the package we want to install. Existing packages can be updated using the following command:

$ conda update packagename

We can also search for packages using the following command:

$ anaconda search –t conda packagename

This will bring up a whole list of packages available through individual users. A package called packagename from a user called username can then be installed as follows:

$ conda install -c username packagename

Additionally, conda can be used to create and manage virtual environments. For example, creating a test environment and installing NumPy version 1.7 is as simple as typing the next command:

$ conda create --name test numpy=1.7

In a similar fashion as working with virtualenv, environments can be activated and deactivated. To do this on macOS and Linux, just run the following:

$ source activate test
 $ python
 ...
 $ source deactivate

On Windows, run the following:

$ activate test
 $ python
 ...
 $ deactivate

Note

See the conda cheat sheet PDF (1 MB) for a single-page summary of the most important information about using conda: https://conda.io/docs/_downloads/conda-cheatsheet.pdf.

Finally, it should be pointed out that we can work with conda under the PyCharm IDE, in a similar way as virtualenv to create and manage virtual environments, because PyCharm can work with both tools.

 

 

 

Packages for scientific computing, data science, machine learning, deep learning, and computer vision


So far, we have seen how to install Python, OpenCV, and a few other packages (numpy and matplotlib) from scratch, or using Anaconda distribution, which includes many popular data-science packages. In this way, some knowledge about the main packages for scientific computing, data science, machine learning, and computer vision is a key point because they offer powerful computational tools. Throughout this book, many Python packages will be used. Not all of the cited packages in this section will, but a comprehensive list is provided for the sake of completeness in order to show the potential of Python in topics related to the content of this book:

  • NumPy (http://www.numpy.org/) provides support for large, multi-dimensional arrays. NumPy is a key library in computer vision because images can be represented as multi-dimensional arrays. Representing images as NumPy arrays has many advantages.
  • OpenCV (https://opencv.org/) is an open source computer vision library.
  • Scikit-image (https://scikit-image.org/) is a collection of algorithms for image processing. Images manipulated by scikit-image are simply NumPy arrays.
  • The Python Imaging Library (PIL) (http://www.pythonware.com/products/pil/) is an image-processing library that provides powerful image-processing and graphics capabilities.
  • Pillow (https://pillow.readthedocs.io/) is the friendly PIL fork by Alex Clark and contributors. The PIL adds image-processing capabilities to your Python interpreter. 
  • SimpleCV (http://simplecv.org/) is a framework for computer vision that provides key functionalities to deal with image processing. 
  • Mahotas (https://mahotas.readthedocs.io/) is a set of functions for image processing and computer vision in Python. It was originally designed for bioimage informatics. However, it is useful in other areas as well. It is completely based on numpy arrays as its datatype. 
  • Ilastik (http://ilastik.org/) is a user-friendly and simple tool for interactive image segmentation, classification, and analysis. 
  • Scikit-learn (http://scikit-learn.org/) is a machine learning library that features various classification, regression, and clustering algorithms. 
  • SciPy (https://www.scipy.org/) is a library for scientific and technical computing. 
  • NLTK (https://www.nltk.org/) is a suite of libraries and programs to work with human-language data. 
  • spaCy (https://spacy.io/) is an open-source software library for advanced natural language processing in Python. 
  • LibROSA (https://librosa.github.io/librosa/) is a library for both music and audio processing. 
  • Pandas (https://pandas.pydata.org/) is a library (built on top of NumPy) that provides high-level data computation tools and easy-to-use data structures. 
  • Matplotlib (https://matplotlib.org/) is a plotting library that produces publication-quality figures in a variety of formats. 
  • Seaborn (https://seaborn.pydata.org/) is a graphics library that is built on top of Matplotlib. 
  • Orange (https://orange.biolab.si/) is an open source machine learning and data-visualization toolkit for novices and experts.
  • PyBrain (http://pybrain.org/) is a machine learning library that provides easy-to-use state-of-the-art algorithms for machine learning. 
  • Milk (http://luispedro.org/software/milk/) is a machine learning toolkit focused on supervised classification with several classifiers. 
  • TensorFlow (https://www.tensorflow.org/) is an open source machine learning and deep learning library.
  • PyTorch (https://pytorch.org/) is an open source machine learning and deep learning library. 
  • Theano (http://deeplearning.net/software/theano/) is a library for fast mathematical expressions, evaluation, and computation, which has been compiled to run on both CPU and GPU architectures (a key point for deep learning). 
  • Keras (https://keras.io/) is a high-level deep learning library that can run on top of TensorFlow, CNTK, Theano, or Microsoft Cognitive Toolkit.
  • Django (https://www.djangoproject.com/) is a Python-based free and open source web framework that encourages rapid development and clean, pragmatic design. 
  • Flask (http://flask.pocoo.org/) is a micro web framework written in Python based on Werkzeug and Jinja 2.

 

 

All these packages can be organized based on their main purpose:

  • To work with images: NumPy, OpenCV, scikit-image, PIL Pillow, SimpleCV, Mahotas, ilastik
  • To work in text: NLTK, spaCy, NumPy, scikit-learn, PyTorch
  • To work in audio: LibROSA
  • To solve machine learning problem: pandas, scikit-learn, Orange, PyBrain, Milk
  • To see data clearly: Matplotlib, Seaborn, scikit-learn, Orange
  • To use deep learning: TensorFlow, Pytorch, Theano, Keras
  • To do scientific computing: SciPy
  • To integrate web applications: Django, Flask

Note

Additional Python libraries and packages for AI and machine learning can be found at https://python.libhunt.com/packages/artificial-intelligence.

 

Jupyter Notebook


The Jupyter Notebook an open source web application that allows you to edit and run documents via a web browser. These documents, which are called Notebook documents (or notebooks), contain code (more than 40 programming languages, including Python, are supported) and rich text elements (paragraphs, equations, figures). The Jupyter Notebook can be executed on a local computer or can be installed on a remote server. You can start with notebooks, trying them online or installing the Jupyter Notebook.

Trying Jupiter Notebook online 

First, go to https://jupyter.org/try. You will see something like this:

To try Jupyter with Python online, click on the Python option, or paste this URL into your web browser: https://mybinder.org/v2/gh/ipython/ipython-in-depth/master?filepath=binder/Index.ipynb. Once the page has loaded, you can start coding/loading notebooks.

Installing the Jupyter Notebook

To install Jupyter, you can follow the main steps at http://jupyter.org/install.html. Installing Jupyter Notebook can also be done using Anaconda or using Python's package manager, pip.

Installing Jupyter using Anaconda

It is strongly recommended you install Python and Jupyter using the Anaconda Distribution, which includes Python, the Jupyter Notebook, and other commonly used packages for scientific computing and data science. To install Jupyter using Anaconda, download Anaconda (https://www.anaconda.com/distribution/) and install it. This way, you have installed Jupyter Notebook. To run the notebook, run the following command in Command Prompt (Windows) or Terminal (macOS/Linux):

$ jupyter notebook

 

 

 

 

Installing Jupyter with pip

You can also install Jupyter using Python's package manager, pip, by running the following commands:

$ python -m pip install --upgrade pip
$ python -m pip install jupyter

At this point, you can start the notebook server by running the following command:

$ jupyter notebook

The previous command will show you some key information in connection with the notebook server, including the URL of the web application (which by default is http://localhost:8888). It will then open your default web browser to this URL. To start a specific notebook, the following command should be used:

$ jupyter notebook notebook.ipynb

This was a quick introduction to notebooks. In the next chapters, we are going to create some notebooks, so you will have the opportunity to play with them and gain full knowledge of this useful tool.

 

The OpenCV and Python project structure


The project structure is the way you organize all the files inside a folder in a way that the project best accomplishes the objectives. We are going to start with a .py script (sampleproject.py) that should be with other files in order to complete the information about this script – dependencies, license, how to install it, or how to test it. A common approach for structuring this basic project is as follows:

sampleproject/
│
├── .gitignore
├── sampleproject.py
├── LICENSE
├── README.rst
├── requirements.txt
├── setup.py
└── tests.py

sampleproject.py—if your project is only a single Python source file, then put it into the directory and name it something related to your project.

 

The README (.rst or .md extension) is used to register the main properties of the project, which should cover, at least, the following:

  • What your project does 
  • How to install it
  • Example usage
  • How to set up the dev environment
  • How to ship a change
  • Change log
  • License and author info

A template you can use can be downloaded from this GitHub repository: https://github.com/dbader/readme-template. For further information, please see https://dbader.org/blog/write-a-great-readme-for-your-github-project.

The LICENSE.md document contains the applicable license. This is arguably the most important part of your repository, aside from the source code itself. The full license text and copyright claims should exist in this file. It is always a good idea to have one if you are distributing code. Typically, the GNU General Public License (GPL) (http://www.gnu.org/licenses/gpl.html) or the MIT license (https://opensource.org/licenses/MIT) are used in open source projects. You can check out http://choosealicense.com/ if you are not sure which license should be applied to your project.

A requirements.txt pip requirements file (https://pip.pypa.io/en/stable/user_guide/#requirements-files) should be placed at the root of the repository, which is used to specify the dependencies required to contribute to the project. The requirements.txt file can be generated using the following:

$ pip freeze > requirements.txt

To install these requirements, you can use the following command:

$ pip install -r requirements.txt

The setup.py file allows you to create packages you can redistribute. This script is meant to install your package on the end user's system, not to prepare the development environment as pip install -r < requirements.txt does. It is a key file because it defines information of your package (such as versioning, package requirements, and the project description).

The tests.py script contains the tests.

 

 

The .gitignore file tells Git what kind of files to ignore, such as IDE clutter or local configuration files. You can find sample .gitignore files for Python projects at https://github.com/github/gitignore.

 

Our first Python and OpenCV project


Based on the minimal project structure that was shown in the previous section, we are going to create our first Python and OpenCV project. This project has the following structure:

helloopencv/
│
├── images/
│
├── .gitignore
├── helloopencv.py
├── LICENSE
├── README.rst
├── requirements.txt
├── setup.py
└── helloopencvtests.py

README.rst (.rst extension) follows a basic structure, which was shown in the previous section. Python and ReStructuredText (RST) are deeply linked—RST is the format of docutils and sphinx (the de facto standard for documenting python code). RST is used both to document objects via docstrings, and to write additional documentation. If you go to the official Python documentation (https://docs.python.org/3/library/stdtypes.html), you can view the RST source of each page (https://raw.githubusercontent.com/python/cpython/3.6/Doc/library/stdtypes.rst). Using RST for the README.rst makes it directly compatible with the entire documentation setup. In fact, README.rst is often the cover page of a project's documentation. 

There are some RST editors you can use to help you write the README.rst. You can also use some online editors. For example, the online Sphinx editor is a good choice (https://livesphinx.herokuapp.com/).

The .gitignore file specifies intentionally untracked files that Git should ignore (https://git-scm.com/docs/gitignore). .gitignore tells git which files (or patterns) Git should ignore. It is usually used to avoid committing transient files from your working directory that are not useful to other collaborators, such as compilation products and temporary files that IDEs create. Open https://github.com/github/gitignore/blob/master/Python.gitignore to see a .gitignore file that you can include in your Python projects.

 

setup.py (see the previous section for a deeper description), it is a Python file that is usually shipped with libraries or programs, also written in Python. Its purpose is to correctly install the software. A very complete example of this file can be seen at https://github.com/pypa/sampleproject/blob/master/setup.py, which is full of comments to help you understand how to adapt it to your needs. This file is proposed by the Python Packaging Authority (PyPa) (https://www.pypa.io/en/latest/). One key point is in connection with the packages option, as we can read in the aforementioned setup.py file.

You can just specify package directories manually here if your project is simple. Or you can use find_packages(). Alternatively, if you just want to distribute a single Python file, use the py_modules argument instead as follows, which will expect a file called my_module.py to exist, py_modules=["my_module"]. 

Therefore, in our case, py_modules =["helloopencv"] is used.

Additionally, setup.py allows you to easily install Python packages. Often it is enough to write the following:

$ python setup.py install

Therefore, if we want to install this simple package, we can write the previous command, python setup.py install, inside the helloopencv folder. For example, in Windows, run the following command:

C:\...\helloopencv>python setup.py install

You should see something like this:

 running install
 ...
 ...
 Installed c:\python37\lib\site-packages\helloopencv-0.1-py3.7.egg
 Processing dependencies for helloopencv==0.1
 ...
 ...
 Finished processing dependencies for helloopencv==0.1

When finished, helloopencv is installed in our system (like any other Python package). You can also install helloopencv with pip, pip install ., inside the helloopencv folder. For example, in Windows, run the following command:

C:\...\helloopencv>pip install .

 

 

You should see something like this:

 Processing c:\...\helloopencv
 ...
 ...
 Successfully installed helloopencv-0.1

This indicates that helloopencv has been installed successfully. To use this package, we can write a Python file and import the helloopencv package. Alternatively, we can perform a quick use of this package by importing it directly from the Python interpreter. Following this second approach, you can open Command Prompt, import the package, and make use of it. First, open Command Prompt, then type python to run the interpreter:

C:\...\helloopencv>python
Python 3.7.0 (v3.7.0:1bf9cc5093, Jun 27 2018, 04:06:47) [MSC v.1914 32 bit (Intel)] on win32
Type "help", "copyright", "credits" or "license" for more information.
>>>

Once the interpreter is loaded, we can import the package:

>>> import helloopencv
helloopencv.py is being imported into another module
>>>

The helloopencv.py is being imported into another module output is a message from the helloopencv package (specifically from the helloopencv.py file) indicating that this file has been imported. Therefore, this message indicates that the module has been successfully imported. Once imported, we can make use of it. For example, we can call the show_message method:

>>> helloopencv.show_message()
'this function returns a message'
>>>

We can see that the result of calling this method is a message that is printed on the screen. This method is a simple way to know that everything has been installed correctly because it involves installing, importing, and making use of a function from the package. Furthermore, we can call a more useful method contained in the helloopencv package. You can, for example, call the load_image method to load an image from disk and afterwards, you can display it using the show_image method:

>>> image = helloopencv.load_image("C:/.../images/logo.png")
>>> helloopencv.show_image(image)

 

 

 

 

 

 

 

 

 

 

Here, the parameter of the load_image function is the path of an image from your computer. In this case, the logo.png image is loaded. After calling the show_image method, an image should be displayed. To close the window, a key must be pressed. Then you should be able to write in the interpreter again. To see all the methods that are available in the helloopencv package, you can open the helloopencv.py file with your favorite editor or IDE and have a look at it. In this Python file, you can see some methods that conform to our first Python project:

  • show_message(): This function prints the this function returns a message message.
  • load_image(): This function loads an image from its path.
  • show_image(): This function shows an image once it has been loaded.
  • convert_to_grayscale(): This function converts an image to grayscale once it has been loaded.
  • write_image_to_disk(): This function saves an image on disk.

All of these methods perform simple and basic operations. Most of them make use of the OpenCV library, which is imported at the beginning of this file (import cv2). Do not worry about the Python code contained in this file, because only basic operations and calls to the OpenCV library are performed. 

You can execute the helloopencv.py script without installing the package. To execute this file, you should run the python helloopencv.py command once Command Prompt is opened:

 C:\...\helloopencv>python helloopencv.py
 helloopencv.py is being run directly

After the execution of this file, the helloopencv.py is being run directly message will appear, which means that the file is executed directly and not imported from another module or package (or the Python interpreter). You can also see that an image is loaded and displayed. You can press any key to continue the execution. Once again, the grayscale version of the logo is displayed and any key should be pressed again to end the execution. The execution ends after the grayscale image is saved on the disk.

Lastly, the helloopencvtests.py file can be used for unit testing. Testing applications has become a standard skill for any competent developer. The Python community embraces testing, and the Python standard library has good built-in tools to support testing (https://docs.python.org/3/library/unittest.html).

 

 

In the Python ecosystem, there are a lot of testing tools. The two most common packages used for testing are nose (https://pypi.org/project/nose/) and pytest (https://pypi.org/project/pytest/). In this first Python project, we are going to use pytest for unit testing. 

To execute the test, run the py.test -s -v helloopencvtests.py command once Command Prompt is opened:

C:\...\helloopencv>py.test -s -v helloopencvtests.py
============================= test session starts =============================
 platform win32 -- Python 3.7.0, pytest-3.8.0, py-1.6.0, pluggy-0.7.1 -- c:\python37\python.exe
 cachedir: .pytest_cache
 collected 4 items
 helloopencvtests.py::test_show_message testing show_message
 PASSED
 helloopencvtests.py::test_load_image testing load_image
 PASSED
 helloopencvtests.py::test_write_image_to_disk testing
 write_image_to_disk
 PASSED
 helloopencvtests.py::test_convert_to_grayscale testing    test_convert_to_grayscale
 PASSED
========================== 4 passed in 0.57 seconds      ===========================

After the execution of the tests, you can see that four tests were executed. The PASSED message means that the tests were executed successfully. This is a quick introduction to unit testing in Python. Nevertheless, the full pytest documentation can be found at https://docs.pytest.org/en/latest/contents.html#toc. 

 

Summary


In this first chapter, we covered the main steps to set up OpenCV and Python to build your computer vision projects. At the beginning of this chapter, we quickly looked at the main concepts in this book – Artificial Intelligence, machine learning, neural networks, and deep learning. Then we explored the OpenCV library, including the history of the library and its main modules. As OpenCV and other packages can be installed in many operating systems and in different ways, we covered the main approaches.

 

 

Specifically, we saw how to install Python, OpenCV, and other packages globally or in a virtual environment. In connection with the installation of the packages, we introduced Anaconda/Miniconda and Conda, because we can also create and manage virtual environments. Additionally, Anaconda/Miniconda comes with many open source scientific packages, including SciPy and NumPy.

We explored the main packages for scientific computing, data science, machine learning, and computer vision, because they offer powerful computational tools. Then we discussed the Python-specific IDEs, including PyCharm (the de facto Python IDE environment). PyCharm (and other IDEs) can help us create virtual environments in a very intuitive way. We also looked at Jupyter Notebooks, because it can be a good tool for the readers of this book. In the next chapters, more Jupyter Notebooks will be created to give you a better understanding of this useful tool. Finally, we explored an OpenCV and Python project structure, covering the main files that should be included. Then we built our first Python and OpenCV sample project, where we saw the commands to build, run, and test this project.

In the next chapter, you will start to write your first scripts as you get better acquainted with the OpenCV library. You will see some basic concepts necessary to start coding your computer vision projects (for example, understanding main image concepts, the coordinate system in OpenCV, and accessing and manipulating pixels in OpenCV). 

 

Questions


  1. What is a virtual environment?
  2. What is the connection between pip, virtualenv, pipenv, Anaconda, and conda?
  3. What is the Jupyter Notebook?
  4. What are the main packages to work with computer vision in Python?
  5. What does pip install -r requirements.txt do?
  6. What is an IDE and why should you use one during the development of your projects?
  7. Under what license is OpenCV published?

 

 

 

Further reading


The following references will help you dive deeper into concepts presented in this chapter:

Check out these references (mainly books) for more information on concepts that will be presented in future chapters of the book. Keep this list handy; it will be really helpful:

About the Author

  • Alberto Fernández Villán

    Alberto Fernández Villán is a software engineer with more than 12 years of experience in developing innovative solutions. In the last couple of years, he has been working in various projects related to monitoring systems for industrial plants, applying both Internet of Things (IoT) and big data technologies. He has a Ph.D. in computer vision (2017), a deep learning certification (2018), and several publications in connection with computer vision and machine learning in journals such as Machine Vision and Applications, IEEE Transactions on Industrial Informatics, Sensors, IEEE Transactions on Industry Applications, IEEE Latin America Transactions, and more. As of 2013, he is a registered and active user (albertofernandez) on the Q&A OpenCV forum.

    Browse publications by this author

Latest Reviews

(5 reviews total)
Excelente livro.
very good overview of the possibilities of open-cv
Gute Übersicht zum Thema Zweckdienlich und verständlich

Recommended For You

Book Title
Unlock this full book with a FREE 10-day trial
Start Free Trial