OpenCV is a simple yet powerful tool for any computer vision enthusiast. One can learn computer vision in an easy way by writing OpenCV programs in Python. The Raspberry Pi family of single-board computers uses Python as the preferred development platform. Using a Raspberry Pi computer and Python for OpenCV programming is one of the best ways to start your journey into the world of computer vision. We will commence our journey with this chapter by getting ourselves familiar with the following topics:
Setting up Raspberry Pi
Installing OpenCV and its dependencies
Computer vision is an area of computer science, mathematics, and electrical engineering. It includes ways to acquire, process, analyze, and understand images and videos from the real world in order to mimic human vision. Also, unlike human vision, computer vision can also be used to analyze and process depth and infrared images.
Computer vision is also concerned with the theory of information extraction from images and videos. A computer vision system can accept different forms of data as an input, including, but not limited to, images, image sequences, and videos that can be streamed from multiple sources to further process and extract useful information from for decision making.
Don't worry about this jargon as of now. We will explore most of these concepts in detail in the later chapters.
OpenCV (Open Source ComputerVision) is a library of programming functions for computer vision. It was initially developed by the Intel Russia research center in Nizhny Novgorod, and it is currently maintained by Itseez.
You can read more about Itseez at http://itseez.com/.
This is a cross-platform library, which means that it can be implemented and operated on different operating systems. It focuses mainly on image and video processing. In addition to this, it has several GUI and event handling features for the user's convenience.
OpenCV was released under a Berkeley Software Distribution (BSD) license and hence, it is free for both academic and commercial use. It has interfaces for popular programming languages, such as C/C++, Python, and Java, and it runs on a variety of operating systems including Windows, Android, and Unix-like operating systems.
You can explore the OpenCV homepage, www.opencv.org, for further details.
OpenCV was initially an Intel Research initiative to develop tools to analyze images.
A single-board computer system is a complete computer on a single board. The board includes processor(s), RAM, I/O, and networking ports for interfacing devices. Unlike traditional computer systems, single-board computers are not modular and its hardware cannot be upgraded as it's integrated on the board itself. Single-board computers are used as low-cost computers in academic and research settings. The use of single-board computers in embedded systems is very prevalent, and many individuals and organizations have developed and released fully functional products based on single-board computers.
Popular single-board computers available in the market include, but are not limited to, Raspberry Pi, Banana Pi, BeagleBone, and Cubieboard.
Raspberry Pi is a series of low-cost, palm-sized, single-board computers developed by the Raspberry Pi Foundation in the UK. The intention behind the creation of Raspberry Pi was to promote the teaching of basic computer skills in schools, and the former serves this purpose well. Raspberry Pi has expanded its footprint well beyond its intended purpose by penetrating into the market of embedded systems and research.
Raspberry Pi modelsâA, A+, B, and B+âare based on SoC (system on a chip) Broadcom BCM2835, which includes an ARM11 700 MHz CPU (which can be overclocked). RPi generation 2 uses a quad core ARM Cortex-A7, the first multicore Raspberry Pi. Raspberry Pi A and B use SD cards for boot and persistent storage, whereas models A+, B+, and Pi 2 use microSD cards for the same. The models A and A+ have 256 MB of RAM, B and B+ have 512 MB of RAM, and Pi 2 has 1 GB of RAM.
Model A+ (currently in production and available for purchase)
Model B (available for purchase but not in production)
Model B+ (currently in production and available for purchase)
Raspberry Pi 2 (currently in production and available for purchase)
The Raspberry Pi Foundation provides Debian and Arch variants and Linux ARM distributions for download. Python is the main programming platform and languages like C, C++, Java, Perl, and Ruby can be used to program Raspberry Pi.
We will use Raspberry Pi B+ for our Computer Vision learning. However, these examples can also be executed on other Raspberry Pi Models.
700 MHz ARM1176JZF-S core (ARM11 family, ARM v6 instruction set)
Broadcom VideoCore IV @250 MHz
512 MB SDRAM (shared with GPUâthe limit of memory used by GPU can be set using
USB 2.0 ports
10/100 Mbit/s Fast Ethernet, no onboard Wi-Fi or Bluetooth
600 mA (3 W), 5 V via http://en.wikipedia.org/wiki/MicroUSB or GPIO header (using MicroUSB for power is recommended)
However, the recently launched Raspberry Pi 2 is based on ARM Cortex A7, which is capable of running both Windows 10 and Ubuntu (Snappy Core). The following operating systems are officially supported by all the models of Raspberry Pi and are available for download at the download page:
Pidora (Fedora Remix)
Raspbianâwe will use this with a Raspberry Pi B+ throughout this book.
Raspbian is an unofficial variant of Debian Wheezy armhf (ARM hard float) that is compiled for hard float code that will run on Raspberry Pi computers. It is a free operating system based on Debian that is optimized for the Raspberry Pi hardware. Raspbian is more than a pure OS. It comes with over 35,000 packages and precompiled software for Raspberry Pi.
To know more about Raspbian, you can visit http://www.raspbian.org/.
Raspberry Pi B+
A micro USB power supply
Considering that we are going for a bit of power-intensive usage for our Pi B+ (like connecting picamera), a 5V 2A power supply is recommended.
A standard USB keyboard
A microSD card and a microSD to SD card converter
We need a microSD card of minimum 4 GB.
A USB mouse
You can either use an HDMI monitor or a standard VGA monitor.
A monitor connection cable and a converter
If you are using an HDMI Monitor, then an HDMI cable is sufficient. If you are using a VGA monitor, then you need to use an HDMI to VGA converter with a VGA cable. A few special changes should be made to
/boot/config.txtif you're using a VGA monitor, which will be explained in the next section.
This is the original way of installing an OS into a microSD card, and many users, including me, still prefer it. It allows the SD card to be prepared manually before it is used, and it allows you to have easier access to the configuration files like
/boot/config.txt, which might have to be modified in a few cases before booting up the system. The default Raspbian image consists of only two partitionsâboot and system. These would fit into a 2 GB card. However, I recommend you to use a minimum 4 GB card to be on the safe side. Choosing an 8 GB card will be adequate for most of the applications.
Following are the instructions for the Windows users:
Download the installable file of Win32DiskImager that is available at http://sourceforge.net/projects/win32diskimager/files/latest/download and then install it.
Download the installable version of WinZip that is available at http://www.winzip.com/prod_down.html%20 and install it.
Go to http://www.raspberrypi.org/downloads and download the latest image of Raspbian. It will be a compressed file in the ZIP format, and it needs to be extracted.
Extract the ZIP file using WinZip. The extracted file will be in the
Insert your microSD card into the card reader and plug the card reader into your computer. Nowadays, many computers have an inbuilt SD card reader. In this case, you need to insert the microSD card into the microSD to SD card converter and insert that into your computer's inbuilt card reader.
Win32DiskImager.exefile and write the image into your SD card.
Browse the SD card. It will appear as a drive labeled
bootin the Windows file explorer. Open the
config.txtfile from the explorer. You will have to edit the file in the following manner to enable a proper display on your VGA monitor
Save the file.
By default, the commented options (which have
# at the beginning) are disabled. We will enable these options by uncommenting their respective lines by removing
# at the beginning of these commented lines.
If you are using Linux or Mac OS, then you will find the instructions to install the OS on your Micro SD card for these operating systems at https://www.raspberrypi.org/documentation/installation/installing-images/.
Insert the microSD card into the microSD card slot of Pi.
Connect the Pi to the HDMI monitor. In case you have connected the VGA monitor, connect it using the HDMI to VGA converter.
Connect the USB mouse and USB keyboard.
Connect the Pi to the power supply with a micro USB power cable. Make sure that the power is switched off at this point.
Check all the connections once and then switch on the power supply of Pi.
At this stage, your Pi will start booting up. You will see a green light blinking on the Pi board. It means that it's working! Now, there are a few more things that you need to do before you can really start using your Pi. Once it boots up, it will show the
raspi-config menu, as follows:
Use Expand Filesystem.
Go to Enable Boot to Desktop/Scratch | Desktop. Log in as
piat the graphical desktop.
If you do not enable this option, you will be asked for the username and password every time you boot. The default username is
piand the password is
raspberry. Once you enter the username and password, the command prompt will appear. The default shell of Raspbian is bash. You can confirm it by typing this in the following command:
You can always go to the graphical desktop by typing in the
startxcommand. To use OpenCV with Python, we are required to use the GUI of Raspbian to display images and video.
Navigate to Internationalisation Options | Change Keyboard Layout. Change it to US (the default is UK).
Navigate to Advanced Options | Memory Split and select 64 MB for GPU.
This option decides how much RAM is used by the Graphic Processor Unit (GPU). The more the RAM is allocated to the GPU, the more will the processing of intensive graphics be done. 64 MB is a good value for most graphics-related purposes.
sudo shutdown âh now
An alternative command is as follows:
You can reboot Pi by using the following command:
Connect your Pi to an Internet modem or router with an Ethernet cable.
Run the following command to restart the networking service:
sudo service networking restart
Make sure that Raspberry Pi is connected to the Internet by typing in the following command:
ping âc4 www.google.com
Run the following commands in a sequence:
Advanced Package Tool (apt) is the utility that can be used to install and remove software in Debian and its variants. We need to use it to update the Pi software.
sudo apt-get update
sudo apt-get upgrade
This command is used to upgrade the firmware. The kernel and firmware are installed as a Debian package, and hence, we will also get the updates. These packages are updated infrequently after extensive testing.
Now, we will need to install a few necessary packages and dependencies for OpenCV. Following is a list of packages we need to install. You just need to connect Pi to the Internet and type in
sudo apt-get install <package-name>, where
<package-name>is one of following packages:
For example, if you want to install x264, you have to type
sudo apt-get install x264. This will install the necessary package. Similarly, you can install all of the aforementioned packages in like manner. If a package is already installed on Pi, it will show the following message:
[email protected] ~ $ sudo apt-get install x264 Reading package lists... Done Building dependency tree Reading state information... Done x264 is already the newest version. 0 upgraded, 0 newly installed, 0 to remove and 0 not upgraded.
In such cases, don't worry. The package you wanted to install has already been installed, and it is up to date. Just proceed with the installation of all the other packages in the list one-by-one.
Finally, install OpenCV for Python by using the following command:
sudo apt-get install python-opencv
This is the easiest way to install OpenCV for Python. However, there is a problem with this. Raspbian repositories may not always contain the latest version of OpenCV. For example, at the time of writing this book, Raspbian repository contains version 2.4.1, while the latest OpenCV version is 2.4.10. With respect to Python API, the latest version will always contain much better support and more functionality.
It's very easy to code for OpenCV in Python. It requires very few lines of code compared to C/C++, and powerful libraries like NumPy can be exploited for multidimensional data structures that are required for image processing.
On a terminal, type
python, and then type the following lines:
>>> import cv2 >>> print cv2.__version__
This will show us the version of OpenCV that was installed on Pi, which, in our case is 2.4.1.
NumPy is a fundamental package that can be used to scientifically compute with Python. It is a matrix library for linear algebra. NumPy can also be used as an efficient multidimensional container of generic data. Arbitrary data types can be defined and used. NumPy is an extension of the Python programming language. It adds support for large multidimensional arrays and matrices, along with a large library of high-level mathematical functions that can be used to operate on these arrays. We will use NumPy arrays throughout this book to represent images and carry out complex mathematical operations. NumPy comes with many inbuilt functions for all of these operations. So, we do not have to worry about basic array operations. We can directly focus on the concepts and code for computer vision. All the OpenCV array structures are converted to and from NumPy arrays. So, whatever operations you perform in NumPy, you can combine NumPy with OpenCV.
We will use NumPy with OpenCV a lot in this book. Let's start with some simple example programs that will demonstrate the real power of NumPy.
Let's see some examples on array creation. The
array() method is used very frequently in this book. There are many ways to create different types of arrays. We will explore these ways as and when they are needed in this book. Follow these commands for array creation:
>>> import numpy as np >>> x=np.array([1,2,3]) >>> x array([1, 2, 3]) >>> y=arange(10) >>> y array([0, 1, 2, 3, 4, 5, 6, 7, 8, 9])
We are going to learn about the
linspace() function now. It takes three parametersâ
count. This creates an array with equally spaced points, starting from
start_num and ending with
end_num. You can try out the following example:
>>> a=np.array([1,3,6,9]) >>> b=np.linspace(0,15,4) >>> c=a-b >>> c array([ 1., -2., -4., -6.])
Following is the code that can be used to calculate the square of every element in an array:
>>> a**2 array([ 1, 9, 36, 81])
Let's explore some examples with regard to linear algebra. You will learn how to use the
dot() functions, which are useful while performing operations related to linear algebra:
>>> a=np.array([[1,2,3],[4,5,6],[7,8,9]]) >>> a.transpose() array([[1, 4, 7], [2, 5, 8], [3, 6, 9]]) >>> np.linalg.inv(a) array([[ -4.50359963e+15, 9.00719925e+15, -4.50359963e+15], [ 9.00719925e+15, -1.80143985e+16, 9.00719925e+15], [ -4.50359963e+15, 9.00719925e+15, -4.50359963e+15]]) >>> b=np.array([3,2,1]) >>> np.linalg.solve(a,b) array([ -9.66666667, 15.33333333, -6. ]) >>> c=np.random.rand(3,3) >>> c array([[ 0.69551123, 0.18417943, 0.0298238 ], [ 0.11574883, 0.39692914, 0.93640691], [ 0.36908272, 0.53802672, 0.2333465 ]]) >>> np.dot(a,c) array([[ 2.03425705, 2.59211786, 2.60267713], [ 5.57528539, 5.94952371, 6.20140877], [ 9.11631372, 9.30692956, 9.80014041]])
You can explore NumPy in detail at http://www.numpy.org/.
In this chapter, we learned about the background of OpenCV, Raspberry Pi, and computer vision. We learned how to set up Raspberry Pi to program computer vision with OpenCV. We also went through some examples on NumPy.
In the next chapter, we will learn how to work with images, videos, webcam, and the Pi camera.