Raspberry Pi Computer Vision Programming

4 (6 reviews total)
By Ashwin Pajankar
  • Instant online access to over 7,500+ books and videos
  • Constantly updated with 100+ new titles each month
  • Breadth and depth in over 1,000+ technologies
  1. Introduction to Computer Vision and Raspberry Pi

About this book

This book will provide you with the skills you need to successfully design and implement your own Raspberry Pi and Python-based computer vision projects.

From the beginning, this book will cover how to set up your Raspberry Pi for computer vision applications, exploring the basics of OpenCV, and how to design and implement real-life computer vision applications on your own. By sequentially working through the steps in each chapter, you will quickly be able to master the features of OpenCV. In the end of the book, you will also be introduced to SimpleCV, which is another powerful computer vision library for Python. Featuring plenty of coding examples and exercises, this book offers you an unparalleled learning experience.

Publication date:
May 2015
Publisher
Packt
Pages
178
ISBN
9781784398286

 

Chapter 1. Introduction to Computer Vision and Raspberry Pi

OpenCV is a simple yet powerful tool for any computer vision enthusiast. One can learn computer vision in an easy way by writing OpenCV programs in Python. The Raspberry Pi family of single-board computers uses Python as the preferred development platform. Using a Raspberry Pi computer and Python for OpenCV programming is one of the best ways to start your journey into the world of computer vision. We will commence our journey with this chapter by getting ourselves familiar with the following topics:

  • Computer vision

  • OpenCV

  • Raspberry Pi

  • Setting up Raspberry Pi

  • Installing OpenCV and its dependencies

  • NumPy basics

 

Computer vision


Computer vision is an area of computer science, mathematics, and electrical engineering. It includes ways to acquire, process, analyze, and understand images and videos from the real world in order to mimic human vision. Also, unlike human vision, computer vision can also be used to analyze and process depth and infrared images.

Computer vision is also concerned with the theory of information extraction from images and videos. A computer vision system can accept different forms of data as an input, including, but not limited to, images, image sequences, and videos that can be streamed from multiple sources to further process and extract useful information from for decision making.

Artificial intelligence and computer vision share many topics, such as image processing, pattern recognition, and machine learning techniques, as shown in the following diagram:

The typical tasks of computer vision include the following:

  • Object recognition and classification

  • Motion detection and analysis

  • Image and scene reconstruction

Don't worry about this jargon as of now. We will explore most of these concepts in detail in the later chapters.

 

OpenCV


OpenCV (Open Source ComputerVision) is a library of programming functions for computer vision. It was initially developed by the Intel Russia research center in Nizhny Novgorod, and it is currently maintained by Itseez.

Note

You can read more about Itseez at http://itseez.com/.

This is a cross-platform library, which means that it can be implemented and operated on different operating systems. It focuses mainly on image and video processing. In addition to this, it has several GUI and event handling features for the user's convenience.

OpenCV was released under a Berkeley Software Distribution (BSD) license and hence, it is free for both academic and commercial use. It has interfaces for popular programming languages, such as C/C++, Python, and Java, and it runs on a variety of operating systems including Windows, Android, and Unix-like operating systems.

Note

You can explore the OpenCV homepage, www.opencv.org, for further details.

OpenCV was initially an Intel Research initiative to develop tools to analyze images.

Following is the timeline of OpenCV in brief:

In August 2012, support for OpenCV was taken over by a nonprofit foundation, OpenCV.org, which is currently developing it further. It also maintains a developer and user site for OpenCV.

Note

At the time of writing this book, the stable version of OpenCV was 2.4.10. Version 3.0 Beta is also available.

 

Single-board computers and the Raspberry Pi


A single-board computer system is a complete computer on a single board. The board includes processor(s), RAM, I/O, and networking ports for interfacing devices. Unlike traditional computer systems, single-board computers are not modular and its hardware cannot be upgraded as it's integrated on the board itself. Single-board computers are used as low-cost computers in academic and research settings. The use of single-board computers in embedded systems is very prevalent, and many individuals and organizations have developed and released fully functional products based on single-board computers.

Popular single-board computers available in the market include, but are not limited to, Raspberry Pi, Banana Pi, BeagleBone, and Cubieboard.

Raspberry Pi

Raspberry Pi is a series of low-cost, palm-sized, single-board computers developed by the Raspberry Pi Foundation in the UK. The intention behind the creation of Raspberry Pi was to promote the teaching of basic computer skills in schools, and the former serves this purpose well. Raspberry Pi has expanded its footprint well beyond its intended purpose by penetrating into the market of embedded systems and research.

Note

The homepage of the Raspberry Pi Foundation is:

http://www.raspberrypi.org

Raspberry Pi models—A, A+, B, and B+—are based on SoC (system on a chip) Broadcom BCM2835, which includes an ARM11 700 MHz CPU (which can be overclocked). RPi generation 2 uses a quad core ARM Cortex-A7, the first multicore Raspberry Pi. Raspberry Pi A and B use SD cards for boot and persistent storage, whereas models A+, B+, and Pi 2 use microSD cards for the same. The models A and A+ have 256 MB of RAM, B and B+ have 512 MB of RAM, and Pi 2 has 1 GB of RAM.

As of now, there are five major models of Raspberry Pi, which are as follows:

  • Model A

  • Model A+ (currently in production and available for purchase)

  • Model B (available for purchase but not in production)

  • Model B+ (currently in production and available for purchase)

  • Raspberry Pi 2 (currently in production and available for purchase)

Note

Check out the product page of Raspberry Pi at the following location:

http://www.raspberrypi.org/products/

The Raspberry Pi Foundation provides Debian and Arch variants and Linux ARM distributions for download. Python is the main programming platform and languages like C, C++, Java, Perl, and Ruby can be used to program Raspberry Pi.

We will use Raspberry Pi B+ for our Computer Vision learning. However, these examples can also be executed on other Raspberry Pi Models.

The Raspberry Pi B+ specifications are as follows:

Component

Specification

CPU

700 MHz ARM1176JZF-S core (ARM11 family, ARM v6 instruction set)

GPU

Broadcom VideoCore IV @250 MHz

Memory

512 MB SDRAM (shared with GPU—the limit of memory used by GPU can be set using raspi-config utility)

USB 2.0 ports

4

Video output

HDMI, composite video (PAL and NTSC) via 3.5 mm TRRS jack shared with audio out (you need to use converters for VGA type displays)

Audio output

Analog via 3.5 mm phone jack; digital via HDMI port

Onboard storage

microSD

Networking

10/100 Mbit/s Fast Ethernet, no onboard Wi-Fi or Bluetooth

Power

600 mA (3 W), 5 V via http://en.wikipedia.org/wiki/MicroUSB or GPIO header (using MicroUSB for power is recommended)

Operating systems

Raspberry Pi primarily uses Unix-like, Linux-kernel-based operating systems, like the variants of Debian and Fedora.

The Raspberry Pi models A, A+, B, and B+ are based on the ARM11 family chip, which runs on the ARM v6 instruction set. The ARM v6 instruction set does not support Ubuntu and Windows.

However, the recently launched Raspberry Pi 2 is based on ARM Cortex A7, which is capable of running both Windows 10 and Ubuntu (Snappy Core). The following operating systems are officially supported by all the models of Raspberry Pi and are available for download at the download page:

  • OpenELEC

  • Pidora (Fedora Remix)

  • RASPBMC

  • RISC OS

  • Raspbian—we will use this with a Raspberry Pi B+ throughout this book.

Note

Windows 10 and Ubuntu are supported by only the recently launched Pi 2.

Raspbian

Raspbian is an unofficial variant of Debian Wheezy armhf (ARM hard float) that is compiled for hard float code that will run on Raspberry Pi computers. It is a free operating system based on Debian that is optimized for the Raspberry Pi hardware. Raspbian is more than a pure OS. It comes with over 35,000 packages and precompiled software for Raspberry Pi.

Note

To know more about Raspbian, you can visit http://www.raspbian.org/.

 

Setting up your Raspberry Pi B+


This is the front view of Raspberry Pi B+:

This is the flipside of Raspberry Pi B+:

We need the following hardware to set up a Pi:

  • Raspberry Pi B+

  • A micro USB power supply

    Considering that we are going for a bit of power-intensive usage for our Pi B+ (like connecting picamera), a 5V 2A power supply is recommended.

  • A standard USB keyboard

  • A microSD card and a microSD to SD card converter

    We need a microSD card of minimum 4 GB.

  • A USB mouse

  • A monitor

    You can either use an HDMI monitor or a standard VGA monitor.

  • A monitor connection cable and a converter

    If you are using an HDMI Monitor, then an HDMI cable is sufficient. If you are using a VGA monitor, then you need to use an HDMI to VGA converter with a VGA cable. A few special changes should be made to /boot/config.txt if you're using a VGA monitor, which will be explained in the next section.

  • A Windows computer with a microSD card reader and a wired Internet connection

Preparing your microSD card manually

This is the original way of installing an OS into a microSD card, and many users, including me, still prefer it. It allows the SD card to be prepared manually before it is used, and it allows you to have easier access to the configuration files like /boot/config.txt, which might have to be modified in a few cases before booting up the system. The default Raspbian image consists of only two partitions—boot and system. These would fit into a 2 GB card. However, I recommend you to use a minimum 4 GB card to be on the safe side. Choosing an 8 GB card will be adequate for most of the applications.

Following are the instructions for the Windows users:

  1. Download the installable file of Win32DiskImager that is available at http://sourceforge.net/projects/win32diskimager/files/latest/download and then install it.

  2. Download the installable version of WinZip that is available at http://www.winzip.com/prod_down.html%20 and install it.

  3. Go to http://www.raspberrypi.org/downloads and download the latest image of Raspbian. It will be a compressed file in the ZIP format, and it needs to be extracted.

  4. Extract the ZIP file using WinZip. The extracted file will be in the img format.

  5. Insert your microSD card into the card reader and plug the card reader into your computer. Nowadays, many computers have an inbuilt SD card reader. In this case, you need to insert the microSD card into the microSD to SD card converter and insert that into your computer's inbuilt card reader.

  6. Run the Win32DiskImager.exe file and write the image into your SD card.

  7. If you have an HDMI monitor, then please skip this step. This additional step is required only if you are planning to use a VGA monitor in place of an HDMI monitor.

  8. Browse the SD card. It will appear as a drive labeled boot in the Windows file explorer. Open the config.txt file from the explorer. You will have to edit the file in the following manner to enable a proper display on your VGA monitor

    • Change #disable_overscan=1 to disable_overscan=1

    • Change #hdmi_force_hotplug=1 to hdmi_force_hotplug=1

    • Change #hdmi_group=1 to hdmi_group=2

    • Change #hdmi_mode=1 to hdmi_mode=16

    • Change #hdmi_drive=2 to hdmi_drive=2

    • Change #config_hdmi_boost=4 to config_hdmi_boost=4

  9. Save the file.

By default, the commented options (which have # at the beginning) are disabled. We will enable these options by uncommenting their respective lines by removing # at the beginning of these commented lines.

Note

If you are using Linux or Mac OS, then you will find the instructions to install the OS on your Micro SD card for these operating systems at https://www.raspberrypi.org/documentation/installation/installing-images/.

Booting up your Raspberry Pi for the first time

Let's boot up our Pi for the first time with the microSD card in the following way:

  1. Insert the microSD card into the microSD card slot of Pi.

  2. Connect the Pi to the HDMI monitor. In case you have connected the VGA monitor, connect it using the HDMI to VGA converter.

  3. Connect the USB mouse and USB keyboard.

  4. Connect the Pi to the power supply with a micro USB power cable. Make sure that the power is switched off at this point.

  5. Check all the connections once and then switch on the power supply of Pi.

At this stage, your Pi will start booting up. You will see a green light blinking on the Pi board. It means that it's working! Now, there are a few more things that you need to do before you can really start using your Pi. Once it boots up, it will show the raspi-config menu, as follows:

Perform the following steps and reboot the Pi at the end:

Note

You will have to use the arrow keys and the Enter key to select options in the text-based menu.

  1. Use Expand Filesystem.

  2. Go to Enable Boot to Desktop/Scratch | Desktop. Log in as pi at the graphical desktop.

    Note

    If you do not enable this option, you will be asked for the username and password every time you boot. The default username is pi and the password is raspberry. Once you enter the username and password, the command prompt will appear. The default shell of Raspbian is bash. You can confirm it by typing this in the following command:

    echo $SHELL
    

    You can always go to the graphical desktop by typing in the startx command. To use OpenCV with Python, we are required to use the GUI of Raspbian to display images and video.

  3. Navigate to Internationalisation Options | Change Keyboard Layout. Change it to US (the default is UK).

  4. Enable Camera.

  5. Navigate to Advanced Options | Memory Split and select 64 MB for GPU.

This option decides how much RAM is used by the Graphic Processor Unit (GPU). The more the RAM is allocated to the GPU, the more will the processing of intensive graphics be done. 64 MB is a good value for most graphics-related purposes.

You can always invoke this tool from the command prompt with the following command and change the settings:

sudo raspi-config

Shutting down and rebooting your Pi safely

In the Raspbian GUI, there are options that allow you to shut down and reboot Pi. From the command prompt, you can shut down Pi safely by issuing the following command:

sudo shutdown –h now

An alternative command is as follows:

sudo halt

You can reboot Pi by using the following command:

sudo reboot
 

Preparing your Pi for computer vision


Now, we have a working Pi running the Raspbian OS. Please make sure that you have a working wired Internet connection with a reasonable speed for this activity. Let's prepare Pi for computer vision:

  1. Connect your Pi to an Internet modem or router with an Ethernet cable.

  2. Run the following command to restart the networking service:

    sudo service networking restart
    
  3. Make sure that Raspberry Pi is connected to the Internet by typing in the following command:

    ping –c4 www.google.com
    
  4. Run the following commands in a sequence:

    Advanced Package Tool (apt) is the utility that can be used to install and remove software in Debian and its variants. We need to use it to update the Pi software.

    • sudo apt-get update

    This command synchronizes the package list from the source. Indexes of all the packages are refreshed. This command must be issued before we issue the upgrade command.

    • sudo apt-get upgrade

    This will install the newest versions of the already installed software. Obsolete packages/utilities are not removed automatically. If the software is up to date, then it's left as it is.

    • sudo rpi-update

    This command is used to upgrade the firmware. The kernel and firmware are installed as a Debian package, and hence, we will also get the updates. These packages are updated infrequently after extensive testing.

  5. Now, we will need to install a few necessary packages and dependencies for OpenCV. Following is a list of packages we need to install. You just need to connect Pi to the Internet and type in sudo apt-get install <package-name>, where <package-name> is one of following packages:

    libopencv-dev

    libpng3

    libdc1394-22-dev

    build-essential

    libpnglite-dev

    libdc1394-22

    libavformat-dev

    zlib1g-dbg

    libdc1394-utils

    x264

    zlib1g

    libv4l-0

    v4l-utils

    zlib1g-dev

    libv4l-dev

    ffmpeg

    pngtools

    libpython2.6

    libcv2.3

    libtiff4-dev

    python-dev

    libcvaux2.3

    libtiff4

    python2.6-dev

    libhighgui2.3

    libtiffxx0c2

    libgtk2.0-dev

    python-opencv

    libtiff-tools

    libpngwriter0-dev

    opencv-doc

    libjpeg8

    libpngwriter0c2

    libcv-dev

    libjpeg8-dev

    libswscale-dev

    libcvaux-dev

    libjpeg8-dbg

    libjpeg-dev

    libhighgui-dev

    libavcodec-dev

    libwebp-dev

    python-numpy

    libavcodec53

    libpng-dev

    python-scipy

    libavformat53

    libtiff5-dev

    python-matplotlib

    libgstreamer0.10-0-dbg

    libjasper-dev

    python-pandas

    libgstreamer0.10-0

    libopenexr-dev

    python-nose

    libgstreamer0.10-dev

    libgdal-dev

    v4l-utils

    libxine1-ffmpeg

    python-tk

    libgtkglext1-dev

    libxine-dev

    python3-dev

    libpng12-0

    libxine1-bin

    python3-tk

    libpng12-dev

    libunicap2

    python3-numpy

    libpng++-dev

    libunicap2-dev

    libeigen3-dev

    For example, if you want to install x264, you have to type sudo apt-get install x264. This will install the necessary package. Similarly, you can install all of the aforementioned packages in like manner. If a package is already installed on Pi, it will show the following message:

    [email protected] ~ $ sudo apt-get install x264
    Reading package lists... Done
    Building dependency tree       
    Reading state information... Done
    x264 is already the newest version.
    0 upgraded, 0 newly installed, 0 to remove and 0 not upgraded.

    In such cases, don't worry. The package you wanted to install has already been installed, and it is up to date. Just proceed with the installation of all the other packages in the list one-by-one.

  6. Finally, install OpenCV for Python by using the following command:

    sudo apt-get install python-opencv
    

    This is the easiest way to install OpenCV for Python. However, there is a problem with this. Raspbian repositories may not always contain the latest version of OpenCV. For example, at the time of writing this book, Raspbian repository contains version 2.4.1, while the latest OpenCV version is 2.4.10. With respect to Python API, the latest version will always contain much better support and more functionality.

    Another method is to compile OpenCV from the source, which I will not recommend for beginners as it's a bit complex and it will take a lot of time.

Testing OpenCV installation with Python

It's very easy to code for OpenCV in Python. It requires very few lines of code compared to C/C++, and powerful libraries like NumPy can be exploited for multidimensional data structures that are required for image processing.

On a terminal, type python, and then type the following lines:

>>> import cv2
>>> print cv2.__version__

This will show us the version of OpenCV that was installed on Pi, which, in our case is 2.4.1.

 

NumPy


NumPy is a fundamental package that can be used to scientifically compute with Python. It is a matrix library for linear algebra. NumPy can also be used as an efficient multidimensional container of generic data. Arbitrary data types can be defined and used. NumPy is an extension of the Python programming language. It adds support for large multidimensional arrays and matrices, along with a large library of high-level mathematical functions that can be used to operate on these arrays. We will use NumPy arrays throughout this book to represent images and carry out complex mathematical operations. NumPy comes with many inbuilt functions for all of these operations. So, we do not have to worry about basic array operations. We can directly focus on the concepts and code for computer vision. All the OpenCV array structures are converted to and from NumPy arrays. So, whatever operations you perform in NumPy, you can combine NumPy with OpenCV.

We will use NumPy with OpenCV a lot in this book. Let's start with some simple example programs that will demonstrate the real power of NumPy.

Open Python via the terminal. Try the following examples.

Array creation

Let's see some examples on array creation. The array() method is used very frequently in this book. There are many ways to create different types of arrays. We will explore these ways as and when they are needed in this book. Follow these commands for array creation:

>>> import numpy as np
>>> x=np.array([1,2,3])
>>> x
array([1, 2, 3])

>>> y=arange(10)
>>> y
array([0, 1, 2, 3, 4, 5, 6, 7, 8, 9])

Basic operations on arrays

We are going to learn about the linspace() function now. It takes three parameters—start_num, end_num, and count. This creates an array with equally spaced points, starting from start_num and ending with end_num. You can try out the following example:

>>> a=np.array([1,3,6,9])
>>> b=np.linspace(0,15,4)
>>> c=a-b
>>> c
array([ 1., -2., -4., -6.])

Following is the code that can be used to calculate the square of every element in an array:

>>> a**2
array([ 1,  9, 36, 81])

Linear algebra

Let's explore some examples with regard to linear algebra. You will learn how to use the transpose(), inv(), solve(), and dot() functions, which are useful while performing operations related to linear algebra:

>>> a=np.array([[1,2,3],[4,5,6],[7,8,9]])
>>> a.transpose()
array([[1, 4, 7],
       [2, 5, 8],
       [3, 6, 9]])

>>> np.linalg.inv(a)
array([[ -4.50359963e+15,   9.00719925e+15,  -4.50359963e+15],
       [  9.00719925e+15,  -1.80143985e+16,   9.00719925e+15],
       [ -4.50359963e+15,   9.00719925e+15,  -4.50359963e+15]])

>>> b=np.array([3,2,1])
>>> np.linalg.solve(a,b)
array([ -9.66666667,  15.33333333,  -6.        ])

>>> c=np.random.rand(3,3)
>>> c
array([[ 0.69551123,  0.18417943,  0.0298238 ],
       [ 0.11574883,  0.39692914,  0.93640691],
       [ 0.36908272,  0.53802672,  0.2333465 ]])
>>> np.dot(a,c)
array([[ 2.03425705,  2.59211786,  2.60267713],
       [ 5.57528539,  5.94952371,  6.20140877],
       [ 9.11631372,  9.30692956,  9.80014041]])

Note

You can explore NumPy in detail at http://www.numpy.org/.

 

Summary


In this chapter, we learned about the background of OpenCV, Raspberry Pi, and computer vision. We learned how to set up Raspberry Pi to program computer vision with OpenCV. We also went through some examples on NumPy.

In the next chapter, we will learn how to work with images, videos, webcam, and the Pi camera.

About the Author

  • Ashwin Pajankar

    Ashwin Pajankar is a software professional and IoT enthusiast with more than 5 years' experience in software design, development, testing, and automation.

    He graduated from IIIT Hyderabad, earning an M.Tech in computer science and engineering. He holds multiple professional certifications from Oracle, IBM, Teradata, and ISTQB in development, databases, and testing. He has won several awards in college through outreach initiatives, at work for technical achievements, and community service through corporate social responsibility programs.

    He was introduced to Raspberry Pi while organizing a hackathon at his workplace, and he's been hooked on Pi ever since. He writes plenty of code in C, Bash, Python, and Java on his cluster of Pis. He's already authored one book on Raspberry Pi and reviewed three other titles related to Python for Packt Publishing.

    His LinkedIn Profile is at https://www.linkedin.com/in/ashwinpajankar/.

    Browse publications by this author

Latest Reviews

(6 reviews total)
I bought this book on a whim. If you have no previous exposure to opencv, this book is a quick jumpstart and gives a rundown of the common features. For me, there weren't concrete examples of applications that you could work from - Packt has other books on opencv that have a number of actual projects that one could try the adapt to your own needs.
Good
Useful starting guide to OpenCV and Raspberry Pi. I would recommend it to friends.

Recommended For You

Book Title
Unlock this full book FREE 10 day trial
Start Free Trial