OpenCV is a simple and powerful programming framework for computer vision. It is preferred by both novices and experts in the field of computer vision. We can easily learn computer vision by writing OpenCV programs using Python 3 as the programming language. The Raspberry Pi family of single-board computers uses Python as its preferred development language. Using a Raspberry Pi board and Python 3 for learning OpenCV programming is one of the best approaches that we can follow to commence our wonderful journey into the amazing field of computer vision programming. In this chapter, you will become familiar with all of the important concepts that you need in order to get started with the Raspberry Pi and computer vision. By the end of this chapter, you will be able to set up the Raspbian Operating System (OS) on various Raspberry Pi board models. You will also learn how to connect the boards to the internet.
In this chapter, we will cover the following topics:
- Understanding computer vision
- Single-board computers
- The Raspberry Pi family of single-board computers
- Setting up the Raspbian OS on a Raspberry Pi
- Connecting various Pi board models to the internet with LAN or Wi-Fi
By the end of this chapter, you will be able to set up your own Raspberry Pi board.
Understanding computer vision
The field of computer vision is a combination of different fields, including (but not limited to) computer science, mathematics, and electrical engineering. It includes ways to capture, process, and analyze images and videos from the real world in order to assist in decision making. Computer vision means mimicking biological (that is, human and non-human) vision. The end goal of most computer vision systems is to extract useful information from still images and videos (including prerecorded videos and live feeds) for the purpose of decision making. Biological vision systems work in a similar fashion. Additionally, unlike biological vision, computer vision can also acquire and work with images from the visual spectrum that are not visible to biological entities, for example, infrared and depth images.
Computer vision also relates to the area of extracting information from captured images and videos. A computer vision system may accept various types of data, such as images, videos, and live video streams, as inputs to further process, analyze, and extract meaningful information for the purpose of making important decisions.
The fields of artificial intelligence, machine vision, and computer vision overlap and share many topics, such as image processing, pattern recognition, and machine learning, as depicted in the following diagram:
In order to work as a researcher in the area of computer vision, you need to have a solid background and understanding of mathematics. However, to write programs for computer vision using OpenCV and Python 3, you don't need to know a lot of mathematics. Note that, in this book, you will be learning all of the mathematical and theoretical concepts required to get started with image processing and computer vision.
- The recognition of objects, the classification of visual detection, and an analysis of motion
- The reconstruction of scenes using images
- Image denoising and restoration
Do not get stressed if you are unfamiliar with these key terms. We will explore and implement many of these concepts throughout our journey.
OpenCV (also known as Open Source Computer Vision) is an open source library for computer vision and machine learning. It has many functionalities for image processing and computer vision. It is a cross-platform library, and it works with many programming languages and OSes. It has a large collection of computer vision and machine learning-related functions. It also has several Graphical User Interface (GUI) and event handling features.
OpenCV is free for academic and commercial usage as it is under the Berkley Software Distribution (BSD) license. It is written with the C++ programming language. It has interfaces for most of the popular programming languages, including (but not limited to) C/C++, Python, and Java. It runs on a variety of OSes, including Windows, Android, Linux, macOS, and other Unix-like OSes. In this book, we will write computer vision-related programs with OpenCV and Python 3.
The library has more than 2,500 optimized algorithms for machine learning and computer vision tasks. It has a community of more than 47,000 computer vision professionals, and it has been downloaded more than 18 million times. OpenCV is extensively used in academics for teaching, research organizations, government organizations, and various industry segments. Reputed able organizations such as Google, Yahoo, Microsoft, Intel, IBM, Sony, Honda, and Toyota all use OpenCV.
Let's take a look at the history of OpenCV. OpenCV was originally an in-house initiative of Intel Research and was used to develop a framework to work with images and videos. It was initially supported by Willow Garage and then Itseez.
You can visit the website of Willow Garage at http://www.willowgarage.com/.
In August 2012, the responsibility for further development and support for OpenCV was assumed by an independent, not-for-profit, organization, OpenCV.org. It maintains the website for OpenCV. In May 2016, Intel acquired Itseez. The following URLs have the press announcement from Intel and OpenCV.org:
Here's a brief timeline of the developments related to OpenCV:
You can find all the details, including different versions and press releases of the OpenCV library, at https://opencv.org/.
As we will be writing computer vision programs with Raspberry Pi as the platform, we will study single-board computers and Raspberry Pi in detail. We will learn how to set up the Raspbian OS on various models of a Raspberry Pi single-board computer.
A single-board computer (abbreviated to SBC) is a complete computer system on a single printed circuit board (abbreviated to PCB). The board usually has a processor(s), RAM, input/output (I/O), an Ethernet port for networking, and USB ports for interfacing with USB devices. A few single-board computers have Wi-Fi and Bluetooth, too. SBCs run OS distributions such as Ubuntu, Windows, Debian, and more. These OS distributions have specially tailored versions for use with these SBCs.
Unlike traditional computers, an SBC is not modular and its hardware cannot be upgraded because all the components (such as the CPU, RAM, GPU, and interfacing ports) are integrated on a single PCB itself. SBCs are used as low-cost computers in academia, research, and various other industries. The use of SBCs in embedded systems is quite widespread, and many individuals, research organizations, and companies have developed and released fully functional and usable products based on SBCs. Many of these products are crowdfunded. The main advantage of SBCs is onboard General-Purpose Input/Output (GPIO) pins. These pins provide functionalities such as various buses (Serial Peripheral Interface (SPI), I2C, and SMBus), digital I/O, analog input, and Pulse Width Modulation (PWM) output. Try not to get overwhelmed with all of this technical vocabulary. We will learn most of these concepts in more detail with the help of experiments. Almost all of the popular SBCs have GPIO in some form or other. Due to their small form factor and onboard GPIO, they are popular in schools, universities, training centers, boot camps, and maker spaces. They are frequently used in the areas of sensor networks and the internet of things (IoT).
- Low cost
- Small size
- Low power consumption
- Provision for onboard networking and I/O
However, SBCs come with their own set of disadvantages. As all the components of an SBC are on the same PCB, it can be very difficult to repair if a component is damaged due to mechanical or electrical reasons. For the same reason, we cannot even upgrade anything on an SBC. These are the only major disadvantages of SBCs.
The Microcomputer Trainer MMD-1, designed by John Titus in 1976, is the first true single-board microcomputer that was based on the Intel microprocessor, C8080A. It was called dyna-micro during the prototyping phase, and the production units were called MMD-1 (short for Mini-Micro Designer 1).
We are now going to take a look at the Raspberry Pi series in detail. However, before that, we will become acquainted with other popular SBC families.
The Beagleboard family
The BeagleBoard.org Foundation is an organization based in the USA. It is a non-profit entity, and its objective is to provide education and collaboration around the design, development, testing, and use of open source hardware and software in the area of embedded systems. They have developed various SBCs named after beagles (a popular breed of domestic canine species). You can find a list of the current SBCs that they developed, which are in production, at http://beagleboard.org/boards. You can also find related products and accessories for Beagle boards at the same URL.
Their latest product, at the time of writing, is PacketBeagle (http://beagleboard.org/pocket).
The ASUS Tinkerboard is designed and manufactured by ASUS (a Taiwan-based multinational corporation). Its size, layout, and pins are compatible with second- and third-generation Raspberry Pi boards. You can find more details about all the editions of the ASUS Tinkerboard at https://www.asus.com/us/Single-Board-Computer/. The following photograph shows the top view of an ASUS Tinkerboard:
NVIDIA Jetson is a family of modules that is used for computer vision, AI, and speech processing tasks (https://developer.nvidia.com/embedded/develop/hardware). The best member for beginners to get started with is Jetson Nano. And the best place to begin is on the web page of Jetson Nano Developer Kit at https://developer.nvidia.com/embedded/jetson-nano-developer-kit. Here's a side view of the developer kit:
Intel Corporation also produces many boards that can be called SBCs. You can find details on the current generation of modules that are in production at https://software.intel.com/en-us/iot/hardware/all. We have had the privilege of working with several of the excellent Intel SBCs and modules. Many of them are discontinued, and you can find the full list and support documentation for them at https://software.intel.com/en-us/iot/hardware/discontinued. Note that you might be able to get a good deal on used and discontinued boards from Intel. They are also great for learning. For beginners of computer vision, I like to recommend Intel Up Squared Kit. You can find out more at https://software.intel.com/en-us/iot/hardware/up-squared-ai-vision-dev-kit.
Raspberry Pi is a series of low-cost and credit card-sized SBCs developed by the Raspberry Pi Foundation in the United Kingdom. The purpose of developing Raspberry Pi was to promote the teaching of basic computer skills and programming in schools, in which it has served very well. Raspberry Pi has expanded its footprint well beyond its intended purpose by gaining prominence in the embedded systems market and computer science research in academia and industrial applications.
The Raspberry Pi Foundation offers downloads for many popular OS distributions. We can use a variety of programming languages such as Python, C, C++, and Java with Raspberry Pi. You can find more information on the Raspberry Pi Foundation website (https://www.raspberrypi.org/).
Raspberry Pi models
The Raspberry Pi board comes in many models. Additionally, there are a lot of associated accessories with these models. You can find the current list of models under production on the products page of the Raspberry Pi Foundation (https://www.raspberrypi.org/products/). Unfortunately, the page does not have any information on the discontinued product boards of the Raspberry Pi family.
Additionally, Raspberry Pi is also available in a more flexible form that is intended for industrial and embedded applications. This is known as a compute module. The compute module also has many iterations. A compute module prototyping kit is also made available by the foundation. You can find out more about compute modules and the prototyping kit on the same Raspberry Pi products page we discussed earlier.
As we have discussed, there are many models of Raspberry Pi boards available. And while it is tempting to discuss the technical specifications in detail for all of those boards, it is difficult to achieve that in brief. In the first edition of the book, I discussed the specifications of all the available Raspberry Pi board models in detail, since the number of models was far lower and we could count them on our fingers. Since writing the second edition of this book, there are over a dozen Raspberry Pi models. Therefore, we will discuss the technical specifications of only a couple of board models of Raspberry Pi.
We will use the Raspberry Pi 4B 4 GB and Raspberry Pi Zero W with header models for our computer vision examples. However, these examples can also be run on the other board models of Raspberry Pi. This is because all the software that we use (the OS, the programming language, and the OpenCV library) is fully backward compatible.
Raspberry Pi model 4B
You can find the product specifications of the Raspberry Pi 4B at https://www.raspberrypi.org/products/raspberry-pi-4-model-b/specifications/.
Raspberry Pi Zero W
You can find the specifications of the Raspberry Pi Zero W at https://www.raspberrypi.org/products/raspberry-pi-zero-w/.
Where can you buy these models?
OSes for Raspberry Pi
Many OSes have tailored distributions of OSes for Raspberry Pi boards. However, the early board models do not support all OSes. The latest model board, Raspberry Pi 4B, supports all the OSes mentioned at https://www.raspberrypi.org/downloads/.
Raspbian is a free OS based on Debian, which is a popular distribution of Linux. Raspbian is optimized to Raspberry Pi hardware. You can find more information about the Raspbian project on its home page (http://raspbian.org/).
Raspbian's home page mentions that it is not affiliated with the Raspberry Pi Foundation and is managed by fans of the Raspberry Pi and Debian projects.
The Raspbian web page provides a list of recommended Raspbian images at http://raspbian.org/RaspbianImages. An OS image is a file that can be written onto an SD card, and this SD card can then be used to boot the Raspberry Pi board. This is the easiest way of getting started with RPi; we will try to use it from now on. The image provided on the RPi Foundation's download page is the one most recommended by Raspbian. We will learn how to use this image to get started with RPi in the next section.
Setting up Raspbian on a Raspberry Pi
The setup is the one thing that usually deters many novice enthusiasts from getting started with SBCs. Many times, the instructions are very generic and do not cover all the cases for various types of hardware components. That is why I have dedicated an entire section to the setup of Raspbian on RPi. In this section, we will demonstrate the setup in detail with all the board models ever produced, with the exception of the compute modules.
- A Raspberry Pi board of any model.
- If you have a Raspberry Pi 4B board, then you will need a power supply of 5V 3A with a USB Type-C pin. Here is a photograph of a USB Type-C pin:
- To be on the safe side, you might want to purchase the official Raspberry Pi 15.3W USB-C power supply by the Raspberry Pi Foundation. The URL for this product is https://www.raspberrypi.org/products/type-c-power-supply/.
- For all the other models of Raspberry Pi, a 5V 2.5A power supply with a Micro-USB type pin should be compatible. Here's a photograph of a Micro-USB pin:
- You might want to purchase a Raspberry Pi Universal Power Supply (https://www.raspberrypi.org/products/raspberry-pi-universal-power-supply/) for this purpose.
- A USB keyboard and mouse: It is a good idea to purchase a USB keyboard with an integrated mousepad, as follows:
- For RPi Zero and RPi Zero W, the keyboard with a mousepad is mandatory because these board models have only one Micro-USB type of connector to the peripherals interface. Additionally, for RPi Zero and RPi Zero W, we need a USB to Micro-USB OTG converter, as follows:
- Raspberry Pi boards of any model work with any microSD card. The guidelines say that we should use a class 10 microSD card with a minimum of 16 GB. You might want to visit https://www.raspberrypi.org/documentation/installation/sd-cards.md for guidelines, and https://elinux.org/RPi_SD_cards for a compatibility list. RPi 1 Model A and RPi 1 Model B use SD cards. Therefore, it is better to have a microSD to SD card adapter, as follows:
- An HDMI monitor or a VGA monitor for visual display.
- All RPi board models, except RPi 4B, RPi Zero, and RPi Zero W, have an HDMI output and can be directly connected to the HDMI monitor with an HDMI male-to-male cable:
RPi 4B has a micro-HDMI output. Therefore, we need a micro-HDMI to HDMI converter. RPi Zero and RPi Zero W both have mini-HDMI outputs. So, for them, we need a mini-HDMI to HDMI converter. The following photograph shows the HDMI, mini-HDMI, and micro-HDMI ports, respectively:
We also need to plug the mini- and micro-HDMI ends to the RPi boards and the HDMI to the monitor. If you are planning to use a VGA monitor, then we will need HDMI/mini-HDMI/micro-HDMI to VGA converters depending on the board models.
Here is a photograph of an HDMI to VGA converter:
Finally, we require an SD card reader, as follows:
We will need a few more hardware components by the end of the chapter. We will discuss them when the need arises. For now, we are okay to proceed further.
Downloading the necessary software
- We need the latest image file of the Raspbian OS. This can be downloaded from the download page of the Raspberry Pi Foundation website at https://www.raspberrypi.org/downloads/raspbian/. The following screenshot shows the various options that are available for download:
- By the time you visit the URL, the page might have been updated, but the download options will usually remain the same. The first option is Raspbian Buster with desktop and recommended software and is the most recommended for beginners. The second option is Raspbian Buster with desktop. The third option is Raspbian Buster Lite and comes with the bare minimum software; it has the smallest size among all the download options.
- We can either download the ZIP file directly, or we can download the torrent file of the image. I recommend downloading the torrent file. Once the torrent file for Raspbian Buster with desktop and recommended software is downloaded, we can download the torrent software from https://www.bittorrent.com/. Download the free classic version and install it on your PC. Then, open the torrent file with BitTorrent and the download will begin. The following is a screenshot of a finished download:
- At the bottom of the screen, we can see the location of the download. Additionally, we can right-click on the finished installation and click on the Open Containing Folder option, as follows:
This will open the folder that has the ZIP file for the Raspbian OS image.
- We require software for unzipping the file. 7-Zip is the free and open source software for this. We can download the appropriate installable file (32-bit x86 or 64-bit x64) and install it. Once the installation is complete, open the ZIP file using the software. The following is a screenshot of 7-Zip:
Double-click on the ZIP file and then click on the Extract button in the menu. This will extract the file. The extracted file has the
- We require software to write this image to the microSD card and Win32DiskImager is the perfect software for the task. Download it from http://sourceforge.net/projects/win32diskimager/files/latest/. Run the installation file and install it.
Preparing the microSD card manually
The best way of installing an OS on a microSD card is to do it manually. This allows us to prepare the SD card manually so that we have easier access to the
/boot/config.txt configuration file, which must be modified, in a few cases, before booting up the RPi. We will discuss this in detail later. The default Raspbian image has only two partitions—boot and system. I recommend choosing, at a minimum, a 16 GB class 10 microSD card. Then, follow these steps:
- Unpack the fresh microSD card and insert it into the card reader. Plug the card reader into your Windows laptop or computer. Many laptops and computers come with an SD card reader. For these, insert the microSD card into the microSD to SD card adapter, and insert the adapter into the slot for the SD card reader of the computer or laptop.
- Then, a new drive will appear in the left-hand panel of Windows File Explorer. Right-click on the drive and choose Format. Here is a screenshot of the Format window:
- Make sure that you check the Quick Format checkbox. Then, click on the Start button. It will show a warning message, as follows:
- Click on the OK button to finish formatting.
- Once the formatting is complete, we need to write the Raspbian OS image file to the microSD card. Open
Win32DiskImagerand choose the Raspbian OS image file, as shown in the following screenshot:
- Then, click on the Write button. It will show the following warning box. Simply click on the OK button:
- Once the OS is successfully written to the SD card, it shows the following message box:
This means that the image has been successfully written to the microSD card. Now we can use it to boot up the RPi.
- Now, this step is necessary only if you are using a VGA monitor and not the HDMI monitor. Readers using the HDMI monitor can safely ignore this step. The microSD card's BOOT partition can be accessed using Windows File Explorer. It has the
config.txtfile. Double-click and open the file. We must edit the settings in the
/boot/config.txtfile, as follows, to enable a proper display on the VGA monitor:
g) Save the file.
If you are using Linux or macOS, then you will find the instructions to install the Raspbian OS on your microSD card for these OSes at https://www.raspberrypi.org/documentation/installation/installing-images/.
Booting up the Raspberry Pi for the first time
- Insert the microSD card into the microSD card slot of Pi. RPi 1 Model A and RPi 1 Model B do not have slots for an SD card. So, for these board models, we must use a microSD to SD card converter.
- Connect the Pi to the HDMI monitor. As discussed earlier, in case you have a VGA monitor, connect it using the HDMI/mini-HDMI/micro-HDMI to VGA converter.
- Connect the USB mouse and USB keyboard. It is recommended that you have a single keyboard with a mousepad. For RPi Zero and RPi Zero W, you need to first connect it to a USB OTG cable, and then connect the USB OTG cable to the board.
- Connect the RPi board to an appropriate power supply. Connect the monitor to the power supply. We need to make sure that the power is switched off at this point.
- Ensure you verify all the connections once. Then, turn on the power supply of the monitor. Finally, turn on the power supply for the RPi.
Now, our RPi board will start booting up. The green LED on the board will start blinking. Congratulations! The RPi board is booting for the first time.
If your HDMI monitor is showing no signal, then power down the RPi and change
/boot/config.txt on the microSD card. Boot up the RPi with this changed setting and the HDMI monitor will show the signal.
In the preceding window, set Country: and Language:. It will automatically select the time zone according to the country you selected. You can change that too if you wish. Click on the Next button, and the following window will appear:
Check the checkbox if there are black borders on the edges of the desktop view. The Raspbian OS will rectify it upon the next boot. The following window will appear after you click on the Next button, but only if the board model has Wi-Fi:
We can update the Raspbian OS and installed software here. We are going to learn how to do it manually in the latter part of this chapter. Click on the Skip or Next button, and the following window will appear:
We have finished most of the setup. Now, there are a few more things to do before we reboot out RPi, so click on the Later button.
Now, in the top-left corner of the desktop, you should see a Raspberry icon. It is the menu for Raspbian and functions in a similar way to the Windows logo on Microsoft Windows. Click on the logo and navigate to Preferences | Raspberry Pi Configuration:
This menu has an option for overclocking and GPU memory. For the RPi 4B, overclocking is disabled. We will learn how to overclock an RPi 4B board manually in the next chapter. The Localisation tab is as follows:
Once all these settings have been changed as per our choice, we can restart the RPi board by clicking on the Shutdown button in the Raspbian menu:
Here, we find the option to reboot the RPi. Once we reboot, and if we have chosen to retain the original password for the default user,
pi, the following warning message window will appear when booting up:
And this will keep on appearing after every boot as long as we choose to retain the default password.
Connecting various RPi board models to the internet
Make sure that DHCP (Dynamic Host Configuration Protocol) is enabled at the Wi-Fi router, the managed switch, or the internet gateway.
PRi 1 A, PRi 1 A+, RPi Zero, RPi Zero W, and RPi 3 A+ do not have Ethernet ports. However, RPi Zero W and RPi 3 A+ have built-in Wi-Fi. We can use a USB Wi-Fi dongle for the remaining models:
After plugging in the USB Wi-Fi adapter, we need to open
lxterminal. This is the command-line utility. We can find it as a small black icon in Raspbian's taskbar and under Accessories in the Raspbian menu. Once we click on it, the following window will appear:
We can type in the Linux commands here. After typing them in, press Enter to execute the command. We have opened this so that we can manually configure the network interface of Raspbian. It is quite easy. All the network-related information is stored in the
/etc/network/interfaces file. To connect to the Wi-Fi after plugging in the USB Wi-Fi dongle, we need to add a few entries to this file. First, take the backup of the original file by executing the following command:
mv /etc/network/interfaces /etc/network/interfaces.bkp
Then, we can create the
interfaces file from scratch by running the following command:
sudo nano /etc/network/interfaces
The preceding command will open the network interface's file with a plain text editor known as
nano. It is a simple WYSIWYG editor. Enter the following lines there:
source-directory /etc/network/interfaces.d auto lo iface lo inet loopback auto wlan0 allow-hotplug wlan0 iface wlan0 inet dhcp wpa-ssid "AshwinIon" wpa-psk "internet1"
After entering the lines, press Ctrl + X and then press Y. In the preceding settings, substitute
AshwinIon with your own SSID and
internet1 with a password for the same. Then, run the following command in Command Prompt:
sudo service networking restart
This restarts the networking service and connects to the Wi-Fi. In any case (Ethernet or Wi-Fi), the RPi is assigned with a unique IP address. We can find it out by running the
ifconfig command at
lxterminal. The output of the command will have the Ipv4 address listed under
Another way to know the IP address of the RPi is to check the active client tables in the router or the managed switch that the RPi board is connected to. The following is a screenshot of my router's active client table where we can see an entry for RPi:
Updating the RPi
Advanced Package Tool (APT) is a package management utility in Debian, Ubuntu, Raspbian, and their derivatives. APT is used to install, upgrade, and remove software. We will learn how to use it to update the OS and the software on the RPi board.
Run the following command:
sudo apt-get update
This command synchronizes the package list from the online source repository of software. Indexes of all the packages are refreshed. This updates all the repositories of the apps to all the latest update lists. This command must be executed before we execute the
Then, run the following command:
sudo apt-get dist-upgrade –fix-missing -y
This downloads and installs all the packages. It also removes obsolete packages. Depending on the speed of the internet, it takes some time. Finally, update the firmware by running the following command:
Finally, we can run the following command to shut down the RPi:
sudo shutdown -h now
And the following command reboots it:
This will update the firmware. Following this, the RPi board will be up to date in all aspects.
In this chapter, we learned important terms such as computer vision, OpenCV, SBCs, and Raspberry Pi. We learned how to set up a Raspbian OS on Raspberry Pi and how to configure a Pi to access the internet. We also learned how to update a Pi.
With the completion of this chapter, you can go ahead and set up the Raspbian OS on your Raspberry Pi. Additionally, you can connect your RPi board to the internet using Wi-Fi or Ethernet. This will make you ready for the computer vision adventure that will soon follow.
In the next chapter, you will learn how to remotely access a RPi, how to overclock it, and the installation of OpenCV 4 for Python 3 on an RPi.