Detecting and Recognizing Faces

Computer vision makes many futuristic-sounding tasks a reality. Two such tasks are face detection (locating faces in an image) and face recognition (identifying a face as a specific person). OpenCV implements several algorithms for face detection and recognition. These have applications in all sorts of real-world contexts, from security to entertainment.

This chapter introduces some of OpenCV's face detection and recognition functionality, along with the data files that define particular types of trackable objects. Specifically, we look at Haar cascade classifiers, which analyze the contrast between adjacent image regions to determine whether or not a given image or sub image matches a known type. We consider how to combine multiple Haar cascade classifiers in a hierarchy so that one classifier identifies a parent region (for our purposes,...

Technical requirements

This chapter uses Python, OpenCV, and NumPy. As part of OpenCV, it uses the optional opencv_contrib modules, which include functionality for face recognition. Some parts of this chapter use OpenCV's optional support for OpenNI 2 to capture images from depth cameras. Please refer back to Chapter 1, Setting Up OpenCV, for installation instructions.

The complete code for this chapter can be found in this book's GitHub repository, https://github.com/PacktPublishing/Learning-OpenCV-4-Computer-Vision-with-Python-Third-Edition, in the chapter05 folder. Sample images are in the repository in the images folder.

Conceptualizing Haar cascades

When we talk about classifying objects and tracking their location, what exactly are we hoping to pinpoint? What constitutes a recognizable part of an object?

Photographic images, even from a webcam, may contain a lot of detail for our (human) viewing pleasure. However, image detail tends to be unstable with respect to variations in lighting, viewing angle, viewing distance, camera shake, and digital noise. Moreover, even real differences in physical detail might not interest us for classification. Joseph Howse, one of this book's authors, was taught in school that no two snowflakes look alike under a microscope. Fortunately, as a Canadian child, he had already learned how to recognize snowflakes without a microscope, as the similarities are more obvious in bulk.

Hence, some means of abstracting image detail is useful in producing stable classification...

Getting Haar cascade data

The OpenCV 4 source code, or your installation of a prepackaged build of OpenCV 4, should contain a subfolder called data/haarcascades. If you are unable to locate this, refer back to Chapter 1, Setting Up OpenCV, for instructions on obtaining the OpenCV 4 source code.

The data/haarcascades folder contains XML files that can be loaded by an OpenCV class called cv2.CascadeClassifier. An instance of this class interprets a given XML file as a Haar cascade, which provides a detection model for a type of object such as a face. cv2.CascadeClassifier can detect this type of object in any image. As usual, we could obtain a still image from a file, or we could obtain a series of frames from a video file or a video camera.

Once you find data/haarcascades, create a directory elsewhere for your project; in this folder, create a subfolder called cascades, and copy...

Using OpenCV to perform face detection

With cv2.CascadeClassifier, it makes little difference whether we perform face detection on a still image or a video feed. The latter is just a sequential version of the former: face detection on a video is simply face detection applied to each frame. Naturally, with more advanced techniques, it would be possible to track a detected face continuously across multiple frames and determine that the face is the same one in each frame. However, it is good to know that a basic sequential approach also works.

Let's go ahead and detect some faces.

Performing face detection on a still image

The first and most basic way to perform face detection is to load an image and detect faces in it...

Swapping faces in the infrared

Face detection and recognition are not limited to the visible spectrum of light. With a Near-Infrared (NIR) camera and NIR light source, face detection and recognition are possible even when a scene appears totally dark to the human eye. This capability is quite useful in security and surveillance applications.

We studied basic usage of NIR depth cameras, such as the Asus Xtion PRO, in Chapter 4, Depth Estimation and Segmentation. We extended the object-oriented code of our interactive application, Cameo. We captured frames from a depth camera. Based on depth, we segmented each frame into a main layer (such as the user's face) and other layers. We painted the other layers black. This achieved the effect of hiding the background so that only the main layer (the user's face) appeared on-screen in the interactive video feed.

Now, let&apos...

Summary

By now, you should have a good understanding of how face detection and face recognition work and how to implement them in Python and OpenCV 4.

Face detection and face recognition are constantly evolving branches of computer vision, with algorithms being developed continuously, and they will evolve even faster in the near future with a growing interest in robotics and the Internet of Things (IoT).

For now, the accuracy of detection and recognition algorithms heavily depends on the quality of the training data, so make sure you provide your applications with a large number of training images covering a variety of expressions, poses, and lighting conditions.

As human beings, we might be predisposed to think that human faces are particularly recognizable. We might even be overconfident in our own face recognition abilities. However, in computer vision, there is nothing very...