Face Detection, Tracking, and Recognition

Face processing is a hot topic in artificial intelligence because a lot of information can be automatically extracted from faces using computer vision algorithms. The face plays an important role in visual communication because a great deal of non-verbal information, such as identity, intent, and emotion, can be extracted from human faces. Face processing is a really interesting topic for computer vision learners because it touches on different areas of expertise, such as object detection, image processing, and landmark detection or object tracking.

In this chapter, you will be introduced to the main topics related to face processing using state-of-the-art algorithms and techniques in order to achieve impressive results.

We will cover the following topics:

• Face processing introduction
• Face detection
• Detecting facial landmarks
• Face tracking...

Technical requirements

The technical requirements are listed as follows:

• Python and OpenCV
• A Python-specific IDE
• The NumPy and Matplotlib packages
• The Git client
• The Dlib package
• The face_processing package

Further details about how to install these requirements are mentioned in Chapter 1, Setting Up OpenCV. The GitHub repository for Mastering OpenCV 4 with Python, which contains all the supporting project files necessary to work through the book, from the first chapter to the last, can be accessed at https://github.com/PacktPublishing/Mastering-OpenCV-4-with-Python.

Installing dlib

Dlib (http://dlib.net/python/index.html) is a C++ software library that contains computer vision, machine learning, and deep learning algorithms...

Face processing introduction

In this chapter, we will cover the main topics in connection with face processing. In order to do so, we will be using the OpenCV library, but also the dlib (http://dlib.net/python/index.html, https://pypi.org/project/dlib/, https://github.com/davisking/dlib), face_recognition (https://pypi.org/project/face_recognition/, https://github.com/ageitgey/face_recognition) and cvlib (https://pypi.org/project/cvlib/, https://github.com/arunponnusamy/cvlib, https://www.cvlib.net/) Python packages. In the previous section, you saw how to install these packages.

In order to introduce this chapter, we will be using different approaches throughout all the sections to see the different possibilities you have at hand to solve a concrete face processing task, and it can be helpful to have a high-level overview of all of these alternatives.

This diagram attempts to...

Face detection

Face detection, which can be defined as the task of determining the location and size of faces in digital images, is usually the first key step when building face processing applications (for example, facial expression recognition, drowsiness detection, gender classification, face recognition, head-pose estimation, or human-computer interaction). This is because the aforementioned applications requires as an input the location and size of the detected faces. Therefore, automatic face detection plays a critical role, and is one of the most studied topics in the artificial intelligence community.

Face detection seems like an effortless task for a human, but it is a very challenging task for computers because many problems/challenges are usually involved (for example, appearance variations, scale, rotation, facial expressions, occlusion, or lighting condition). Face...

Detecting facial landmarks

In computer vision, the localization of the fiducial facial key points (also called facial landmarks) is usually a key step in a lot of facial analysis methods and algorithms. Facial expression recognition, head pose estimation algorithms, and drowsiness detection systems are only a few examples, relying heavily on the facial shape information provided by the detection of facial landmarks.

A facial landmark detection algorithm aims to automatically identify the locations of the facial landmark points in images or videos. More specifically, those key points are either the dominant points that describe the unique location of a facial component (for example, corners of mouth or eyes) or an interpolated point that connects those dominant points around the facial components and facial contour. Formally, given a facial image denoted as I, a landmark detection...

Face tracking

Object tracking tries to estimate the trajectory of the target throughout the video sequence where only the initial location of a target is known. This task is really challenging on account of several factors, such as appearance variations, occlusions, fast motion, motion blur, and scale variations.

In this sense, discriminative correlation filter (DCF)-based visual trackers provide state-of-the-art performance. Additionally, these trackers are computationally efficient, which is critical in real-time applications. Indeed, the state-of-the-art performance of DCF-based trackers can be seen in the results of the visual object tracking (VOT) 2014 Challenge. In the VOT2014 Challenge, the top three trackers are based on correlation filters. VOT2014 evaluated 38 trackers (33 submitted trackers and 5 baselines from the VOT2014 committee: http://www.votchallenge.net/vot2014...

Face recognition

With the development of computer vision, machine learning, and deep learning, face recognition has become a hot topic. Face recognition can be applied to a wide range of uses, including crime prevention, surveillance, forensic applications, biometrics, and, more recently, in social networks. Automatic face recognition has various challenges, such as occlusions, appearance variations, expression, aging, and scale variations. Following its success with object recognition, CNNs have been widely used for face recognition.

In this chapter, we will see the functionality that OpenCV offers in connection with face recognition, and will also explore some deep learning approaches, which can be easily integrated into your computer vision projects to perform state-of-the-art face recognition results.

...

Summary

In this chapter, we looked at state-of-the-art algorithms and techniques for face detection, detecting facial landmarks, face tracking, and face recognition. We reviewed the methods the main Python libraries and packages offer for face processing. More specifically, OpenCV, dlib, face_processing, and cvlib were introduced in the context of face processing. Some of these reviewed methods are based on deep learning techniques.

In the next chapter, we will explore deep learning in depth.

Questions

1. What packages and libraries were reviewed in this chapter in connection with face processing?
2. What is the main difference between face identification and face verification?
3. What does the cv2.face.getFacesHAAR() OpenCV function do?
4. What does the cv2.dnn.blobFromImage() function do?
5. What function does the cvlib package provide to detect faces?
6. What function does face_recognition provide to detect facial landmarks?
7. What function does dlib provide to initialize the correlation tracker?
8. What function does dlib provide to start the correlation tracker?
9. What function does dlib provide to get the position of the tracked object?
10. Calculate the 128D descriptor to perform face recognition of the image BGR image with dlib.

Further reading

The following resource will help you dive deeper into face processing with Python:

• Artificial Intelligence with Python, by Prateek Joshi (2017) (https://subscription.packtpub.com/book/big_data_and_business_intelligence/9781786464392)