Fun with Faces

In Chapter 3, Home Security Applications, we created a new application named Gazer, with which we can capture video and detect motion from the webcams attached to our computers. In this chapter, we will continue playing with our webcams—instead of detecting motion, we will create a new application that is able to detect faces using our camera. First, we will detect the faces in our webcams. Then, we will detect the facial landmarks on the detected faces. With these facial landmarks, we can know where the eyes, the nose, the mouth, and the cheeks are on each detected face so that we can apply some funny masks onto the faces.

The following topics will be covered in this chapter:

• Taking photos from webcams
• Detecting faces using OpenCV
• Detecting facial landmarks using OpenCV
• The resource system of the Qt library
• Applying masks on faces

...

Technical requirements

As we saw in the previous chapters, users are required to have at least Qt version 5 installed and have basic knowledge of C++ and Qt programming. Also, the latest version of OpenCV, 4.0, should be correctly installed. Additionally, besides the core and imgproc modules, the video and videoio modules of OpenCV will also be used in this chapter. If you have been through the previous chapters, then these requirements will have already been met.

We will use some pretrained machine learning models provided by OpenCV to detect faces and facial landmarks, so it is better if you have some basic knowledge of machine learning technology. Some of these machine learning models are from the extra modules of the OpenCV library, so the extra modules of OpenCV must also be installed along with the core modules. If you are not sure about this, don't worry, we will install...

The Facetious application

Since the application we are going to create in this chapter will give us a lot of fun by applying funny masks onto detected faces in real time, I name the application Facetious. The first thing that the Facetious application could do is open a webcam and play the video feed from it. That is the work we had done in our Gazer application, which was built by us in the preceding chapter. So here, I will borrow the skeleton of the Gazer application as the basis of our new application. The plan is, first, we make a copy of Gazer, rename it to Facetious, delete the features about motion detecting and change the video recording feature into a new feature of photo taking. By doing so, we will get a simple and clean application onto which we can add our new features of faces and facial landmark detecting.

...

Detecting faces using cascade classifiers

In the preceding section, we created our new application, Facetious, with which we can play the video feed from our camera and take photos. In this section, we will add a new feature to it—detecting faces in videos in real time by using the OpenCV library.

We will use some facilities called cascade classifiers, which are offered by OpenCV, to detect faces. A cascade classifier is not only used to detect faces, but it is also used to detect objects. As a classifier, it tells us whether a certain region of interest in an image is a certain type of object or not. The classifier contains several simpler classifiers or stages and subsequently applies these simpler classifiers to a region of interest. If any simpler classifier gives a negative result, we say the region of interest does not contain any objects of interest. Otherwise, if...

Detecting facial landmarks

In the preceding section, by detecting faces with a cascade classifier provided by OpenCV we know which regions are faces in an image. But with only rectangular regions, we don't know many details about the faces: where are the eyes, eyebrows, and nose on the faces? In face recognition technology, we call these details facial landmarks. In this section, we will try to find a way to detect these facial landmarks.

Unfortunately, the OpenCV core modules don't provide an algorithm to detect facial landmarks, so we should resort to the face module, which is an extra OpenCV module.

Before using the face extra module, we must ensure that the module is installed on our computer. In Chapter 2, Editing Images Like a Pro, in the Building and installing OpenCV from the source section, we built and installed OpenCV v4.0.0 from a source without the extra...

Applying masks to faces

In the previous sections of this chapter, we successfully detected the faces and the facial landmarks in our video feed. In this section, we will do something more interesting—I have three masks or ornaments here. Let's try to apply them onto the detected faces in real time:

These ornaments are images on the disk. Different to the data from the users, for example the images we viewed in Chapter 1, Building an Image Viewer, the images we edited in Chapter 2, Editing Images Like a Pro, and the videos we recorded in Chapter 3, Home Security Applications, these ornaments are not data from the users; they are a part of our application, just like our code source files. We must somehow bind them to our application, especially when we ship the application to the users. You can simply pack up these resources along the compiled binary, let the users...

Summary

In this chapter, we created a new application named Facetious by abridging the Gazer application, which was built in the preceding chapter. In the abridging process, the features of video saving and motion detecting were removed and a new feature of photo taking was added. As a result, we got a clean application that allows us to focus on face recognition. Then, in the application, we developed the feature of face detecting using a pretrained cascade classifier and the feature of facial landmarks detecting using another pretrained machine learning model. At last, which is the funny part, we applied many ornaments on the detected faces with the help of the detected landmarks.

In the next chapter, we will talk about Optical Character Recognition (OCR) technology and use that technology to extract text from images or scanned documents.

...

Questions

Try out these questions to test your knowledge of this chapter:

1. Try the LBP cascade classifier to detect faces by yourself.
2. There are some other algorithms that can be used to detect facial landmarks in the OpenCV library and most of them can be found at https://docs.opencv.org/4.0.0/db/dd8/classcv_1_1face_1_1Facemark.html. Please try them by yourself.
3. How do you apply a colored ornament to a face?