Applying the Lessons of Deepfakes

The techniques in this book can be used for a lot more than face replacements. In this chapter, we’ll examine just a few examples of how you can apply the lessons and tools of this book in other fields. We’ll look at how to tweak and modify the techniques to use the results in new and unique ways.

In particular, we’ll look at just a few techniques from earlier in this book and see how they can be used in a new way. The examples in this chapter are not exhaustive, and there are always more ways that you could implement the abilities that deepfakes bring. In this chapter, we are more focused on the technique than the specifics, but in examining the technique, we’ll explore the following in new ways:

• Aligning other types of images
• The power of masking images
• Getting data under control

Technical requirements

For this chapter, there is one section with a small amount of code that demonstrates how to use a non-module Git repo for your own uses.

While this isn’t part of the hands-on section of the book, we’ve included the code to interface with a library: PeCLR. This code is also included in the book’s code repo with some additional functionality, including visualizing the points, but is just an example and is not meant to be a complete API for using PeCLR in your own project:

1. First, open Anaconda Command Prompt.
2. On Windows, hit Start and then type anaconda. This should bring up the following option:

Figure 8.1 – Anaconda Prompt

Click on this, and it will open an Anaconda prompt for the rest of the following commands.

3. Next, we need to clone a copy of the PeCLR library:

   git clone https://github.com/dahiyaaneesh/peclr.git

4. Download the model data.

The library includes a copy...

Aligning other types of images

Aligning faces is a critical tool for getting deepfakes to work. Without the alignment of faces, we’d be doomed with extremely long training times and huge models to correct the faces. It’s not a stretch to say that without alignment, modern deepfakes would effectively be impossible today.

Alignment saves time and compute power by removing the need for the neural network to figure out where the face is in the image and adapt for the many different locations the face may be. By aligning in advance, the AI doesn’t even need to learn what a face is in order to do its job. This allows the AI to focus on learning the task at hand, such as generating realistic facial expressions or speech, rather than trying to locate and correct misaligned faces.

In addition to improving the efficiency of the training process, aligning faces also helps to improve the quality and consistency of the final deepfake. Without proper alignment, the generated...

The power of masking images

When you take a photograph, you are capturing everything that the camera sees. However, the chances are that you’re not equally interested in every part of the image. If you’re on vacation, you might take a selfie of yourself in front of a waterfall, and while you value yourself and the waterfall, you care less about the cars or other people in the image. While you can’t remove the cars without adding something into the gaps for your vacation photos, sometimes, you’re only interested in the main subject and might want to cut it from the rest of the image.

With deepfakes, we can use a mask to help us remove the face from the image so that we replace only the face and leave the rest of the image alone. In other AI tasks, you might have similar needs but different objects that you want to cut out:

Figure 8.3 – An example of the mask used in the deepfake process

Next, let’s look at other...

Getting data under control

There’s a common saying in the AI community that an ML scientist’s job is only 10% ML and 90% data management. This, like many such sayings, is not far from the truth. While every ML task is focused on the actual training of the model, first, you must get your data into a manageable form before you can start the training. Hours of training can be completely wasted if your data isn’t properly prepared.

Before you can start training a model, you have to decide what data it is that you’re going to train it with. That data must be gathered, cleaned, converted into the right format, and generally made ready to train. Often, this involves a lot of manual processes and verification.

Defining your rules

The most important thing in the manual process is to make sure that all your data meets your requirements and meets a consistent level of quality. To do this, you need to define exactly what “good” data means. Whether...

Summary

In this chapter, we looked at how you can apply the lessons and techniques of deepfakes to other environments. First, we examined how to align other types of images, using hands as an example. Then, we looked at the different types of masks and considered using them in a parking lot monitoring solution. Following this, we examined data management and considered how a dataset to detect different animals might be built.

This process of figuring out how to apply techniques in new environments used throughout this chapter is itself a valuable technique that can help you throughout your development career, especially if you’re going to work at the edge of your computer’s capabilities like AI does now. Sometimes, the only difference between a successful project and an impossible one is the technique you borrow from a previous project.

In the next chapter, we’re going to look at the potential and future of deepfakes and other generative AIs.

...