Object Detection and Image Segmentation

In Chapter 3, Advanced Convolutional Networks, we discussed some of the most popular and best performing convolutional neural network (CNN) models. To focus on the architecture specifics of each network, we viewed the models in the straightforward context of the classification problem. In the universe of computer vision tasks, classification is fairly straightforward, as it assigns a single label to an image. In this chapter, we'll shift our focus to two more interesting computer vision tasks—object detection and semantic segmentation, while the network architecture will take a backseat. We can say that these tasks are more complex compared to classification, because the model has to obtain a more comprehensive understanding of the image. It has to be able to detect different objects as well as their positions on the image. At...

Introduction to object detection

Object detection is the process of finding object instances of a certain class, such as faces, cars, and trees, in images or videos. Unlike classification, object detection can detect multiple objects as well as their location in the image.

An object detector would return a list of detected objects with the following information for each object:

• The class of the object (person, car, tree, and so on).
• Probability (or confidence score) in the [0, 1] range, which conveys how confident the detector is that the object exists in that location. This is similar to the output of a regular classifier.
• The coordinates of the rectangular region of the image where the object is located. This rectangle is called a bounding box.

We can see the typical output of an object-detection algorithm in the following photograph. The object type and confidence score are...

Introducing image segmentation

Image segmentation is the process of assigning a class label (such as person, car, or tree) to each pixel of an image. You can think of it as classification, but on a pixel level—instead of classifying the entire image under one label, we'll classify each pixel separately. There are two types of segmentation:

• Semantic segmentation: This assigns a class to each pixel, but doesn't differentiate between object instances. For example, the middle image in the following screenshot shows a semantic segmentation mask, where the pixels of each vehicle have the same value. Semantic segmentation can tell us that a pixel is part of a vehicle, but cannot make a distinction between two vehicles.
• Instance segmentation: This assigns a class to each pixel and differentiates between object instances. For example, the image on the right in the following...

Summary

In this chapter, we discussed object detection and image segmentation. We started with the one-shot detection algorithm, YOLO, and then we continued with the two-stage Faster R-CNN algorithm. Next, we discussed the semantic segmentation network architecture, U-Net. Finally, we talked about Mask R-CNN—an extension of Faster R-CNN for instance segmentation.

In the next chapter, we'll explore new types of ML algorithms called generative models. We can use them to generate new content, such as images. Stay tuned—it will be fun!