Introduction to PointRend

PointRend is a project as a part of Detectron2. It helps provide better segmentation quality for detected object boundaries. It can be used with instance segmentation and semantic segmentation. It is an extension of the Mask R-CNN head, which we discussed in the previous chapter. It performs point sampling on the detected mask and performs predictions on the sampled points instead of all the points in the mask. This technique allows us to compute the mask with a higher resolution, thus providing a higher mask resolution. Figure 11.1 illustrates an example of two images when not using (left) and when using (right) PointRend:

Figure 11.1: Segmentation quality with and without PointRend

PointRend helps render segmentations that are of a higher resolution with object boundaries that are crisp and less smooth. Therefore, it is useful if the objects to detect have sharp edges.

At inference time, starting from the coarse prediction...