Deploying Detectron2 Models into Server Environments

This chapter walks you through the steps of the export process to convert Detectron2 models into deployable artifacts. Specifically, it describes the standard file formats of deep learning models such as TorchScript and the corresponding runtimes for these formats, such as PyTorch and C++. This chapter then provides the steps to convert Detectron2 models to the standard file formats and deploy them to the corresponding runtimes.

By the end of this chapter, you will understand the standard file formats and runtimes that Detectron2 supports. You can perform steps to export Detectron2 models into TorchScript format using tracing or scripting method. Additionally, you can create a C++ application to load and execute the exported models.

In this chapter, we will cover the following topics:

• Supported file formats and runtimes for PyTorch models
• Deploying custom Detectron2 models

Technical requirements

You should have completed Chapter 1 to have an appropriate development environment for Detectron2. All the code, datasets, and results are available on the GitHub page of the book at https://github.com/PacktPublishing/Hands-On-Computer-Vision-with-Detectron2.

Supported file formats and runtimes

This section introduces the development environments, file formats, and runtimes for PyTorch applications in general and Detectron2 applications specifically. It then uses simple examples to explain these concepts.

Development environments, file formats, and runtimes

There are three main concepts to grasp while developing and deploying deep learning models: the development environment and programming languages, the model file formats, and the runtime environments. Figure 12.1 illustrates these three concepts and examples of each.

Figure 12.1: Development environments, file formats, and deployment environments

The development environment and programming languages or frameworks such as PyTorch or Python, in general, prioritize ease of use and customizations. However, the Python language and PyTorch framework may have runtime issues at deployment. For instance, they are relatively slow compared to other runtime environments...

Deploying custom Detectron2 models

The previous section described the model formats and respective runtimes for PyTorch. It also used simple models for illustration purposes. This section will focus on deploying custom Detectron2 models into server environments using the techniques described in the previous section. This section will first describe the main export utilities that Detectron2 provides to support exporting its models. It will then provide the code to export a custom Detectron2 model into TorchScript using the tracing and scripting approaches.

Detectron2 utilities for exporting models

Detectron2 provides a wrapper class called TracingAdapter, which helps wrap a Detectron2 model and supports exports using the tracing method. Detectron2 models take rich inputs and also produce rich outputs (i.e., they can be dictionaries or objects). Conversely, the tracing function (torch.jit.trace) takes tensors as inputs/outputs. Therefore, this adapter class helps flatten the inputs...

Summary

This chapter introduced the file formats and their corresponding runtimes supported by PyTorch in general and Detectron2 specifically. It then provided simple code to export PyTorch models into TorchScript using tracing and scripting approaches. It also provided a tutorial on deploying the TorchScript model into the low-latency C++ environment at production time. This chapter then described the main utilities that Detectron2 provides to perform exports of its models to TorchScript. It then provided the steps and code to export a custom Detectron2 model into TorchScript with tracing and scripting approaches using the described utilities.

The server deployment is useful in mass and large system production. However, in many cases, there are requirements to deploy models in different environments, such as web and mobile environments. Therefore, the next chapter discusses the necessary steps to perform these deployment tasks.