Deploying Detectron2 Models into Browsers and Mobile Environments

This chapter introduces the Open Neural Network Exchange (ONNX) framework. This framework helps share deep neural networks across multiple frameworks and platforms. It is extremely helpful when there is a need to deploy Detectron2 models into browsers or mobile environments. This chapter also describes D2Go, a framework for training, quantizing, and deploying models with minimal memory storage and computation requirements. Models created using this framework are extremely helpful for deploying into mobile or edge devices.

By the end of this chapter, you will understand the ONNX framework and when it is helpful. You can export custom Detectron2 models into this format and deploy them into the web or browser environments. You will also understand the advantage of the D2Go framework and know how to utilize pre-trained models on its Model Zoo or train custom models using D2Go. You can also perform quantization on a custom...

Technical requirements

You should have completed Chapter 1 to have an appropriate development environment for Detectron2. All the code, datasets, and results are available on the GitHub page of the book at https://github.com/PacktPublishing/Hands-On-Computer-Vision-with-Detectron2.

Deploying Detectron2 models using ONNX

ONNX is an open source format for representing and sharing deep learning models between different frameworks. The models can then be deployed in various platforms (e.g., servers or mobile devices) that support these frameworks. The following sections introduce ONNX and its supported frameworks and platforms, export a PyTorch model to ONNX format, and load the exported model into the browser environment.

Introduction to ONNX

ONNX aims to be a universal standard for deep learning models, allowing for interoperability between different tools, libraries, and frameworks. Microsoft and Facebook initiated the ONNX project in 2017. However, this project is currently an open source project managed by the ONNX community, which includes contributors from a wide range of organizations. This means that the project has great potential and support. The format is designed to be flexible and extensible, supporting a wide range of deep learning models, including...

Developing mobile computer vision apps with D2Go

This section introduces D2Go, the steps to use existing D2Go models or train custom D2G models, and the model quantization.

Introduction to D2Go

Exporting Detectron2 models to TorchScript or ONNX formats may serve the purpose of deploying into the server, web, and browser environments. However, deploying Detectron2 models into mobile or edge devices may require further optimizations. Therefore, Facebook AI Research also developed D2Go, which supports training computer vision applications ready to be deployed into mobile and edge devices.

Specifically, it is a deep learning model developed on top of Detectron2 and PyTorch. Similar to Detectron2, it also contains advanced and efficient backbone neural networks that support deployments for mobile and edge devices. It provides tools and techniques for training, quantization, and deployment into mobile and edge devices. It also supports exporting to TorchScript format for deployment...

Summary

This chapter introduced the ONNX model format and the platforms and frameworks that support it. This framework helps Detectron2 models to be interoperable with different frameworks and platforms. It then provides the steps to export Detectron2 models to this format and the code to deploy the exported model in the browser environments. This chapter also introduced D2Go, a framework for training, optimizing, and deploying neural networks for computer vision applications with minimal memory storage and computation resources. Additionally, its models are prepared to be further optimized using the quantization technique, which converts the model weights and activations in lower-precision number systems. This quantization step further reduces the model memory requirement and improves computation performance. Therefore, D2Go models are suitable for deploying into mobile or edge devices. D2Go also has pre-trained models on its Model Zoo. Thus, this chapter provides the steps to build...