Moving a Model to Production

Moving a model to production is a step toward enabling the consumption of our model by an external party. We should expose our model to the world and start rendering predictions on real, unseen input.

It is not sufficient to have a trained PyTorch model for deployment. We need additional server components for creating communication channels from the real world to the PyTorch model and back to the real world. It is important that we know how to create an API (through which the user can interact with the model), wrap it as a self-contained application (so that it can be deployed on any computer), and ship it to the cloud – so that anybody with the required URL and credentials can interact with the model. To successfully move a model to production, all these steps are necessary. In this chapter, we will deploy a simple application that is accessible...

Understanding the basics of an API

By now, we know how to create a deep learning model for various tasks. It accepts/returns tensors as input/output. But an outsider such as a client/end user would talk only in terms of images and classes. Furthermore, they would expect to send and receive input/output over channels that might have nothing to do with Python. The internet is the easiest channel to communicate on. Hence, for a client, the best-case deployment scenario would be if we can set up a publically available URL and ask them to upload their images there. One such paradigm is called an Application Programming Interface (API), which has standard protocols that accept input and post output over the internet while abstracting the user away from how the input is processed or the output is generated.

Some common protocols in APIs are POST, GET, PUT, and DELETE, which are sent as requests by the client to the host server along with relevant data. Based on the request and data, the server...

Creating an API and making predictions on a local server

In this section, we will learn about making predictions on a local server (that has nothing to do with the cloud). At a high level, this involves the following steps:

1. Installing FastAPI
2. Creating a route to accept incoming requests
3. Saving an incoming request on disk
4. Loading the requested image, then preprocessing and predicting with the trained model
5. Postprocessing the results and sending back the predictions as a response to the same incoming request

All of the steps in this section are summarized as a video walk-through here: https://tinyurl.com/MCVP-Model2FastAPI.

Let's begin by installing FastAPI in the following subsection.

Installing the API module and dependencies

Since FastAPI is a Python module, we can use pip for installation, and be ready to code an API. We will now open a new terminal and run the following command:

$pip install fastapi uvicorn aiofiles jinja2 

We have installed a couple more dependencies that...

Moving the API to the cloud

So far, we have learned about making predictions on a local server (http://127.0.0.1 is a URL of the local server that cannot be accessed on the web) – so, only the owner of the local machine can use the model. In this section, we will learn about moving this model to the cloud so that anyone can predict using an image.

In general, companies deploy services in redundant machines to ensure reliability and there is little control over the hardware provided by the cloud provider. It is not convenient to keep track of all folders and their code, or copy-paste all the code, then install all the dependencies, ensuring the code works as expected on the new environment, and forward ports on all the cloud machines. There are too many steps to be followed for the same code on every new machine. Repeating these steps is a waste of time for the developer and such a process is highly prone to mistakes.

We would rather install one package that has everything than...

Summary

In this chapter, we learned what additional steps are required in moving a model to production. We learned what an API is and what its components are. After creating an API, with the use of FastAPI, we glanced at the core steps of creating a Docker image of the API. Using AWS, we created our own Docker registry in the cloud and went through the steps to push our Docker image there. We saw what it takes to create an EC2 instance and install the required libraries to pull the Docker image from ECR, build a Docker container from it, and deploy it for any user to make predictions.

In the next and final chapter, we will learn about OpenCV, which has utilities that help in addressing some of the image-related problems in a constrained environment. We will go through five different use cases to gain an understanding of leveraging OpenCV for image analysis. Learning the functionalities of OpenCV will further strengthen our computer vision repertoire.

...