Chapter 13: Creating an Efficient Prediction API Endpoint with FastAPI

In the previous chapters, we introduced the most common data science techniques and libraries largely used in the Python community. Thanks to those tools, we can now build machine learning models that can make efficient predictions and classify data. Of course, we now have to think about a convenient interface so that we can take advantage of their intelligence. This way, microservices or frontend applications can ask our model to make predictions to improve the user experience or business operations.

In this chapter, we'll learn how to do that with FastAPI. As we've seen throughout this book, FastAPI allows us to implement very efficient REST APIs with clear and lightweight syntax. In this chapter, you'll learn how to do this as efficiently as possible so that it can serve thousands of prediction requests. To help us with this task, we'll introduce another library, Joblib, that provides tools...

Technical requirements

You'll need a Python virtual environment, similar to the one we set up in Chapter 1, Python Development Environment Setup.

You can find all the code examples for this chapter in this book's dedicated GitHub repository: https://github.com/PacktPublishing/Building-Data-Science-Applications-with-FastAPI/tree/main/chapter13.

Persisting a trained model with Joblib

In the previous chapter, you learned how to train an estimator with scikit-learn. When building such models, you'll likely obtain a rather complex Python script to load your training data, pre-process it, and train your model with the best set of parameters. However, when deploying your model in a web application, such as FastAPI, you don't want to repeat this script and run all those operations when the server is starting. Instead, you need a ready-to-use representation of your trained model that you can just load and use.

This is what Joblib does. This library aims to provide tools for efficiently saving Python objects to disk, such as large arrays of data or function results: this operation is generally called dumping. Joblib is already a dependency of scikit-learn, so we don't even need to install it. scikit-learn uses it internally to load the bundled toy datasets.

As we'll see, dumping a trained model involves just...

Implementing an efficient prediction endpoint

Now that we have a way to save and load our machine learning models, it's time to use them in a FastAPI project. As you'll see, the implementation shouldn't be too much of a surprise if you've followed this book. The main part of the implementation is the class dependency, which will take care of loading the model and making predictions. If you need a refresher on class dependencies, check out Chapter 5, Dependency Injections in FastAPI.

Let's go! Our example will be based on the newgroups model we dumped in the previous section. We'll start by showing you how to implement the class dependency, which will take care of loading and making predictions:

chapter13_prediction_endpoint.py

class PredictionInput(BaseModel):
    text: str
class PredictionOutput(BaseModel):
    category: str
class NewsgroupsModel:
    model: Optional[Pipeline]
 ...

Caching results with Joblib

If your model takes time to make predictions, it may be interesting to cache the results: if the prediction for a particular input has already been done, it makes sense to return the same result we saved on disk, rather than running the computations again. In this section, we'll learn how to do this with the help of Joblib.

Joblib provides us with a very convenient and easy-to-use tool to do this, so the implementation is quite straightforward. The main concern will be about whether we should choose standard or async functions to implement the endpoints and dependencies. This will allow us to explain some of the technical details of FastAPI in more detail.

We'll build upon the example we provided in the previous section. The first thing we must do is initialize a Joblib Memory class, which is the helper for caching functions results. Then, we can add a decorator to the functions we want to cache. You can see this in the following example...

Summary

Congratulations! You're now able to build a fast and efficient REST API to serve your machine learning models. Thanks to Joblib, you've learned how to dump a trained scikit-learn estimator into a file that's easy to load and use inside your application. We've also seen an approach to caching prediction results using Joblib. Finally, we discussed how FastAPI handles synchronous operations by sending them to a separate thread to prevent blocking. While this was a bit technical, it's important to bear this aspect in mind when dealing with blocking I/O operations.

We're near the end of our FastAPI journey. Before letting you build awesome data science applications by yourself, we have provided one last chapter to push this a bit further: using WebSockets and a library dedicated to computer vision, OpenCV, we'll learn how to implement an application that can perform real-time face detection.