Creating an Efficient Prediction API Endpoint with FastAPI

In the previous chapter, we introduced the most common data science techniques and libraries largely used in the Python community. Thanks to those tools, we can now build machine learning models that can make efficient predictions and classify data. Of course, we now have to think about a convenient interface so that we can take advantage of their intelligence. This way, microservices or frontend applications can ask our model to make predictions to improve the user experience or business operations. In this chapter, we’ll learn how to do that with FastAPI.

As we’ve seen throughout this book, FastAPI allows us to implement very efficient REST APIs with a clear and lightweight syntax. In this chapter, you’ll learn how to use them as efficiently as possible in order to serve thousands of prediction requests. To help us with this task, we’ll introduce another library, Joblib, which provides tools...

Technical requirements

For this chapter, you’ll require a Python virtual environment, just as we set up in Chapter 1, Python Development Environment Setup.

You’ll find all the code examples for this chapter in the dedicated GitHub repository at https://github.com/PacktPublishing/Building-Data-Science-Applications-with-FastAPI-Second-Edition/tree/main/chapter12.

Persisting a trained model with Joblib

In the previous chapter, you learned how to train an estimator with scikit-learn. When building such models, you’ll likely obtain a rather complex Python script to load your training data, pre-process it, and train your model with the best set of parameters. However, when deploying your model in a web application such as FastAPI, you don’t want to repeat this script and run all those operations when the server is starting. Instead, you need a ready-to-use representation of your trained model that you can just load and use.

This is what Joblib does. This library aims to provide tools for efficiently saving Python objects to disk, such as large arrays of data or function results: this operation is generally called dumping. Joblib is already a dependency of scikit-learn, so we don’t even need to install it. Actually, scikit-learn itself uses it internally to load the bundled toy datasets.

As we’ll see, dumping a...

Implementing an efficient prediction endpoint

Now that we have a way to save and load our machine learning models, it’s time to use them in a FastAPI project. As you’ll see, the implementation shouldn’t be too much of a surprise if you’ve followed this book. The main part of the implementation is the class dependency, which will take care of loading the model and making predictions. If you need a refresher on class dependencies, check out Chapter 5, Dependency Injection in FastAPI.

Let’s go! Our example will be based on the newgroups model we dumped in the previous section. We’ll start by showing you how to implement the class dependency, which will take care of loading the model and making predictions:

chapter12_prediction_endpoint.py

class PredictionInput(BaseModel):           text: str
class PredictionOutput(BaseModel):
         ...

Caching results with Joblib

If your model takes time to make predictions, it may be interesting to cache the results: if the prediction for a particular input has already been done, it makes sense to return the same result we saved on disk rather than running the computations again. In this section, we’ll learn how to do this with the help of Joblib.

Joblib provides us with a very convenient and easy-to-use tool to do this, so the implementation is quite straightforward. The main concern will be about whether we should choose standard or async functions to implement the endpoints and dependencies. This will allow us to explain some of the technical details of FastAPI in more detail.

We’ll build upon the example we provided in the previous section. The first thing we must do is initialize a Joblib Memory class, which is the helper for caching function results. Then, we can add a decorator to the functions we want to cache. You can see this in the following example...

Summary

Congratulations! You’re now able to build a fast and efficient REST API to serve your machine learning models. Thanks to Joblib, you learned how to dump a trained scikit-learn estimator into a file that’s easy to load and use inside your application. We also saw an approach to caching prediction results using Joblib. Finally, we discussed how FastAPI handles synchronous operations by sending them to a separate thread to prevent blocking. While this was a bit technical, it’s important to bear this aspect in mind when dealing with blocking I/O operations.

We’re near the end of our FastAPI journey. Before letting you build awesome data science applications by yourself, we will provide three more chapters to push this a bit further and study more complex use cases. We’ll start with an application that can perform real-time object detection, thanks to WebSockets and a computer vision model.