Integrating ML Systems in Ecosystems

ML systems have gained a lot of popularity for two reasons – their ability to learn from data (which we’ve explored throughout this book), and their ability to be packaged into web services.

Packaging these ML systems into web services allows us to integrate them into workflows in a very flexible way. Instead of compiling or using dynamically linked libraries, we can deploy ML components that communicate over HTTP protocols using JSON protocols. We have already seen how to use that protocol by using the GPT-3 model that is hosted by OpenAI. In this chapter, we’ll explore the possibility of creating a Docker container with a pre-trained ML model, deploying it, and integrating it with other components.

In this chapter, we’re going to cover the following main topics:

• ML system of systems – software ecosystems
• Creating web services over ML models using Flask
• Deploying ML models using Docker
...

Ecosystems

In the dynamic realm of software engineering, the tools, methodologies, and paradigms are in a constant state of evolution. Among the most influential forces driving this transformation is ML. While ML itself is a marvel of computational prowess, its true genius emerges when integrated into the broader software engineering ecosystems. This chapter delves into the nuances of embedding ML within an ecosystem. Ecosystems are groups of software that work together but are not connected at compile time. A well-known ecosystem is the PyTorch ecosystem, where a set of libraries work together in the context of ML. However, there is much more than that to ML ecosystems in software engineering.

From automated testing systems that learn from each iteration to recommendation engines that adapt to user behaviors, ML is redefining how software is designed, developed, and deployed. However, integrating ML into software engineering is not a mere plug-and-play operation. It demands a rethinking...

Creating web services over ML models using Flask

In this book, we’ve mostly focused on training, evaluating, and deploying ML models. However, we did not discuss the need to structure them flexibly. We worked with monolithic software. Monolithic software is characterized by its unified, single code base structure where all the functionalities, from the user interface to data processing, are tightly interwoven and operate as one cohesive unit. This design simplifies initial development and deployment since everything is bundled together and they are compiled together. Any change, however minor, requires the entire application to be rebuilt and redeployed. This makes it problematic when the evolution of contemporary software is fast.

On the other hand, web service-based software, which is often associated with microservices architecture, breaks down the application into smaller, independent services that communicate over the web, typically using protocols such as HTTP and REST...

Deploying ML models using Docker

To create a Docker container with our newly created web service (or two of them), we need to install Docker on our system. Once we’ve installed Docker, we can use it to compile the container.

The crucial part of packaging the web service into the Docker container is the Dockerfile. It is a recipe for how to assemble the container and how to start it. If you’re interested, I’ve suggested a good book about Docker containers in the Further reading section so that you can learn more about how to create more advanced components than the ones in this book.

In our example, we need two containers. The first one will be the container for the measurement instrument. The code for that container is as follows:

FROM alpine:latest
RUN apk update
RUN apk add py-pip
RUN apk add --no-cache python3-dev
RUN pip install --upgrade pip
WORKDIR /app
COPY . /app
RUN pip --no-cache-dir install -r requirements.txt
CMD ["python3", "...

Combining web services into ecosystems

Now, let’s develop the software that will connect these two web services. For this, we’ll create a new file that will send one file to the first web service, get the data, and then send it to the second web service to make predictions:

import requests
# URL of the Flask web service for file upload
upload_url = 'http://localhost:5000/success'  # Replace with the actual URL
# URL of the Flask web service for predictions
prediction_url = 'http://localhost:5001/predict/'  # Replace with the actual URL
def upload_file_and_get_metrics(file_path):
    try:
        # Open and read the file
        with open(file_path, 'rb') as file:
            # Create a dictionary to hold the file data
       ...

Summary

In this chapter, we learned how to deploy ML models using web services and Docker. Although we only deployed two web services, we can see that it can become an ecosystem for ML. By separating predictions and measurements, we can separate the computational-heavy workloads (prediction) and the data collection parts of the pipeline. Since the model can be deployed on any server, we can reuse the servers and therefore reduce the energy consumption of these models.

With that, we have come to was last technical chapter of this book. In the next chapter, we’ll take a look at the newest trends in ML and peer into our crystal ball to predict, or at least guess, the future.

References

• Masse, M., REST API design rulebook: designing consistent RESTful web service interfaces. 2011: “O’Reilly Media, Inc.”.
• Raj, P., J.S. Chelladhurai, and V. Singh, Learning Docker. 2015: Packt Publishing Ltd.
• Staron, M., et al. Robust Machine Learning in Critical Care—Software Engineering and Medical Perspectives. In 2021 IEEE/ACM 1st Workshop on AI Engineering-Software Engineering for AI (WAIN). 2021. IEEE.
• McCabe, T.J., A complexity measure. IEEE Transactions on Software Engineering, 1976(4): p. 308-320.