Deploying Machine Learning Applications

In the previous chapters, we learned how to create an application that can prepare data (Chapter 2, Setting Up the Development Environment) for either a supervised (Chapter 3, Supervised Learning) or unsupervised (Chapter 4, Unsupervised Learning) ML algorithm. We also learned how to evaluate and test the output of these algorithms with the added complication that we have incomplete knowledge about the algorithm's inner state and workings, and must therefore treat it as a black box. In Chapter 5, Using Pre-Trained Models, we looked at model persistence and how Go applications can leverage models written in other languages. Together, the skills you have learned so far constitute the fundamentals required to successfully prototype ML applications. In this chapter, we will look at how to prepare your prototype for commercial readiness...

The continuous delivery feedback loop

Continuous delivery (CD) is the practice of using short feedback loops in the software development life cycle to ensure that the resulting application can be released at any moment in time^[1]. While there are alternative approaches to release management, we will only consider this one because creating a meaningful, short—and therefore automated—feedback loop with ML applications presents unique challenges that are not created by alternative methodologies that may not require this degree of automation.

The CD feedback loop consists of the following process:

Fig. 1: The continuous delivery feedback loop

Developing

The development portion of the feedback loop is what we have...

Deployment models for ML applications

In the preceding example, we explained how to deploy an ML application using Docker to encompass it and its dependencies. We deliberately stayed away from any discussion pertaining to the infrastructure that was going to run these containers or any Platform-as-a-Service offerings that could facilitate the development or deployment itself. In the current section, we consider different deployment models for ML applications under the assumption that the application will be deployed to a cloud platform that supports both IAAS and platform-as-a-service models, such as Microsoft Azure and Amazon Web Services.

This section is specifically written to help you decide what virtual infrastructure to use if you are deploying an ML application to the cloud.

There are two main deployment models for any cloud application:

• Infrastructure-as-a-service:...

Summary

In this chapter, we discussed how to take a prototype ML application to production. Along the way, we explored concerns that a software developer or DevOps engineer would typically think of, but from an ML application developers point of view. Specifically, we learned how to apply a continuous development life cycle to an ML application and the different ways to deploy ML applications in the cloud.

In the next and final chapter, we will take a step back and look at ML development from a project management point of view.

Further readings

1. Continuous Software Engineering and Beyond: Trends and Challenges Brian Fitzgerald, 1st International Workshop on Rapid Continuous Software Engineering. New York, NY: Association for Computing Machinery, pp. 1–9.
2. Google's solution to accidental algorithmic racism: ban gorillas: https://www.theguardian.com/technology/2018/jan/12/google-racism-ban-gorilla-black-people. Retrieved May 3, 2019.
3. Building Numpy from source: http://robpatro.com/blog/?p=47. Retrieved May 5, 2019.
4. Python—Compiling Numpy with OpenBLAS integration: https://stackoverflow.com/questions/11443302/compiling-numpy-with-openblas-integration. Retrieved May 5, 2019.
5. Issues—TensorFlow: https://github.com/tensorflow/tensorflow/issues. Retrieved May 5, 2019.
6. Python Wheels: https://pythonwheels.com/. Retrieved May 5, 2019.
7. Chocolateay—The Package Manager for Windows...