Introduction to MLOps and OpenShift

If you have chosen to read this book, chances are that you have a background in the machine learning (ML) domain. The primary purpose of this book is to show you how Red Hat OpenShift provides the basis for developing, deploying, and monitoring your models in production. In addition, you will learn about different components of the OpenShift ecosystem and how you can weave them together to build a path toward automating the life cycle of your ML project. You will also learn how to leverage Red Hat OpenShift Data Science and its partner components.

Finally, you will see how the approaches presented in this book can help your organization scale its ML initiatives through MLOps practices.

This first chapter focuses on giving you the basic definitions of the concepts and the technologies involved in the Red Hat OpenShift ecosystem for machine learning.

This chapter will cover the following topics:

• What is machine learning operations...

What is MLOps?

MLOps is a set of practices that aims to streamline the process of deploying and maintaining ML models in production environments. It involves integrating ML workflows with DevOps practices to ensure that ML models are tested, deployed, and monitored in a reliable and scalable manner.

MLOps involves collaboration between data scientists, ML engineers, software developers, and operations teams to create a continuous integration and delivery pipeline for ML models. This pipeline includes steps such as data preparation, model training, model testing, model deployment, and monitoring. MLOps also involves the use of automation, testing, and monitoring tools to ensure that models remain accurate, stable, and scalable over time.

Some common practices in MLOps include version control for models and data, experiment tracking, model validation, continuous integration and delivery, containerization of models, and monitoring of model performance and data drift.

So, what...

Introduction to OpenShift

Although this book is not about operationalizing the OpenShift platform, a basic introduction to the platform is helpful. In this section, you will learn about the core concepts of Kubernetes and OpenShift.

OpenShift is a complete application platform based on Kubernetes. It is also categorized as Enterprise Kubernetes. Kubernetes provides a solid foundation for container hosting and orchestration. Moreover, Kubernetes provides core functionalities, such as cluster-state management, where a reconcile loop makes sure that the cluster state and the desired state are in sync. Kubernetes also includes a set of APIs to interact with the cluster. Kubernetes is a great platform, but in reality, applications need much more than just the core services provided by Kubernetes.

Assume that you want to deploy a Python application on a Kubernetes cluster. Let’s assess what is required. First, you need to package your application as a container image. Secondly...

Understanding operators

In traditional organizations, specialized and dedicated teams were required to maintain applications and other software components such as databases, caches, and messaging components. Moreover, those teams were continuously observing the software ecosystem and doing specific things, such as taking backups for databases, upgrading and patching newer versions of software components, and more.

Operators in Kubernetes are like system administrators or human operators, continuously monitoring applications running on the Kubernetes environment and performing operational tasks associated with the specific component. In summary, an operator extends Kubernetes to automate the management of the complete life cycle of an application. For example, a PostgreSQL operator automates the database’s high availability, installation, patching, and backup abilities, to name a few. Many operators are available for various software components, such as databases, caches,...

Understanding how OpenShift supports MLOps

As you have seen, an application platform provides an opinionated way of running services on Kubernetes. An example of this is OpenShift, which provides Prometheus and Grafana as monitoring services. A similar approach is applied to the software required to run MLOps on OpenShift. Red Hat and its partners provide MLOps components on top of the OpenShift platform that provides the services for a complete ML platform. Using OpenShift, all the MLOps capabilities can be consistently deployed on-premises and on the cloud.

Just like DevOps, one of the primary objectives of MLOps is to bridge the gap between the engineers who are building the applications – or in this case, the data scientists and ML engineers who are developing ML models – and the operations team. To achieve this, we need to have a common platform where engineers and operators will meet. The best tool we have for this is containerization platforms. This allows both...

Red Hat OpenShift Data Science (RHODS)

In this section, you will explore the components that form the ML platform stack. The technology stack is a combination of Red Hat components, Red Hat partner components, and open source software. It’s called RHODS, and it’s Red Hat’s solution for running data science and ML workloads on OpenShift.

Running RHODS on OpenShift gives the freedom to build and deploy models on-premises or on any cloud. The open source version of RHODS is Open Data Hub (https://opendatahub.io). RHODS provides a subset of the components available in Open Data Hub but in a commercially supported way. The RHODS platform integrates well with technology partners to form a complete MLOps stack.

You will learn about the RHODS platform throughout this book. Let’s start by defining some of its building blocks:

• Model development and tuning: RHODS provides out-of-the-box support for JupyterHub, a powerful and popular multi-user Jupyter...

The advantages of the cloud

Cloud computing provides on-demand delivery of IT resources such as servers, networks, storage, databases, and all higher-level application services via the internet with a pay-as-you-go pricing model. The prominent vendors for the cloud are Amazon Web Services (AWS), Microsoft Azure, and Google Cloud, and they all provide usage-based pricing.

The cloud has many benefits, but the top three are agility, innovation, and cost savings. Let’s take a brief look at all three.

Agility ensures that the teams can experiment with new ideas quickly and frequently. That’s because the speed your team can deliver software is higher when the team doesn’t wait for IT to provision infrastructure. This makes the foundation of autonomous product teams, where they can release products, collect feedback, and improve the software to meet your customers’ needs.

Agility powers your team to focus on business competence instead of using the time...

ROSA

ROSA is a fully managed turnkey application platform that enables you to focus on delivering value to your customers. ROSA is a service operated by Red Hat and jointly supported with AWS to provide a fully managed OpenShift platform. Red Hat and AWS teams work on the underlying infrastructure for you to reduce the burden of managing infrastructure. ROSA also includes integration with a wide range of AWS services, such as databases and mobile, to help you further accelerate building solutions for your organization.

The following are some of the benefits of the ROSA platform:

• If you are running OpenShift on-premises, ROSA provides the fastest path to running OpenShift in the cloud. ROSA provides production-ready OpenShift as a service on the AWS platform.
• Red Hat and AWS jointly operate and support ROSA. This provides an integrated support channel for you with a 99.95% uptime. (The 99.95% uptime is quoted from the vendor's site and may change).
• ROSA provides...

Summary

In this chapter, you learned about the problems MLOps aims to tackle and how it can increase the velocity of your data science initiatives. You also refreshed your knowledge of Kubernetes and OpenShift and saw how Red Hat OpenShift provides a consistent and reliable environment where you can run your container workloads on-premises and in the cloud. You have seen how RHODS, using the strengths of the underlying container platform, provides a full set of components for an MLOps platform.

In the next chapter, you will learn about the stages of the ML life cycle, as well as the role MLOps plays in implementing all the stages of model development and deployment. You will also see how teams collaborate during model development and deployment stages and how RHODS components relate to each stage of the ML life cycle.

References

To learn more about the topics that were covered in this chapter, take a look at the following resources:

• Gartner identifies AI Engineering as a top trend for 2021 and beyond. https://www.gartner.com/en/newsroom/press-releases/2020-10-19-gartner-identifies-the-top-strategic-technology-trends-for-2021
• MLflow, an open-source tool, supports model versioning, deployment, and sharing model performance data. https://mlflow.org
• Trino, an open-source distributed query engine, facilitates fetching data from diverse sources for model training. https://trino.io
• Run.ai offers certified software for pooling and sharing GPU resources, addressing the cost challenges of training models. https://www.run.ai/
• OpenVINO optimizes models for Intel-based hardware, from data centers to edge devices like IoT devices and phones. https://docs.openvino.ai/latest/home.html