Vertex AI Deployment and Automation Tools – Orchestration through Managed Kubeflow Pipelines

In a typical machine learning (ML) solution, we often have lots of applications and services as part of the end-to-end workflow. If we try to stitch these services and applications together using some custom scripts with cron jobs, it becomes super tricky to manage the workflows. Thus, it becomes important to make use of some orchestration services to carefully manage, scale, and monitor complex workflows. Orchestration is the process of stitching multiple applications or services together to build an end-to-end solution workflow. Google Cloud provides multiple orchestration services, such as Cloud Scheduler, Workflows, and Cloud Composer, to manage complex workflows at scale. Cloud Scheduler is ideal for single, repetitive tasks, Workflows is more suitable for complex multi-service orchestration, and Cloud Composer is ideal for data-driven workloads.

ML workflows have a lot of steps...

Technical requirements

The code examples shown in this chapter can be found in the following GitHub repo:https://github.com/PacktPublishing/The-Definitive-Guide-to-Google-Vertex-AI/tree/main/Chapter10

Orchestrating ML workflows using Vertex AI Pipelines (managed Kubeflow pipelines)

ML solutions are complex and involve lots of steps, including data preparation, feature engineering, model selection, model training, testing, evaluation, and deployment. On top of these, it is really important to track and version control lots of aspects related to the ML model while in production. Vertex AI Pipelines on GCP lets us codify our ML workflows in such a way that they are easily composable, shareable, and reproducible. Vertex AI Pipelines can run Kubeflow as well as TensorFlow Extended (TFX)-based ML pipelines in a fully managed way. In this section, we will learn about developing Kubeflow pipelines for ML development as Vertex AI Pipelines.

Kubeflow is a Kubernetes-native solution that simplifies the orchestration of ML pipelines and makes experimentation easy and reproducible. Also, the pipelines are sharable. It comes with framework support for things such as execution monitoring, workflow...

Orchestrating ML workflows using Cloud Composer (managed Airflow)

Cloud Composer is a workflow orchestration service on Google Cloud that is built upon the open source project of Apache Airflow. The key difference is that Composer is fully managed and also integrates with other GCP tooling very easily. With Cloud Composer, we can write, execute, schedule, or monitor our workflows that are also supported across multi-cloud and hybrid environments. Composer pipelines are DAGs that can be easily defined and configured using Python. It comes with a rich library of connectors that let us deploy our workflows instantly with one click. Graphical representations of workflows on the Google Cloud console make monitoring and troubleshooting quite convenient. Automatic synchronization of our DAGs ensures that our jobs always stay on schedule.

Cloud Composer is commonly used by data scientists and data engineers to build complex data pipelines (ETL or ELT pipelines). It can also be used as an...

Vertex AI Pipelines versus Cloud Composer

In this section, we will talk about some of the key similarities and differences between Vertex AI Pipelines and Cloud Composer when it comes to orchestrating ML workflows. Based on this comparison, we can choose the best solution for our next ML project. The following is a list of points that summarize the important aspects of both orchestrators for ML-related tasks:

• Both are easy to use and divide the overall ML workflow into smaller execution units in terms of tasks (Composer) or containerized components (Vertex AI Pipelines).
• Passing data between components is similar, and it may require an intermediate storage system if the data size is large.
• Vertex AI Pipelines have an extensive list of prebuilt components available open source and thus developers can avoid writing a lot of boilerplate code. On the other hand, in the case of a Composer-based pipeline, we need to write the entire workflow.
• Based on the ease of setting...

Getting predictions on Vertex AI

In this section, we will learn how to get predictions from our ML models on Vertex AI. Depending on the use case, prediction requests can be of two types – online predictions (real time) and batch predictions. Online predictions are synchronous requests made to a model endpoint. Online predictions are needed by applications that keep requesting outputs for given inputs in a timely manner via an API call in order to update information for end users in near real time. For example, the Google Maps API gives us near real-time traffic updates and requires online prediction requests. Batch predictions, on the other hand, are asynchronous requests. If our use case only requires batch prediction, we might not need to deploy the model to an endpoint as the Vertex AI batchprediciton service also allows us to perform batch prediction from a saved model that is present in a GCS location without even needing to create an endpoint. Batch predictions are suitable...

Managing deployed models on Vertex AI

When we deploy an ML model to an endpoint, we associate it with physical resources (compute) so that it can serve online predictions at low latency. Depending on the requirements, we might want to deploy multiple models to a single endpoint or a single model to multiple endpoints as well. Let’s learn about these two scenarios.

Multiple models – single endpoint

Suppose we already have one model deployed to an endpoint in production and we have found some interesting ideas to improve that model. Now, suppose we have already trained an improved model that we want to deploy but we also don’t want to make any sudden changes to our application. In this situation, we can add our latest model to the existing endpoint and start serving a very small percentage of traffic with the new model. If everything looks great, we can gradually increase the traffic until it is serving the full 100% of the traffic.

Single model –...

Summary

In this chapter, we have learned about two popular ML workflow orchestration tools – Vertex AI Pipelines (managed Kubeflow) and Cloud Composer (managed Airflow). We have also implemented a Vertex Pipeline for an example use case, and similarly, we have also developed and executed an example DAG with Cloud Composer. Both Vertex AI Pipelines and Cloud Composer are managed services on GCP and make it really easy to set up and launch complex ML and data-related workflows. Finally, we have learned about getting online and batch predictions on Vertex AI for our custom models, including some best practices related to model deployments.

After reading this chapter, you should have a good understanding of different ways of carrying out ML workflow orchestration on GCP and their similarities and differences. Now, you should be able to write your own ML workflows and orchestrate them on GCP via either Vertex AI Pipelines or Cloud Composer. Finally, you should also be confident...