Going through the basics of ML pipelines
Before we jump into the implementation of the ML pipeline, let's get the basics right. We will reflect on ML pipelines and set up the needed resources for ML pipeline implementation and then we will get started with data ingestion. Let's demystify ML pipelines by reflecting on the ML pipeline we discussed in Figure 14 of Chapter 1, Fundamentals of MLOps Workflow.
Figure 4.1 – Machine learning pipeline
As shown in Figure 4.1, a comprehensive ML pipeline consists of the following steps:
- Data ingestion
- Model training
- Model testing
- Model packaging
- Model registering
We will implement all these steps of the pipeline using the Azure ML service (cloud-based) and MLflow (open source) simultaneously for the sake of a diverse perspective. Azure ML and MLflow are a power couple for MLOps: they exhibit the features shown in Table 4.1. They are also unique in their capabilities...
