Open-Source ML Platforms

In the previous chapter, we covered how Kubernetes can be used as the foundational infrastructure for running ML tasks, such as running model training jobs or building data science environments such as Jupyter Notebook servers. However, to perform these tasks at scale and more efficiently for large organizations, you will need to build ML platforms with the capabilities to support the full data science lifecycle. These capabilities include scalable data science environments, model training services, model registries, and model deployment capabilities.

In this chapter, we will discuss the core components of an ML platform and explore additional open-source technologies that can be used for building ML platforms. We will begin with technologies designed for building a data science environment capable of supporting a large number of users for experimentation. Subsequently, we will delve into various technologies for model training, model registries, model...

Core components of an ML platform

An ML platform is a complex system encompassing multiple environments for running distinct tasks and orchestrating complex workflow processes. Furthermore, an ML platform needs to cater to a multitude of roles, including data scientists, ML engineers, infrastructure engineers, operations teams, and security and compliance stakeholders. To construct an ML platform, several components come into play.

These components include:

• Data science environment: The data science environment provides data analysis and ML tools, such as Jupyter notebooks, data sources and storage, code repositories, and ML frameworks. Data scientists and ML engineers use the data science environment to perform data analysis, run data science experiments, and build and tune models. The data science environment also provides collaboration capabilities, allowing data scientists to share and collaborate on code, data, experiments, and models.
• Model training environment...

Open-source technologies for building ML platforms

Managing ML tasks individually by deploying standalone ML containers in a Kubernetes cluster can become challenging when dealing with a large number of users and workloads. To address this complexity and enable efficient scaling, many open-source technologies have emerged as viable solutions. These technologies, including Kubeflow, MLflow, Seldon Core, GitHub, Feast, and Airflow, provide comprehensive support for building data science environments, model training services, model inference services, and ML workflow automation.

Before delving into the technical details, let’s first explore why numerous organizations opt for open-source technologies to construct their ML platforms. For many, the appeal lies in the ability to tailor the platform to specific organizational needs and workflows, with open standards and interoperable components preventing vendor lock-in and allowing the flexibility to adopt new technologies over...

Designing an end-to-end ML platform

After discussing several open-source technologies individually, let’s now delve into their integration and see how these components come together. The architecture patterns and technology stack selection may vary based on specific needs and requirements. The following diagram presents the conceptual building blocks of an ML platform architecture:

Figure 7.12: ML platform architecture

Next, let’s delve into different strategies to implement this architecture concept with different combinations of open-source technologies.

ML platform-based strategy

When designing an ML platform using open-source technologies, one effective strategy is to utilize an ML platform framework as a base platform and then integrate additional open-source components to address specific requirements. One such ML platform framework is Kubeflow, which provides a robust foundation with its built-in building blocks for an ML platform. By leveraging...

Summary

In this chapter, you have gained an understanding of the core architecture components of a typical ML platform and their capabilities. We have explored various open-source technologies such as Kubeflow, MLflow, TensorFlow Serving, Seldon Core, Triton Inference Server, Apache Airflow, and Kubeflow Pipelines. Additionally, we have discussed different strategies for approaching the design of an ML platform using open-source frameworks and tools.

While these open-source technologies offer powerful features for building sophisticated ML platforms, it is important to acknowledge that constructing and maintaining such environments requires substantial engineering effort and expertise, especially when dealing with large-scale ML platforms.

In the next chapter, we will delve into fully managed, purpose-built ML solutions that are specifically designed to facilitate the development and operation of ML environments. These managed solutions aim to simplify the complexities of...