How this all comes together – machine learning pipelines

In this chapter, we explored the main characteristics of machine learning systems and compared them to traditional software systems. Let’s finish this comparison by summarizing how we usually design and describe machine learning systems – by using pipelines. A pipeline is a sequence of data processing steps, including the machine learning models. The typical set of steps (also called phases) is shown in Figure 2.14:

Figure 2.14 – A typical sequence of steps in a machine learning pipeline

This kind of pipeline, although drawn linearly, is usually processed in cycles, where, for example, monitoring for concept drift can trigger re-training, re-testing, and re-deployment.

Machine learning pipelines, just like the one presented in Figure 2.14, are often depicted as a set of components as parts of the entire system. However, presenting it using the pipeline analogy helps...