Building the model factory with pipelines

The concept of a software pipeline is intuitive enough. If you have a series of steps chained together in your code, so that the next step consumes or uses the output of the previous step or steps, then you have a pipeline.

In this section, when we refer to a pipeline, we will be specifically dealing with steps that contain processing or calculations that are appropriate to ML. For example, the following diagram shows how this concept may apply to some of the steps the marketing classifier mentioned in Chapter 1, Introduction to ML Engineering:

Figure 3.13: The main stages of any training pipeline and how this maps to a specific case from Chapter 1, Introduction to ML Engineering.

Let’s discuss some of the standard tools for building up your ML pipelines in code.

Scikit-learn pipelines

Our old friend Scikit-Learn comes packaged with some nice pipelining functionality. The API is extremely easy to use, as you...