Executing the build

Execution of the build, in this case, will be very much about how we take the proof-of-concept code shown in Chapter 1, Introduction to ML Engineering, and then split this out into components that can be called by another scheduling tool such as Apache Airflow.

This will provide a showcase of how we can apply some of the ML engineering skills we learned throughout the book. In the next few sections, we will focus on how to build out an Airflow pipeline that leverages a series of different ML capabilities, creating a relatively complex solution in just a few lines of code.

Building an ETML pipeline with advanced Airflow features

We already discussed Airflow in detail in Chapter 5, Deployment Patterns and Tools, but there we covered more of the details around how to deploy your DAGs on the cloud. Here we will focus on building in more advanced capabilities and control flows into your DAGs. We will work locally here on the understanding that when you...