Implementing drift detection on Databricks
The necessary files for this chapter are located within the Chapter-09 folder. This example demonstrates how you can arrange your code into specific modules to keep it organized.
Figure 9.8 – A screenshot showing the layout of the files in our code base
The setup notebook in the config folder is designed to establish the folder structure for data reading and writing. It also sets up the MLflow experiment for tracking model performance over time and manages other variables that will be utilized in our model-drift notebook.
The datagen notebook within the data folder serves the purpose of creating a synthetic dataset that effectively demonstrates the concept of model drift. This dataset encompasses time series data of online sales for an e-commerce website spanning three months.
In this dataset, we have a set of independent features and a target feature, along with simulated relationships between them...
