Techniques for drift detection
To ensure effective monitoring of our model’s performance over time, we should track changes in summary statistics and distributions of both the model features and target variables. This will enable us to detect any potential data drift early on.
Furthermore, it’s important to monitor offline model metrics such as accuracy and F1 scores that were utilized during the initial training of the model.
Lastly, we should also keep an eye on online metrics or business metrics to ensure that our model remains relevant to the specific business problem we are trying to solve.
The following table provides an overview of various statistical tests and methods that can be employed to identify drift in your data and models. Please note that this compilation is not all-encompassing.
|
Data Type to Monitor |
Sub-Category |
Statistical Measures and Tests ... |
