Implementing a SHAP explainer using the MLflow pyfunc API
As we know from the previous section, a SHAP explainer can be used offline whenever needed by creating a new instance of an explainer using SHAP APIs. However, as the underlying DL models are often logged into the MLflow server, it is desirable to also log the corresponding explainer into the MLflow server, so that we not only keep track of the DL models, but also their explainers. In addition, we can use the generic MLflow pyfunc model logging and loading APIs for the explainer, thus unifying access to DL models and their explainers.
In this section, we will learn step-by-step how to implement a SHAP explainer as a generic MLflow pyfunc model and how to use it for offline and online explanation. We will break the process up into three subsections:
- Creating and logging an MLflow pyfunc explainer
- Deploying an MLflow pyfunc explainer for an EaaS
- Using an MLflow pyfunc explainer for batching explanation
