Section 5 – Deep Learning Model Explainability at Scale
In this section, we will learn about the foundational concepts of explainability and explainable artificial intelligence (XAI) and how to implement deep learning (DL) explainability with MLflow. We will start with an overview of the eight dimensions of explainability and then learn how to use SHapley Additive exPlanations (SHAP) and Transformers Interpret to perform explainability for a natural language processing (NLP) pipeline. Furthermore, we will learn and analyze the current MLflow integration with SHAP to understand the trade-offs and avoid potential implementation problems. Then, we will show how to implement SHAP using MLflow's logging APIs. Finally, we will learn how to implement a SHAP explainer as an MLflow Python model and then load it as either a Spark UDF for batch explanation or as a web service for online Explanation-as-a-Service (EaaS).
This section comprises the following chapters:
