Exploring the SHAP Explainability toolbox

For our learning purpose, let's review some popular explainability toolboxes while experimenting with some examples. Based on the number of GitHub stars (16,000 as of April 2022, https://github.com/slundberg/shap), SHAP is the most widely used and integrated open source model explainability toolbox. It is also the foundation explanation tool that is integrated with MLflow. Here, we would like to run a small experiment to get some hands-on experience on how this works. Let's use a sentimental analysis NLP model to explore how SHAP can be used for explaining the model behavior:

1. Set up the virtual environment on your local environment after checking out this chapter's code from GitHub. Running the following command will create a new virtual environment called dl-explain:

   conda env create -f conda.yaml

This will install SHAP and its related dependencies such as matplotlib in this virtual environment. Once this virtual...