Identifying influential features with factor prioritization

The Morris Method is one of several global sensitivity analysis methods that range from simpler Fractional Factorial to complicated Monte Carlo Filtering. Morris is somewhere in-between this spectrum, falling into two categories. It uses one-at-a-time sampling, which means that only one value changes between consecutive simulations. It's also elementary effects (EE), which means that it doesn't quantify the exact effect of a factor in a model but rather gauges its importance and relationship with other factors. By the way, factor is just another word for a feature or variable that's commonly used in applied statistics. To be consistent with the related theory, we will use this word in this and the next section.

Another property of Morris is that it's less computationally expensive than the variance-based methods we will study next. It can provide more insights than simpler and less costly methods such as regression...

Quantifying uncertainty and cost sensitivity with factor fixing

With the Morris indices, it became evident that all the factors are non-linear or non-monotonic. There's a high degree of interactivity between them – as expected! It should be no surprise that climate factors (temp, rain_1h, snow_1h, and cloud_coverage) are likely multicollinear with hr. There are also patterns to be found between hr, is_holiday, and dow and the target. Many of these factors most definitely don't have a monotonic relationship with the target. We know this already. For instance, traffic doesn't consistently increase as hours increase throughout the day. That's not the case between days of the week either!

However, we didn't know to what degree is_holiday and temp impacted the model, particularly during the crew's working hours, which was an important insight. That being said, factor prioritization with Morris indices is usually to be taken as a starting point or "...

Mission accomplished

The mission was to train a traffic prediction model and understand what factors create uncertainty and possibly increase costs for the construction company. We can conclude a significant portion of the potential $35,000/year in fines can be attributed to the is_holiday factor. Therefore, the construction company should rethink working holidays. There are only seven or eight holidays between March and November, and they could cost more because of the fines than working on a few Sundays instead. With this caveat, the mission was successful, but there's still a lot of room for improvement.

Of course, these conclusions are for the “LSTM_traffic_168_compact1” model – which we can compare with other models. Try replacing the model_name at the beginning of the notebook with “LSTM_traffic_168_compact2”, an equally small but significantly more robust model, or “LSTM_traffic_168_optimal”, a larger slightly better performing...

Summary

After reading this chapter, you should understand how to assess a time series model's predictive performance, know how to perform local interpretations for them with integrated gradients, and know how to produce both local and global attributions with SHAP. You should also know how to leverage sensitivity analysis factor prioritization and factor fixing for any model.

In the next chapter, we will learn how to reduce complexity in a model and make it more interpretable with feature selection and engineering.

Dataset and image sources

• TomTom. (2019). Traffic Index: https://www.tomtom.com/en_gb/traffic-index/ranking/?congestion=WORST,BAD,MODERATE
• UCI Machine Learning Repository (2019). Metro Interstate Traffic Volume Data Set:https://archive.ics.uci.edu/ml/datasets/Metro+Interstate+Traffic+Volume

References

• Wilson, D.R., & Martinez, T. (1997). Improved Heterogeneous Distance Functions. J. Artif. Int. Res. 6-1. pp.1-34.https://arxiv.org/abs/cs/9701101
• Morris, M. (1991). Factorial sampling plans for preliminary computational experiments. Quality Engineering, 37, 307-310. https://doi.org/10.2307%2F1269043
• Saltelli, A., Tarantola, S., Campolongo, F., & Ratto, M. (2007). Sensitivity analysis in practice: A guide to assessing scientific models. Chichester: John Wiley & Sons.
• Sobol, I.M. (2001), Global sensitivity indices for nonlinear mathematical models and their Monte Carlo estimates. MATH COMPUT SIMULAT,55(1–3),271-280 https://doi.org/10.1016/S0378-4754(00)00270-6
• Saltelli, A., P. Annoni, I. Azzini, F. Campolongo, M. Ratto, and S. Tarantola (2010). "Variance based sensitivity analysis of model output. Design and estimator for the total sensitivity index." Computer Physics Communications, 181(2):259-270. https://doi.org/10.1016/j.cpc.2009.09.018

...

Evaluating misclassifications with gradient-based attribution methods

Gradient-based methods calculate attribution maps for each classification with both forward and background passes through the CNN. As the name suggests, these methods leverage the gradients in the backward pass to compute the attribution maps. All of these methods are local interpretation methods because they only derive a single interpretation per sample. Incidentally, attributions in this context mean that we are attributing the predicted labels to areas of an image. They are often called sensitivity maps in academic literature, too.

To get started, we will first need to create an array with all of our misclassification samples (X_misclass) from the test dataset (test_data) using the combined indexes for all of our misclassifications of interest (misclass_idxs). Since there aren’t that many misclassifications, we are loading a single batch of them (next):

misclass_idxs = metal_FP_idxs + plastic_FN_idxs...

Understanding classifications with perturbation-based attribution methods

Perturbation-based methods have already been covered to a great extent in this book so far. So many of the methods we have covered, including SHAP, LIME, anchors, and even permutation feature importance, employ perturbation-based strategies. The intuition behind them is that if you remove, alter, or mask features in your input data and then make predictions with them, you’ll be able to attribute the difference between the new predictions and the original predictions to the changes you made in the input. These strategies can be leveraged in both global and local interpretation methods.

We will now do the same as we did with the misclassification samples, but to the chosen true positives, and gather four of each class in a single tensor (X_correctcls):

correctcls_idxs = wglass_TP_idxs[:4] + battery_TP_idxs[:4] 
correctcls_data = torch.utils.data.Subset(test_data, correctcls_idxs)
correctcls_loader...

Mission accomplished

The mission was to provide an objective evaluation of the garbage classification model for the municipal recycling plant. The predictive performance on out-of-sample validation images was dismal! You could have stopped there, but then you would not have known how to make a better model.

However, the predictive performance evaluation was instrumental in deriving specific misclassifications, as well as correct classifications, to assess using other interpretation methods. To this end, you ran a comprehensive suite of interpretation methods, including activation, gradient, perturbation, and backpropagation-based methods. The consensus between all the methods was that the model was having the following issues:

• Differentiating between the background and the objects
• Understanding that different objects share similar color hues
• Confounding lighting conditions, such as specular highlights as specific material characteristics, like with the wine...

Summary

After reading this chapter, you should understand how to leverage traditional interpretation methods to more thoroughly assess predictive performance on a CNN classifier and visualize the learning process of CNNs with activation-based methods. You should also understand how to compare and contrast misclassifications and true positives with gradient-based and perturbation-based attribution methods. In the next chapter, we will study interpretation methods for NLP transformers.

Further reading

• Smilkov, D., Thorat, N., Kim, B., Viégas, F., and Wattenberg, M., 2017, SmoothGrad: Removing noise by adding noise. ArXiv, abs/1706.03825: https://arxiv.org/abs/1706.03825
• Sundararajan, M., Taly, A., and Yan, Q., 2017, Axiomatic Attribution for Deep Networks. Proceedings of Machine Learning Research, pp. 3319–3328, International Convention Centre, Sydney, Australia: https://arxiv.org/abs/1703.01365
• Zeiler, M.D., and Fergus, R., 2014, Visualizing and Understanding Convolutional Networks. In European conference on computer vision, pp. 818–833: https://arxiv.org/abs/1311.2901
• Shrikumar, A., Greenside, P., and Kundaje, A., 2017, Learning Important Features Through Propagating Activation Differences: https://arxiv.org/abs/1704.02685

Learn more on Discord

To join the Discord community for this book – where you can share feedback, ask the author questions, and learn about new releases – follow the QR code...