Causal Forests and more

In this short section, we’ll provide a brief overview of the idea behind Causal Forests. We’ll introduce one of the EconML classes implementing the method. An in-depth discussion on Causal Forests and their extensions is beyond the scope of this book, but we’ll point to resources where you can learn more about forest-based causal estimators.

Causal Forest is a tree-based model that stems from the works of Susan Athey, Julie Tibshirani, and Stefan Wager (Wager & Athey, 2018; Athey et al., 2019). The core difference between regular random forest and Causal Forest is that Causal Forest uses so-called causal trees. Otherwise, the methods are similar and both use resampling, predictor subsetting, and averaging over a number of trees.

Causal trees

What makes causal trees different from regular trees is the split criterion. Causal trees use a criterion based on the estimated treatment effects, using so-called honest splitting, where...

Heterogeneous treatment effects with experimental data – the uplift odyssey

Modeling treatment effects with experimental data is usually slightly different in spirit from working with observational data. This stems from the fact that experimental data is assumed to be unconfounded by design (assuming our experimental design and implementation were not flawed).

In this section, we’ll walk through a workflow of working with experimental data using EconML. We’ll learn how to use EconML’s basic API and see how to work with discrete treatments that have more than two levels. Finally, we’ll use some causal model evaluation metrics in order to compare the models.

The title of this section talks about heterogeneous treatment effects – we already know what they are, but there’s also a new term: uplift. Uplift modeling and heterogeneous (aka conditional) treatment effect modeling are closely related terms. In marketing and medicine, uplift...

Extra – counterfactual explanations

Imagine that you apply for a loan from your bank. You prepared well – you checked your credit score and other variables that could affect the bank’s decision. You’re pretty sure that your application will be approved.

On Wednesday morning, you see an email from your bank in your inbox. You’re extremely excited! You open the message, already celebrating and ready to welcome the success!

There’s a surprise in the email.

Your loan application has been rejected.

You call the bank. You ask questions. You want to understand why. At the end of the day, its decision impacts some of your most important plans!

The only response you get from the customer service representative is that you did not meet the criteria. “Which criteria?” you ask. You don’t get a satisfying answer.

You’d like to make sure that you meet the criteria the next time you re-apply, yet it seems that...

Wrapping it up

Congratulations! You just reached the end of Chapter 10.

In this chapter, we introduced four new causal estimators: DR-Learner, TMLE, DML, and Causal Forest. We used two of them on our synthetic earnings dataset, comparing their performance to the meta-learners from Chapter 9.

After that, we learned about the differences in workflows between observational and experimental data and fit six different models to the Hillstrom dataset. We discussed popular metrics used to evaluate uplift models and learned how to use confidence intervals for EconML estimators. We discussed when using machine learning models for heterogeneous treatment effects can be beneficial from an experimental point of view. Finally, we summarized the differences between different models and closed the chapter with a short discussion on counterfactual model explanations.

In the next chapter, we’ll continue our journey through the land of causal inference with machine learning, and with...

References

Athey, S., Tibshirani, J., & Wager, S. (2018). Generalized random forests. The Annals of Statistics, 47(2). 1148-1178.

Balestriero, R., Pesenti, J., & LeCun, Y. (2021). Learning in High Dimension Always Amounts to Extrapolation. arXiv, abs/2110.09485.

Barr, D. J. (2008). Analyzing “visual world” eyetracking data using multilevel logistic regression. J. Mem. Lang. 59, 457-474.

Baayen, R. H., Davidson, D. J., and Bates, D. M. (2008). Mixed-effects modeling with crossed random effects for subjects and items. J. Mem. Lang. 59, 390-412.

Bühlmann, P. & van de Geer, S. A. (2011). Statistics for High-Dimensional Data. Springer.

Cassel, C.M., Särndal, C., & Wretman, J. H. (1976). Some results on generalized difference estimation and generalized regression estimation for finite populations. Biometrika, 63, 615-620.

Chernozhukov, V., Chetverikov, D., Demirer, M., Duflo, E., Hansen, C., Newey, W., & Robins, J. M. (2018)....

Extra – counterfactual explanations

Imagine that you apply for a loan from your bank. You prepared well – you checked your credit score and other variables that could affect the bank’s decision. You’re pretty sure that your application will be approved.

On Wednesday morning, you see an email from your bank in your inbox. You’re extremely excited! You open the message, already celebrating and ready to welcome the success!

There’s a surprise in the email.

Your loan application has been rejected.

You call the bank. You ask questions. You want to understand why. At the end of the day, its decision impacts some of your most important plans!

The only response you get from the customer service representative is that you did not meet the criteria. “Which criteria?” you ask. You don’t get a satisfying answer.

You’d like to make sure that you meet the criteria the next time you re-apply, yet it seems that...

Wrapping it up

Congratulations! You just reached the end of Chapter 10.

In this chapter, we introduced four new causal estimators: DR-Learner, TMLE, DML, and Causal Forest. We used two of them on our synthetic earnings dataset, comparing their performance to the meta-learners from Chapter 9.

After that, we learned about the differences in workflows between observational and experimental data and fit six different models to the Hillstrom dataset. We discussed popular metrics used to evaluate uplift models and learned how to use confidence intervals for EconML estimators. We discussed when using machine learning models for heterogeneous treatment effects can be beneficial from an experimental point of view. Finally, we summarized the differences between different models and closed the chapter with a short discussion on counterfactual model explanations.

In the next chapter, we’ll continue our journey through the land of causal inference with machine learning, and with...

References

Athey, S., Tibshirani, J., & Wager, S. (2018). Generalized random forests. The Annals of Statistics, 47(2). 1148-1178.

Balestriero, R., Pesenti, J., & LeCun, Y. (2021). Learning in High Dimension Always Amounts to Extrapolation. arXiv, abs/2110.09485.

Barr, D. J. (2008). Analyzing “visual world” eyetracking data using multilevel logistic regression. J. Mem. Lang. 59, 457-474.

Baayen, R. H., Davidson, D. J., and Bates, D. M. (2008). Mixed-effects modeling with crossed random effects for subjects and items. J. Mem. Lang. 59, 390-412.

Bühlmann, P. & van de Geer, S. A. (2011). Statistics for High-Dimensional Data. Springer.

Cassel, C.M., Särndal, C., & Wretman, J. H. (1976). Some results on generalized difference estimation and generalized regression estimation for finite populations. Biometrika, 63, 615-620.

Chernozhukov, V., Chetverikov, D., Demirer, M., Duflo, E., Hansen, C., Newey, W., & Robins, J. M. (2018)....