6.3 Using ensembles for model uncertainty estimates

This section will introduce you to deep ensembles: a popular method for obtaining Bayesian uncertainty estimates using an ensemble of deep networks.

6.3.1 Introducing ensembling methods

A common strategy in machine learning is to combine several single models into a committee of models. The process of learning such a combination of models is called ensemble learning, and the resulting committee of models is called an ensemble. Ensemble learning involves two main components: first, the different single models need to be trained. There are various strategies to obtain different models from the same training data: the models can be trained on different subsets of data, we can train different model types or models with different architectures, or we can initialize the same model types with different hyperparameters. Second, the outputs of the different single models need to be combined. Common strategies for combining the predictions...