The term bagging is derived from a technique called bootstrap aggregation. In order to implement a successful predictive model, it's important to know in what situation we could benefit from using bootstrapping methods to build ensemble models. In this section, we'll talk about a way to use bootstrap methods to create an ensemble model that minimizes variance and look at how we can build an ensemble of decision trees, that is, the Random Forest algorithm. But what is bootstrapping and how does it help us build robust ensemble models?
The bootstrap method refers to random sampling with replacement, that is, drawing multiple samples (each known as a resample) from the dataset consisting of randomly chosen data points, where there can be an overlap in the data points contained in each resample and each data point has an equal probability of being selected from the overall dataset:
Figure 5.5: Randomly choosing data points
From the previous diagram, we can see that each of...
