Combining classifiers via majority vote

After the short introduction to ensemble learning in the previous section, let’s start with a warm-up exercise and implement a simple ensemble classifier for majority voting in Python.

Plurality voting

Although the majority voting algorithm that we will discuss in this section also generalizes to multiclass settings via plurality voting, the term “majority voting” will be used for simplicity, as is often the case in the literature.

Implementing a simple majority vote classifier

The algorithm that we are going to implement in this section will allow us to combine different classification algorithms associated with individual weights for confidence. Our goal is to build a stronger meta-classifier that balances out the individual classifiers’ weaknesses on a particular dataset. In more precise mathematical terms, we can write the weighted majority vote as follows:

Here, w_j is a...