Classification Using Decision Trees

Another powerful classification method that we will be examining in this chapter is decision trees, which have found particular use in applications such as natural language processing, for example. There are a number of different machine learning algorithms that fall within the overall umbrella of decision trees, such as Iterative Dichotomiser 3 (ID3) and Classification and Regression Tree (CART). In this chapter, we will investigate the use of the ID3 method in classifying categorical data, and we will use the scikit-learn CART implementation as another method of classifying the dataset. So, what exactly are decision trees?

As the name suggests, decision trees are a learning algorithm that apply a sequential series of decisions based on input information to make the final classification. Recalling your childhood biology class, you may have used a process similar to decision trees in the classification of different types of animals via dichotomous...