The final classification method that we will be examining in this chapter is decision trees, which have found particular use in applications such as natural language processing. There are a number of different machine learning algorithms that fall within the overall umbrella of decision trees, such as ID3, CART, and the powerful random forest classifiers (covered in Chapter 5, Ensemble Modeling). In this chapter, we will investigate the use of the ID3 method in classifying categorical data, and we will use the scikit-learn CART implementation as another means of classifying the Iris dataset. So, what exactly are decision trees?
As the name suggests, decision trees are a learning algorithm that apply a sequential series of decisions based on input information to make the final classification. Recalling your childhood biology class, you may have used a process similar to decision trees in the classification of different types of animals via dichotomous keys...
