Decision trees and random forests are powerful techniques that you can use to add power to your applications. Let's walk through some concepts and some code and hopefully have you up and running in no time.
In this chapter, we are going to learn about decision trees and random forests. We will:
You will be required to have Microsoft Visual Studio installed in your system. You may also need to refer to open source SharpLearning framework's GitHub repository at https://github.com/mdabros/SharpLearning.
Check out the following video to see Code in Action: http://bit.ly/2O1Lbhr.
Decision trees can be used for both classification and regression. Decision trees answer sequential questions with a yes/no, true/false response. Based upon those responses, the tree follows predetermined paths to reach its goal. Trees are more formally a version of what is known as a directed acyclic graph. Finally, a decision tree is built using the entire dataset and all features.
Here is an example of a decision tree. You may not know it as a decision tree, but for sure you know the process. Anyone for a doughnut?
As you can see, the flow of a decision tree starts at the top and works its way downward until a specific result is achieved. The root of the tree is the first decision that splits the dataset. The tree recursively splits the dataset according to what is known as the splitting metric at each node. Two of the most popular metrics are Gini Impurity...
We have talked about decision trees, and now it’s time to discuss random forests. Very basically, a random forest is a collection of decision trees. In random forests, a fraction of the number of total rows and features are selected at random to train on. A decision tree is then built upon this subset. This collection will then have the results aggregated into a single result.
Random forests can also reduce bias and variance. How do they do this? By training on different data samples, or by using a random subset of features. Let’s take an example. Let’s say we have 30 features. A random forest might only use 10 of these features. That leaves 20 features unused, but some of those 20 features might be important. Remember that a random forest is a collection of decision trees. Therefore, in each tree, if we utilize 10 features, over time most if...
In the next few sections, we are going to look at some code samples without all the verbosity. This will be pure C# code so it should be something easily understood by all.
Let’s take a quick look at how we can use SharpLearning to predict observations. I’ll show you an entire code sample without the verbosity:
var parser = new CsvParser(() =>new StringReader(Resources.AptitudeData));
var observations = parser.EnumerateRows(v => v != "Pass").ToF64Matrix();
var targets = parser.EnumerateRows("Pass").ToF64Vector();
var rows = targets.Length;
var learner = new ClassificationDecisionTreeLearner(100, 1, 2, 0.001, 42);
varsut = learner.Learn(observations, targets);
var predictions = sut.Predict(observations);
var evaluator = new TotalErrorClassificationMetric<double>();
var error = evaluator.Error(targets, predictions...
In this chapter, we learned about decision trees and random forests. We also learned how to use the open source framework, SharpLearn, to add these powerful features to our applications. In the next chapter, we are going to learn about facial and motion detection and show you how you can enable your application with this exciting technology! You’ll meet Frenchie, my pet French Bulldog, who will demonstrate most of the samples we will show. Also, we have a guest poser you will just have to see!
© 2018 Packt Publishing Limited All Rights Reserved
