Chapter 2. Supervised and Unsupervised Learning Algorithms

In the previous chapter, we got some insight into the various aspects of machine learning and were introduced to the various ways in which machine learning algorithms could be categorized. In this chapter, we will go a step further into machine learning algorithms and try to understand supervised and unsupervised learning algorithms. This categorization is based on the learning mechanism of the algorithm, and is the most popular.

In this chapter, we will be covering the following topics:

• An introduction to the supervised learning algorithm in the form of a detailed practical example to help understand it and its guiding principles
• The key supervised learning algorithms and their application areas:
  • Naive Bayes
  • Decision trees
  • Linear regression
  • Logistic regression
  • Support vector machines
  • Random forest
• An introduction to the unsupervised learning algorithm in the form of a detailed practical example to help understand it
• The key unsupervised learning...

Introduction to supervised learning algorithms

Let's look at supervised learning for simple day-to-day activities. A parent asks their 15-year-old son to go to the store and get some vegetables. They give him a list of vegetables, say beets, carrots, beans, and tomatoes, that they want him to buy. He goes to the store and is able to identify the list of vegetables as per the list provided by his mother from all the other numerous varieties of vegetables present in the store and put them in his cart before going to the checkout. How was this possible?

Simple. The parent had provided enough training to the son by providing instances of each and every vegetable, which equipped him with sufficient knowledge of the vegetables. The son used the knowledge he has gained to choose the correct vegetables. He used the various attributes of the vegetables to arrive at the correct class label of the vegetable, which, in this case, is the name of the vegetable. The following table gives us a few of the...

Deep dive into supervised learning algorithms

Assume there are predictor attributes, x1, x2, .... xn, and also an objective attribute, y, for a given dataset. Then, the supervised learning is themachine learning task of finding the prediction function that takes as input both the predictor attributes and the objective attribute from this dataset, and is capable of mapping the predictive attributes to the objective attribute for even unseen data currently not in the training dataset with minimal error.

The data in the dataset used for arriving at the prediction function is called the training data and it consists of a set of training examples where each example consists of an input object, x (typically a vector), and a desired output value, Y. A supervised learning algorithm analyzes the training data and produces an inferred function that maps the input to output and could also be used for mapping new, unseen example data:

Y = f(X) + error

The whole category of algorithms is called supervised...

Introduction to unsupervised learning algorithms

Consider a scenario where a child is given a bag full of beads of different sizes, colors, shapes, and made of various materials. We just leave to the child do whatever they want with the whole bag of beads. 

There are various things the child could do, based on their interests:

• Separate the beads into categories based on size
• Separate the beads into categories based on shape

• Separate the beads into categories based on a combination of color and shape
• Separate the beads into categories based on a combination of material, color, and shape

The possibilities are endless. However, the child without any prior teaching is able to go through the beads and uncover patterns of which it doesn't need any any prior knowledge at all. They are discovering the patterns purely on the basis of going through the beads at hand, that is, the data at hand. We just got introduced to unsupervised machine learning!

We will relate the preceding activity to the key steps...

Deep dive into unsupervised learning algorithms

Unsupervised machine learning deals with learning unlabeled data—that is, data that has not been classified or categorized, and arriving at conclusions/patterns in relation to them.

These categories learn from test data that has not been labeled, classified, or categorized. Instead of responding to feedback, unsupervised learning identifies commonalities in the data and reacts based on the presence or absence of such commonalities in each new piece of data.

The input given to the learning algorithm is unlabeled and, hence, there is no straightforward way to evaluate the accuracy of the structure that is produced as output by the algorithm. This is one feature that distinguishes unsupervised learning from supervised learning. 

Note

The unsupervised algorithms have predictor attributes but NO objective function.

What does it mean to learn without an objective? Consider the following:

• Explore the data for natural groupings.
• Learn association rules, and...

Summary

In this chapter, we learned about what supervised learning is through a naive example and deep dived into concepts of supervised learning. We went through various supervised learning algorithms with practical examples and their application areas and then we started going through unsupervised learning with naive examples. We also covered the concepts of unsupervised learning and then we went through various unsupervised learning algorithms with practical examples and their application areas.

In the subsequent chapters, we will be solving mobile machine learning problems by using some of the supervised and unsupervised machine learning algorithms that we have gone through in this chapter. We will also be exposing you to mobile machine learning SDKs, which will be used to implement mobile machine learning solutions.

References

• Dr. Pedro Domingo's paper—https://homes.cs.washington.edu/~pedrod/papers/cacm12.pdf, summarizes twelve key lessons that machine learning researchers and practitioners have learned, including pitfalls to avoid, important issues to focus on, and answers to common questions in this area.