Supervised Machine Learning Using TensorFlow 2

In this chapter, we will discuss and exemplify the use of TensorFlow 2 for supervised machine learning problems for the following situations: linear regression, logistic regression, and k-Nearest Neighbors (KNN).

In this chapter, we will look at the following topics:

• Supervised learning
• Linear regression
• Our first linear regression example
• The Boston housing dataset
• Logistic regression (classification)
• k-Nearest Neighbors (KNN)

Supervised learning

Supervised learning is the machine learning scenario in which one or more data points from a set of data points is/are associated with a label. The model then learns to predict the labels for unseen data points. For our purposes, each data point will normally be a tensor and will be associated with a label. Supervised learning problems abound in computer vision; for example, an algorithm is shown many pictures of ripe and unripe tomatoes, together with a categorical label indicating whether or not they are ripe, and when the training has concluded, the model is able to predict the status of tomatoes that weren't in its training set. This could have a very direct application in a physical sorting mechanism for tomatoes; or an algorithm that could learn to predict the gender and age of a new face after it has been shown many examples, together with their...

Linear regression

A linear regression problem is one where you have to predict the value of one continuous variable, given the value of one or more other variables (data points); for example, predicting the selling price of a house, given its floor space. You can plot the known features with their associated labels on a simple linear graph in these examples, as in the familiar x, y scatter plots, and plot a line that best fits the data. This is known as a line of best fit. You can then read off the label corresponding to any value of your feature that lies within the x range of the plot.

However, linear regression problems may involve several features in which the terminology multiple or multivariate linear regression is used. In this case, it is not a line that best fits the data, but a plane (two features) or a hyperplane (more than two features). In the house price example...

Our first linear regression example

We will start with a simple, artificial, linear regression problem to set the scene. In this problem, we construct an artificial dataset where we first create, and hence, know, the line to which we are fitting, but then we'll use TensorFlow to find this line.

We do this as follows—after our imports and initialization, we enter a loop. Inside this loop, we calculate the overall loss (defined as the mean squared error over our dataset, y, of points). We then take the derivative of this loss with respect to our weights and bias. This produces values that we can use to adjust our weights and bias to lower the loss; this is known as gradient descent. By repeating this loop a number of times (technically called epochs), we can lower our loss to the point where it is as low as it can go, and we can use our trained model to make predictions...

The Boston housing dataset

Next, we will apply a similar regression technique to the Boston housing dataset.

The main difference between this and our previous artificial dataset, which had just one feature, is that the Boston housing dataset is real data and has 13 features. This is a regression problem because house prices—the label—we take as being continuously valued.

Again, we start with our imports, as follows:

import tensorflow as tf
from sklearn.datasets import load_boston
from sklearn.preprocessing import scale
import numpy as np

And our important constants are shown as follows:

learning_rate = 0.01
epochs = 10000
display_epoch = epochs//20
n_train = 300
n_valid = 100

Next, we load our dataset and split it into training, validation, and test sets. We train on the training set, and check and fine-tune our trained model on the validation set, to make sure we have...

Logistic regression (classification)

This type of problem is confusingly named because regression, as we have seen, implies a continuously valued label, such as the median price of a house, or the height of a tree.

This is not the case with logistic regression. When you have a problem requiring logistic regression, it means that the label is categorical; for example, zero or one, True or False, yes or no, cat or dog, or it may more than two categorical values; for example, red, blue or, green, or one, two, three, four, or five, or the type of a given flower. The labels normally have probabilities associated with them; for example, P(cat=0.92), P(dog=0.08). Thus, logistic regression is also known as classification.

In our next example, we will use logistic regression to predict the category of items of fashion using the fashion_mnist dataset.

Here are a few examples:

Logistic...

k-Nearest Neighbors (KNN)

The idea behind KNN is relatively straightforward. Given the value of a new particular data point, look at the KNN to the point, and assign a label to the point, depending on the labels of those k neighbors, where k is a parameter of the algorithm.

There is no model as such constructed in this scenario; the algorithm merely looks at all the distances between our new point and all the other data points in the dataset, and in what follows, we are going to make use of a famous dataset that consists of three types of iris flowers: iris setosa, iris virginica, and iris versicolor. For each of these labels, the features are petal length, petal width, sepal length, and sepal width. For diagrams showing this dataset, see https://en.wikipedia.org/wiki/Iris_flower_data_set#/media/File:Iris_dataset_scatterplot.svg.

There are 150 data points (each consisting of the...

Summary

In this chapter, we have seen examples of the use of TensorFlow for two situations involving linear regression; where features are mapped to known labels that have continuous values, thus allowing predictions on unseen features to be made. We have also seen an example of logistic regression, better described as classification, where features are mapped to categorical labels, again allowing predictions on unseen features to be made. Finally, we looked at the KNN algorithm for classification.

We will now move on, in Chapter 5, Unsupervised Learning Using TensorFlow 2, to unsupervised learning, where there is no initial mapping between features and labels, and the task of TensorFlow is to discover relationships between the features.