Training CNNs Using ConvNetSharp

In this chapter, we are going to use the phenomenal open source package ConvNetSharp, by Cédric Bovar, to demonstrate how to train our Convolutional Neural Networks (CNNs). In this chapter, we will look at the following topics:

• Common neural network modules
• The various terms and concepts related to CNNs
• Convolutional networks that process images

Technical requirements

You will need Microsoft Visual Studio and ConvNetSharp framework for this chapter.

Getting acquainted

Before we begin diving into code, let's cover some basic terminology so that we are all on the same page when referring to things. This terminology applies to CNNs as well as the ConvNetSharp framework.

Convolution: In mathematics, a convolution is an operation performed on two functions. This operation produces a third function, which is an expression of how the shape of one is modified by the other. This is represented visually in the following diagram:

It is important to note that the convolutional layer itself is the building block of a CNN. This layer's parameters consist of a set of learnable filters (sometimes called kernels). These kernels have a small receptive field, which is a smaller view into the total image, and this view extends through the full depth of the input volume. During the forward propagation phase, each filter is convolved...

Filters

One of the other unique features of a CNN is that many neurons can share the same vector of weights and biases, or more formally, the same filter. Why is that important? Because each neuron computes an output value by applying a function to the input values of the previous layer. Incremental adjustments to these weights and biases are what helps the network to learn. If the same filter can be re-used, then the required memory footprint will be greatly reduced. This becomes very important, especially as the image or receptive field gets larger.

CNNs have the following distinguishing features:

• Three-dimensional volumes of neurons: The layers of a CNN have neurons arranged in three dimensions: width, height, and depth. The neurons inside each layer are connected to a small region of the layer before it called their receptive field. Different types of connected layers are...

Creating a network

Using the ConvNetSharp framework, there are three ways in which to create a neural network. First, we can use the Core.Layers or Flow.Layers objects to create a convolutional network (with or without a computational graph), as shown in the following diagram:

Alternatively, we can create a computational graph like the following:

Example 1 – a simple example

Let's take a look at our first example. This is a minimal example in which we will define a two-layer neural network and train it on a single data point. We are intentionally making this example verbose so that we can walk through each step together to improve our understanding:

var net = new Net<double>();

The InputLayer variable declares...

GPU

In order to use GPU capability in your software using ConvNetSharp, you must have CUDA Version 8 and Cudnn Version 6.0 (April 27, 2017) installed. The Cudnn bin path should also be referenced in the PATH environment variable.

Fluent training with the MNIST database

In the following example, we will train our CNN against the MNIST database of images.

To declare a function, use the following code:

private void MnistDemo()
{

Next, download the training and testing datasets with the following command:

var datasets = new DataSets();

Load 100 validation sets with the following command:

if (!datasets.Load(100))
{
return;
}

Now it's time to create the neural network using the Fluent API, as follows:

this._net = FluentNet<double>.Create(24, 24, 1)
.Conv(5, 5, 8).Stride(1).Pad(2)
.Relu()
.Pool(2, 2).Stride(2)
.Conv(5, 5, 16).Stride(1).Pad(2)
.Relu()
.Pool(3, 3).Stride(3)
.FullyConn(10)
.Softmax(10)
.Build();

Create the stochastic gradient descent trainer from the network with the following command:

this._trainer = new SgdTrainer<double>(this._net)
{
LearningRate = 0.01,
BatchSize = 20,
L2Decay = 0.001,
Momentum...

Training the network

To train the convolutional network, we must perform both forward- and backward-propagation, as shown in the following example:

public virtual void Train(Volume<T> x, Volume<T> y)
{
Forward(x);
Backward(y);
}

The following screenshot illustrates our training in progress:

Testing the data

This section details the Test function, which will show us how to test the data we have trained. We get the network prediction and track the accuracy for each label that we have with the following command:

private void Test(Volume x, int[] labels, CircularBuffer<double> accuracy, bool forward = true)
{
if (forward)
{

Forward momentum can be found with the following code:

this._net.Forward(x);
}
var prediction...

Summary

In this chapter, we used the open source package ConvNetSharp to explain CNNs. We looked at how to test and train these networks and also learned why they are convolutional. We worked with several example applications to explain how ConvNetSharp functions and operates. In the next chapter, we will look at autoencoders and RNNSharp, exposing you further to recurrent neural networks.

References

• ConvNetSharp Copyright (c) 2018 Cédric Bovar. Used with permission granted.
• The mostly complete chart of neural networks, explained, Asimov Institute, used with permission granted.
• http://scs.ryerson.ca/~aharley/vis/conv/flat.html