Building Our First Neural Network Together

Now that we've had a quick refresher on Neural Networks, I thought that perhaps a good starting point, code-wise, would be for us to write a very simple neural network. We're not going to go crazy; we'll just lay the basic framework for a few functions so that you can get a good idea of what is behind the scenes of many of the APIs that you'll use. From start to finish, we'll develop this network application so that you are familiar with all the basic components that are contained in a neural network. This implementation is not perfect or all-encompassing, nor is it meant to be. As I mentioned, this will merely provide a framework for us to use in the rest of the book. This is a very basic neural network with the added functionality of being able to save and load networks and data. In any event, you will have...

Technical requirements

You would need to have Microsoft Visual Studio installed on the system.

Check out the following video to see Code in Action: http://bit.ly/2NYJa5G.

Our neural network

Let's begin by showing you an of what a simple neural network would look like, visually. It consists of an input layer with 2 inputs, a Hidden Layer with 3 neurons (sometimes called nodes), and a final output layer consisting of a single neuron. Of course, neural networks can consist of many more layers (and neurons per layer), and once you get into deep learning you will see much more of this, but for now this will suffice. Remember, each node, which is labeled as follows with an N, is an individual neuron – its own little processing brain, if you will:

Let’s break down the neural network into its three basic parts; inputs, Hidden Layers and outputs:

Inputs: This is the initial data for our network. Each input is a whose output to the Hidden Layer is the initial input value.

Hidden Layers: These are the heart and soul of our network, and...

Neural network training

How do we train a neural network? Basically, we will provide the with a set of input data as well as the results we expect to see, which correspond to those inputs. That data is then run through the network until the network understands what we are looking for. We will train, test, train, test, train, test, on and on until our network understands our data (or doesn't, but that's a whole other conversation). We continue to do this until some designated stop condition is satisfied, such as an error rate threshold. Let's quickly cover some of the terminology we will use while training neural networks.

Back propagation: After our data is run through the network, we to validate that data what we expect to be the correct output. We do this by propagating backward (hence backprop or back propagation) through each of the Hidden Layers of our network...

Neural network functions

The following basic list contains the functions we are going to develop n order to lay down our neural network foundation:

• Creating a new network
• Importing a network
• Manually entering user data
• Importing a dataset
• Training our network
• Testing our network

With that behind us, let's start coding!

Creating a new network

This menu option will allow us to create a new network from scratch:

public NNManager SetupNetwork()
{
    _numInputParameters = 2;

    int[] hidden = new int[2];
    hidden[0] = 3;
    hidden[1] = 1;
    _numHiddenLayers = 1;
    _hiddenNeurons = hidden;
    _numOutputParameters = 1;
    _network = new Network(_numInputParameters, _hiddenNeurons,         
    _numOutputParameters);
   ...

The neural network

With many of the ancillary, but important, functions coded, we now turn our attention to the meat of the neural network, the network itself. Within a neural network, the network part is an all-encompassing universe. Everything resides within it. Within this structure we will need to store the input, output, and Hidden Layers of neurons, as well as the learning rate and Momentum, as follows:

public class Network
{
      public double LearningRate{ get; set; }
      public double Momentum{ get; set; }
      public List<Neuron>InputLayer{ get; set; }
      public List<List<Neuron>>HiddenLayers{ get; set; }
      public List<Neuron>OutputLayer{ get; set; }
      public List<Neuron>MirrorLayer {get; set; }
      public List<Neuron>CanonicalLayer{ get; set; }

...

Examples

Now that we have our code created, let's use a few examples to see how it can be used.

Training to a minimum

In this example, we will use the code we wrote to train a network to a minimum value or threshold. For each step, the network prompts you for the correct data, saving us the process of cluttering up our example code with this. In production, you would probably want to pass in the parameters without any user intervention, in case this is run as a service or microservice:

Training to a maximum

In this example, we are going to train the network to reach...

Summary

In this chapter, we saw how to write a complete neural network from scratch. Although the following is a lot we've left out, it does the basics, and we've gotten to see it as pure C# code! We should now have a much better understanding of what a neural network is and what it comprises than when we first started.

In the next chapter, we will begin our journey into more complicated network structures such as recurrent and convolutional neural networks. There's a lot to cover, so hold on to your coding hats!