Chapter 4: Building and Training a Feedforward Neural Network

In Chapter 3, Getting Started with Neural Networks, you learned the basic theory behind neural networks and deep learning. This chapter sets that knowledge into practice. We will implement two very simple classification examples: a multiclass classification using the iris flower dataset, and a binary classification using the adult dataset, also known as the census income dataset.

These two datasets are quite small and the corresponding classification solutions are also quite simple. A fully connected feedforward network will be sufficient in both examples. However, we decided to show them here as toy examples to describe all of the required steps to build, train, and apply a fully connected feedforward classification network with KNIME Analytics Platform and KNIME Keras Integration.

These steps include commonly used preprocessing techniques, the design of the neural architecture, the setting of the activation functions...

Preparing the Data

In Chapter 3, Getting Started with Neural Networks, we introduced the backpropagation algorithm, which is used by gradient descent algorithms to train a neural network. These algorithms work on numbers and can't handle nominal/categorical input features or class values. Therefore, nominal input features or nominal output values must be encoded into numerical values if we want the network to make use of them. In this section, we will show several numerical encoding techniques and the corresponding nodes in KNIME Analytics Platform to carry them out.

Besides that, we will also go through many other classic data preprocessing steps to feed machine learning algorithms: creating training, validation, and test sets from the original dataset; normalization; and missing value imputation.

Along the way, we will also show you how to import data, how to perform a few additional data operations, and some commonly used tricks within KNIME Analytics Platform. The workflows...

Building a Feedforward Neural Architecture

To build a neural network architecture using the KNIME Keras integration, you can use a chain of Keras layer nodes. The available nodes to construct layers are grouped by categories in the Keras->Layers folder in the Node Repository, such as Advanced Activations, Convolution, Core, Embedding, and Recurrent, to name just a few.

Each layer displayed in the Keras->Layers folder has a specialty. For example, layers in Advanced Activations create layers with units with specific activation functions; layers in Convolution create layers for convolutional neural networks; Core contains all classic layers, such as the Input layer to collect the input values and the Dense layer for a fully connected feedforward neural network; and so on.

We will explore many of these layers along the way in this book. However, in this current chapter, we will limit ourselves to the basic layers needed in a fully connected feedforward neural network.

...

Training the Network

We have the data ready and we have the network. The goal of this section is to show you how to train the network with the data in the training set. This requires the selection of the loss function, the setting of the training parameters, the specification of the training set and the validation set, and the tracking of the training progress.

The key node for network training and for all these training settings is the Keras Network Learner node. This is a really powerful, really flexible node, with many possible settings, distributed over four tabs: Input Data, Target Data, Options, and Advanced Options.

The Keras Network Learner node has three input ports:

• Top port: The neural network you want to train
• Middle port: The training set
• Lowest port: The optional validation set

It has one output port, exporting the trained network.

In addition, the node has the Learning Monitor view, which you can use to monitor the network training progress...

Testing and Applying the Network

Now that the neural network has been trained, the last step is to apply the network to the test set and evaluate its performance.

Executing the Network

To execute a trained network, you can use the Keras Network Executor node, as in Figure 4.22. The node has two input ports: a Keras network port for the trained network and a data input port for the test set or new data.

In the first tab of the configuration window, named Options, you can select, in the upper part, the backend engine, the batch size for the input data, and whether to also keep the original input columns in the output data table.

Under that, you can specify the input columns and the required conversion. Like in the Keras Network Learner node, the input specifications from the neural network are printed at the top. Remember that, since you are using the same network and the same format for the data, the settings for the input features must be the same as the ones in the Keras...

Summary

We have reached the end of this chapter, where you have learned how to perform the different steps involved in training a neural network in KNIME Analytics Platform.

We started with common preprocessing steps, including different encodings, normalization, and missing value handling. Next, you learned how to define a neural network architecture by using different Keras layer nodes without writing code. We then moved on to the training of the neural network and you learned how to define the loss function, as well as how you can monitor the learning progress, apply the network to new data, and extract the predictions.

Each section closed with small example sessions, preparing you to perform all these steps on your own.

In the next chapter, you will see how these steps can be applied to the first use case of the book: fraud detection using an autoencoder.

Questions and Exercises

Check your level of understanding of the concepts presented in this chapter by answering the following questions:

1. How can you set the loss function to train your neural network?

   a) By using the Keras Loss Function node

   b) By using the Keras Output Layer node

   c) In the configuration window of the Keras Network Learner node

   d) In the configuration window of the Keras Network Executor node

2. How can you one-hot encode your features?

   a) By using the One Hot Encoding node

   b) By using the One to Many node 

   c) By creating an integer encoding using the Category to Number node and afterward, the Integer to One Hot Encoding node

   d) By creating an integer encoding, transforming it into a collection cell, and selecting the right conversion

3. How can you define the number of neurons for the input of your network?

   a) By using a Keras Input Layer node.

   b) By using a Keras Dense Layer node without any input network.

   c) The input dimension is set automatically based on...