An introduction to NNs

We can describe NNs as a mathematical model for information processing. As discussed in Chapter 1, this is a good way to describe any ML algorithm, but in this chapter, it has a specific meaning in the context of NNs. An NN is not a fixed program but rather a model, a system that processes information, or inputs. The characteristics of an NN are as follows:

• Information processing occurs in its simplest form, over simple elements called units
• Units are connected, and they exchange signals between them through connection links
• Connection links between units can be stronger or weaker, and this determines how information is processed
• Each unit has an internal state that is determined by all the incoming connections from other units
• Each unit has a different activation function that is calculated on its state and determines its output signal

A more general description of an NN would be as a computational graph of mathematical operations...