Hyperbolic tangent (tanh)

The tanh function is similar to the sigmoid function, but it has the ability to give a negative signal as well. The following diagram illustrates this:

Figure 8.14: Hyperbolic tangent

The y function is as follows:

It can be implemented by the following Python code:

def tanh(x): 
    numerator = 1-np.exp(-2*x) 
    denominator = 1+np.exp(-2*x) 
    return numerator/denominator

Now let's look at the softmax function.