The hyperbolic tangent function, which is also known as tanh, is very similar to the sigmoid function; however, the lower bound of the curve is in negative space to better handle data containing negative values.
Aside from this one difference, everything else about the hyperbolic tangent function is the same as the sigmoid function, and like the sigmoid function, the hyperbolic tangent contains a derivative element and can be used with backpropagation.
Since tanh is bounded between -1 and 1, the gradient is larger and the derivative is more pronounced. Being bounded means that tanh is centered around 0, which can be advantageous in a model with a large number of hidden layers as the results from a layer are easier for the next layer to use.
Let's use the same sequence of values and plot the values after passing all of these values through the hyperbolic tangent function, which is included with base R and can be called using tanh():
- As we did in the preceding sigmoid example, we will create a tibble with the values in our sequence and the transformed values:
vals <- tibble(x = seq(-10, 10, 1), tanh_x = tanh(seq(-10, 10, 1)))
- We then set up the base of our plot by passing the dataset along with the values to use for the x-axis and the y-axis to the ggplot() function:
p <- ggplot(vals, aes(x, tanh_x))
- Lastly, we again add our points and use the stat_function feature to connect the dots and display our shape:
p <- p + geom_point()
p + stat_function(fun = tanh, n = 1000)
As we look at this shape, we can see that it is a very similar shape to the sigmoid shape that we just plotted; however, note that the y-axis now has a range from -1 to 1 rather than 0 to 1, as was the case with the sigmoid shape. As a result, the values are pushed even further to the extremes and negative values remain negative after transformation:
