I was reading about the various activation functions that are available to choose from. For example:
- Sigmoid activation function
- Tanh activation function
- Relu activation function
I came across a post here, explaining why Tanh functions are better than sigmoid functions. One of the point that has been mentioned says:
Tanh function provided stronger gradients
What does this statement mean? What are strong gradients? How do stronger gradients help in the learning process? It will be helpful if it could be explained with the help of a real-world example.
I also came across the following graph that describes the gradients for some activation functions.
but could interpret nothing from it.