To my understanding, the vanishing gradient problem occurs when training neural networks when the gradient of each activation function is less than 1 such that when corrections are back-propagated through many layers, the product of these gradients becomes very small
I know there are other solutions like a rectifier activation function, but my question is why we could not simply use a variation of the often used tanh function.
If the activation function was of the form $\tanh(n x)$ then the maximum possible gradient is $n$. Thus if $n > 1$ we no longer have a case where the product of gradients necessarily goes to 0.
Is there some reason why such an activation function would otherwise fail?