For instance, the default activation function of tf.keras.layers.SimpleRNN is tanh. My doubt is because tanh activation functions may also cause (like sigmoids) the vanishing gradient problem.

  • 1
    $\begingroup$ It's not directly about RNNs but I hope the discussions here will give you some direction: stats.stackexchange.com/questions/330559/… $\endgroup$
    – gunes
    Commented Jul 23, 2020 at 21:19
  • $\begingroup$ Yes. But the vanish gradient problem with the sigmoid is very explicit. $\endgroup$ Commented Jul 23, 2020 at 21:39


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.