0
$\begingroup$

Andrew Ng in one of his deep learning course videos says that the sigmoid function acts as a slow learner in a neural network.

My intuition is that the sigmoid as an activation function contributes very less to the change in the input for the tail part (after 4 in the fig). Sigmoid

Please help me understand the intuition of the statement in detail.

Thanking in anticipation.

Improve this question
$\endgroup$
4
  • $\begingroup$ Slow compared to what? $\endgroup$
    – Dave
    Commented Sep 13, 2020 at 5:02
  • $\begingroup$ Slow compared to ReLU, I assume... for the reasons given in the question itself. $\endgroup$
    – Itamar Mushkin
    Commented Sep 13, 2020 at 7:43
  • 3
    $\begingroup$ This may help you: stats.stackexchange.com/questions/126238/… $\endgroup$
    – Javier TG
    Commented Sep 13, 2020 at 11:30
  • $\begingroup$ Sorry for the ambiguity, but yes, slow in comparison with other activation functions like ReLU. $\endgroup$
    – CRBelhekar
    Commented Sep 13, 2020 at 17:45

0

Reset to default

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Browse other questions tagged or ask your own question.