I'm following the basics of autoencoders here: http://ufldl.stanford.edu/wiki/index.php/Autoencoders_and_Sparsity Here are some of the important parts: enter image description here enter image description here But I don't understand the last part: Why is the backpropagation equation modified like that? Can anyone help me ? I'm not really good with math, although I have been trying a lot :( . Thank you very much

  • $\begingroup$ it seems the sparsity loss should be the sum of $z$s instead of $a$s, because the value of $a$s can exceed the range $(0, 1)$, right? $\endgroup$
    – dontloo
    Commented Jul 11, 2016 at 7:27

1 Answer 1


Because when sparsity cost is added to the loss function we will take the derivative of both of the loss terms. You can check the below link for how you are going to code it. https://mccormickml.com/2014/05/30/deep-learning-tutorial-sparse-autoencoder/

  • 1
    $\begingroup$ Hi and Welcome to Cross Validated. Your answer could be interesting to the readers of this site. Could you provide some details about the link you provide? Links in the future may change and we hope to build stable answers. For further details please refer to our help page $\endgroup$
    – utobi
    Commented Jan 7, 2023 at 11:56
  • $\begingroup$ While this link may answer the question, it is better to include the essential parts of the answer here and provide the link for reference. Link-only answers can become invalid if the linked page changes. - From Review $\endgroup$
    – mdewey
    Commented Jan 7, 2023 at 14:36

Not the answer you're looking for? Browse other questions tagged or ask your own question.