0
$\begingroup$

I am following the CS231n NN case study — a derivation of gradient descent for a simple network with a single hidden layer.

I have followed the rest of the tutorial and have confidence that the derivations are correct.

However, when I run their code (after “The full code looks very similar:”), I encounter increases to the loss function.

How is this possible? Do I have an error in my code? Or is it possible for the loss function to increase? I have seen a reference to loss increases depending on step size…

Improve this question
$\endgroup$
1
  • $\begingroup$ The duplicate explains that if the step size is too large, the loss can increase. While it's possible that you have a bug in your code, but debugging is not on-topic here. In any event, it's impossible for someone to debug code that they can't see. $\endgroup$
    – Sycorax ♦
    Commented Dec 19, 2023 at 17:09

0

Reset to default

Browse other questions tagged or ask your own question.