I'm currently taking Andrew Ng's Machine Learning Coursera course, and I'm not sure that I fully understand the Delta gradients in Backpropagation (BP).
I see that for the first layer of BP (which is really the output layer of the network) we can calculate the gradient by subtracting the real output label vector from the output vector generated by the network. However, there is a lot of reference to these delta terms as gradients and partial derivatives. Specifically, he says:
I'm not sure why we are computing derivatives here, and moreover why computing consecutive derivatives for each layer results in overall better weights (especially if we end up just adding them all to some "accumulator":
Any insight would be awesome.
A link to a one page PDF summery of the lesson is here, if it helps: https://www.coursera.org/learn/machine-learning/supplement/pjdBA/backpropagation-algorithm