1
$\begingroup$

Can someone help explain this backpropagation graph? This is from cs231n Convolutional Neural Networks for Visual Recognition by Stanford. enter image description here

So for this graph, let's say the true value of the output was 1.03 and the alpha value was 1. Then the error*alpha would be (1.03-0.73)*1 = 0.3. So for the backpropagation, it would update all of the green values by multiplying the red values by 0.3 and subtracting it from the green values, not including x0, and x1 because they are the input values? Is this correct? I appreciate the help.

$\endgroup$
2
$\begingroup$

I was not able to understand what you were trying to state so I have written the way I understood it when I did the course online.

Let's say we start from the end of the circuit during backpropagation and assume that the value of our loss is 1 (the value marked in red).

When we backpropagate, we encounter 1/x as our first gate (because things are considered as a circuit). Here the local gradient is (-1/(x ^ 2)). During the forward pass, this gate got a value of 1.37. So the local gradient becomes -0.53, which can be calculated during the forward pass itself. Using chain rule (which backpropagation is all about) we multiply the local gradient of a gate with the gradient that it is receiving from the top. In this case, it's 1. So, the value becomes -0.53.

The next gate is that of (x+c) whose local gradient is that of 1. So using the chain rule again, our gradient becomes (local gradient) * (received gradient) i.e. (1.0 * -0.53) and thus -0.53 is propagated.

You can continue like this to get the other results.

| cite | improve this answer | |
$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.