1
$\begingroup$

I understand that the forward pass of a feed-forward neural network can be thought of as a hidden layer-wise composite function, and backpropagation works by recursively applying the chain rule to that composite function to find local gradients.

What exactly does each application of the chain rule during backpropagation represent? Is it applied to each node in each hidden layer? Does the error gradient have as many axes as there are nodes in the network?

$\endgroup$

1 Answer 1

1
$\begingroup$

That is because when you change the bias or the connection strengths of each node, that may affect the output of the network. And hence each connection strength and bias of each node has their own gradient. So the error gradient has as many axes as there are parameters in the neural network.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.