In Nielsen's explanation of backpropagation, why does the way he defines error change? Is it actually a change?

Question

Specifically, why does Equation (BP1) not have the same form as (29)?1

In order to explain backpropagating the gradient through the neural net, he starts by defining something he calls error (Equation (29)).

The error in a neuron j in a layer l is given by the partial derivative of the cost function C with respect to the weighted input z of the neuron j in layer l. Weighted input to a neuron is simply the input to that neuron, multiplied by its weight, summed with the neuron's bias. It is what the neuron's activation function takes as its input.

He then defines the error for a neuron in the final, output layer of the net as follows (Equation (BP1)):

Where $\sigma$ is the activation function and $\sigma'$ is its derivative.

What I thought he was going to do was use the first definition of error in combination with the chain rule to derive $\partial C$ in terms of $\partial w$ and $\partial b$ for the j-th neuron in layer l, thereby providing a way to backpropagate through the net.

Why does he change the definition for error immediately after he defines it? Are these two equations actually saying the same thing? If so why?

1 Nielsen, Neural Networks and Deep Learning, Chapter 2 - http://neuralnetworksanddeeplearning.com/chap2.html

Answer 1

Okay bit of a goof on my part. Nielsen addresses this a couple sections later in the same chapter, in Proof of the four fundamental equations (optional).

Briefly, Eqn (29) above can be re-expressed (using the chain rule) as partial derivatives with respect to the output activations a^L. Simplifying gives you the form seen in (BP1).

Answer 2

In Nielsens explanation of bp, he doesn't go in to the details and actually complicates it. There is a video by 3blue1brown on youtube that explains bp really well by simplifying the case to just one neuron for each layer and then when you add more neurons it is not that different.

Highly recommend watching that 7 minute video and then coming back to Nielsens for the code.