Specifically, why does Equation (BP1) not have the same form as (29)?1
In order to explain backpropagating the gradient through the neural net, he starts by defining something he calls error (Equation (29)).
The error in a neuron j in a layer l is given by the partial derivative of the cost function C with respect to the weighted input z of the neuron j in layer l. Weighted input to a neuron is simply the input to that neuron, multiplied by its weight, summed with the neuron's bias. It is what the neuron's activation function takes as its input.
He then defines the error for a neuron in the final, output layer of the net as follows (Equation (BP1)):
Where $\sigma$ is the activation function and $\sigma'$ is its derivative.
What I thought he was going to do was use the first definition of error in combination with the chain rule to derive $\partial C$ in terms of $\partial w$ and $\partial b$ for the j-th neuron in layer l, thereby providing a way to backpropagate through the net.
Why does he change the definition for error immediately after he defines it? Are these two equations actually saying the same thing? If so why?
1 Nielsen, Neural Networks and Deep Learning, Chapter 2 - http://neuralnetworksanddeeplearning.com/chap2.html