I am currently reading the Neural networks and deep learning book by Michael Nielsen.
I have a question regarding the backpropagation chapter:
Background:
He explains the influence of a neuron on the cost function by saying that there sits a demon in the neuron. The demon sits at the $j^{th}$ neuron in layer $l$. As the input to the neuron comes in, the demon messes with the neuron's operation. It adds a little change $\Delta z_j^l$ to the neuron's weighted input, so that instead of outputting $\sigma(z_j^l)$, the neuron instead outputs $\sigma (z_j^l+\Delta z_j^l)$. This change propagates through later layers in the network, finally causing the overall cost to change by an amount $\frac{\partial C}{\partial z_j^l} \Delta z_j^l$.
Furthermore, he states that if $\frac{\partial C}{\partial z_j^l}$ has a large value (either positive or negative). Then the demon can lower the cost quite a bit by choosing $\Delta z_j^l$ to have the opposite sign to $\frac{\partial C}{\partial z_j^l}$. By contrast, if $\frac{\partial C}{\partial z_j^l}$ is close to zero, then the demon can't improve the cost much at all by perturbing the weighted input $z_j^l$.
So far, I understand this. $\frac{\partial C}{\partial z_j^l}$ quantifies the influence of $z_j^l$ on the cost function $C$.
Question:
However, afterwards he states that there is a heuristic sense in which $\frac{\partial C}{\partial z_j^l}$ is a measure of the error in the neuron.
Why is $\frac{\partial C}{\partial z_j^l}$ a measure of error in the neuron?
I thought that it is just the influence of the neuron output on the cost function.
Wouldn't that imply that influence equals error?
Aren't there cases in which we want certain neurons to have greater influence on the cost function?