# Backpropagation - partial derivatives

I am currently reading the Neural networks and deep learning book by Michael Nielsen.

I have a question regarding the backpropagation chapter:

Background:

He explains the influence of a neuron on the cost function by saying that there sits a demon in the neuron. The demon sits at the $j^{th}$ neuron in layer $l$. As the input to the neuron comes in, the demon messes with the neuron's operation. It adds a little change $\Delta z_j^l$ to the neuron's weighted input, so that instead of outputting $\sigma(z_j^l)$, the neuron instead outputs $\sigma (z_j^l+\Delta z_j^l)$. This change propagates through later layers in the network, finally causing the overall cost to change by an amount $\frac{\partial C}{\partial z_j^l} \Delta z_j^l$.

Furthermore, he states that if $\frac{\partial C}{\partial z_j^l}$ has a large value (either positive or negative). Then the demon can lower the cost quite a bit by choosing $\Delta z_j^l$ to have the opposite sign to $\frac{\partial C}{\partial z_j^l}$. By contrast, if $\frac{\partial C}{\partial z_j^l}$ is close to zero, then the demon can't improve the cost much at all by perturbing the weighted input $z_j^l$.

So far, I understand this. $\frac{\partial C}{\partial z_j^l}$ quantifies the influence of $z_j^l$ on the cost function $C$.

Question:

However, afterwards he states that there is a heuristic sense in which $\frac{\partial C}{\partial z_j^l}$ is a measure of the error in the neuron.

Why is $\frac{\partial C}{\partial z_j^l}$ a measure of error in the neuron?

I thought that it is just the influence of the neuron output on the cost function.

• Wouldn't that imply that influence equals error?

• Aren't there cases in which we want certain neurons to have greater influence on the cost function?

The expression $\frac{\partial C}{\partial z_j^l}$ represents how much does the change of the weight $j$ in the layer $l$ influence the cost $C$. You are right that this is not equal to the absolute error caused by that weight1.
1 For example, take the $f(x) = |x|$: This function has the same derivative everywhere (1 or -1 on the positive and negative part, respectively), regardless of how far from the minimum you are.