# Clarification needed on gradients in backpropagation

I was going through this book "Practical Convolutional Neural Networks" and there under the backpropagation section, it demonstrates calculating the gradient for x and W for a single neuron with a sigmoid activation function.

z = 1/(1 + np.exp(-np.dot(W, x)))   # forward pass
dx = np.dot(W.T, z*(1-z))           # backward pass: local gradient for x
dW = np.outer(z*(1-z), x)           # backward pass: local gradient for W


I do understand why the gradient of x has np.dot in it. But I don't understand how the gradient for W has np.outer. A proper mathematical derivation corresponding to this would be really helpful. Thanks.

Assume there are multiple neurons, and each row of $$W$$ are weights of them, i.e. $$W$$ is of dimension $$m\times n$$, where we have $$m$$ neurons. Then, $$z$$ is of dimension $$m\times 1$$ and $$x$$ has dimension $$n\times 1$$. This is what's being assumed as far as I can see from the implementation.
If we wanted to find the derivative of loss with respect to $$W_{ij}$$, we'll have the following chain rule: $$\frac{\partial L}{\partial W_{ij}}=\frac{\partial L}{\partial z_i}\frac{\partial z_i}{\partial W_{ij}}$$
Here, $$z_i$$ is the output of the $$i$$-th neuron, and is $$z_i=\sigma(h_i)$$ where $$h_i=\sum_{j=1}^n W_{ij}x_j$$ So, $$\frac{\partial z_i}{\partial W_{ij}}=\frac{\partial z_i}{\partial h_i}\frac{\partial h_i}{\partial W_{ij}}=z_i(1-z_i)x_j$$
So, we have derivative matrix (call it $$dW$$) with $$dW_{ij}=z_i(1-z_i)x_j$$. This is exactly the outer product of vectors $$z(1-z)$$ (element-wise multiplication) and $$x$$.
This is not needed for a single neuron, because $$z(1-z)$$ will be just a scalar.