I was going through this book "Practical Convolutional Neural Networks" and there under the backpropagation section, it demonstrates calculating the gradient for
W for a single neuron with a sigmoid activation function.
z = 1/(1 + np.exp(-np.dot(W, x))) # forward pass dx = np.dot(W.T, z*(1-z)) # backward pass: local gradient for x dW = np.outer(z*(1-z), x) # backward pass: local gradient for W
I do understand why the gradient of
np.dot in it. But I don't understand how the gradient for
np.outer. A proper mathematical derivation corresponding to this would be really helpful.