I am going through Andrew Ng's Deep Learning Specialization and I'm in the third week of the first course. Right now, it's all about neural network backpropagation and I'm having trouble understanding the derivation of the derivatives. It's in this picture:

enter image description here

We are talking here about a neural network with just 1 hidden layer and the output is just one neuron. On the left we have the derivatives of the parameters when we consider just one training example. On the right we have the derivatives of the parameters when we consider the whole matrix $X$ of input examples, and the implementation is vectorized for efficiency.

What I don't understand is why on the right some derivatives are divided by $\frac{1}{m}$ and some aren't. I mean, we are taking the derivatives of the cost function with respect to those parameters. The cost function is:

$$J(...) = \frac{1}{m} \sum_{i = 1}^m L(\hat{y}, y)$$


$$L(\hat{y}, y) = -y \log(\hat{y}) - (1 - y) \log(1 - \hat{y})$$

And if we take the derivative of that cost function $J$ with respect to any parameter, shouldn't we have the term $\frac{1}{m}$ in front of every derivative? Why doesn't that $\frac{1}{m}$ term appear for the deriatives with respect to $z$, $\frac{\partial J}{\partial Z^{[1]}}$ and $\frac{\partial J}{\partial Z^{[2]}}$?


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.