# Gradient Descent and Back Propagation in Neural Networks: derivation in vectorised form

I was learning deep learning from Andrew Ng's course on Coursera and in one of the programming assignments to code out Neural Networks from scratch, the formula for the derivative of cost function wrt W is given as:

$$\frac {dJ} {dW_{i}} = \frac {1} {m} \frac {dJ} {dZ_{i}} A_{i - 1}^T$$

Where,
$$J$$ is the cost function
$$m$$ is the number of training examples,
$$W_{i}$$ are the parameters from layer $$i - 1$$ to layer $$i$$
$$Z_i = W_i^TA_{i - 1} + B_i$$.
$$A_0$$ is $$X^T$$, i.e., $$dim(A_{0}) = n * m$$ and $$A_i = ActivationFn(Z_i)$$.

However, this seems inconsistent dimensionally, assuming the product of $$\frac {dJ} {dZ_{i}}$$ and $$A_{i - 1}^T$$ is dot product. If we denote the number of nodes in layer $$i$$ as $$n_i$$, where layer $$0$$ is the input $$X$$ and $$n_0 = number of features(n)$$,

$$dim(A_{i - 1}) = (n_{i - 1}, m)$$ $$dim(Z_i) = (n_{i}, m)$$ $$dim(\frac{dJ}{dZ_i}) = dim(Z)$$ $$dim(\frac {dJ} {dZ_{i}} A_{i - 1}^T ) = (n_{i}, n_{i - 1})$$ $$dim(\frac {dJ} {dW_{i}}) = (n_{i - 1}, n_i)$$

However,
$$dim(W_i) = (n_{i - 1}, n_i)$$ I do not have much knowledge of vector calculus but when I tried t derive these equations myself I got stuck on what kind of product (dot or Hadamard product or something else) will be there between the derivatives on applying chain rule. Can someone please help me figure out where have I gone wrong?

Here is an image of the formulae listed there. Note, the author means $$\frac{dJ}{dx}$$ when he write $$dx$$

The dimensions in the formula were indeed inconsistent, I could fix it by adding a transpose to the formula. $$\frac {dJ} {dW_{i}} = \frac {1} {m} (\frac {dJ} {dZ_{i}} A_{i - 1}^T)^T$$