$\frac{\partial \mathcal{L}}{\partial W^{[2]}}$ must be 2x3 as just like dimensions of $ W^{[2]}$. I suggest you to use the backprop formulas (and notation) given in Nielsen's [book][1]. When the networks gets bigger it is ease to follow [![BP from Nielsen][2]][2] According to that \begin{align*} \delta^3 &=a^{[3]}-y \\ \delta^2 &= ((W^{[3]^{T}} (a^{[3]}-y)) \odot g'(z^{[2]})) \\ \frac{\partial \mathcal{L}}{\partial w^{[2]}_{jk}} &= a^{[1]}_k \cdot \delta_j^2 \end{align*} And by going one more step: \begin{align*} \delta^1 &= ((W^{[2]^{T}} \delta^2 ) \odot g'(z^{[1]})) \\ \frac{\partial \mathcal{L}}{\partial w^{[1]}_{jk}} &= x^{(i)}_k \cdot \delta_j^1 \end{align*} where $ \delta^1 \in \mathbb{R}^{3\times 1}$ I hope at least for other people it will be usefull [1]: http://neuralnetworksanddeeplearning.com/chap2.html [2]: https://i.sstatic.net/VWCmE.png