# Is my step by step derivation of quadratic cost function's (Neural Networks) partial derivative with respect to some weights matrix correct?

I am trying to revise the details of a Multi-layer Perceptron with a set of weight matrices $$\mathcal W$$ and a set of bias vectors $$\mathbf b$$. Here is the quadratic cost function I am using,

$$C(\mathcal W, \mathbf b) = \frac{1}{n} \sum_x(y-a^L)^2$$

I understand that in order to run (Stocastic) Gradient Descent we need the gradient of the cost function $$\nabla C(\mathcal W, \mathbf b),$$

$$\nabla C(\mathcal W, \mathbf b) = \Bigg[\frac{\partial C}{\partial w^1}, \frac{\partial C}{\partial b^1}, ... , \frac{\partial C}{\partial w^L}, \frac{\partial C}{\partial b^L}\Bigg]$$

Here is my question, for some layer $$l$$, is my derivation of $$\frac{\partial C}{\partial w^l}$$ correct (I am nervous about a negative sign)?

\begin{align*} \frac{\partial C}{\partial w^l} & = \frac{1}{n} \sum_x \Bigg( \frac{\partial (y-a^l)^2}{\partial (y-a^l)} \frac{\partial (y-a^l)}{\partial w^l} \Bigg) && \frac{\partial f(g(x))}{\partial (x)} = \frac{\partial f}{\partial g}\frac{\partial g}{\partial x} \\ & = \frac{1}{n} \sum_x \Bigg( \frac{\partial (y-a^l)^2}{\partial (y-a^l)} \Big(\frac{\partial (y)}{\partial w^l} - \frac{\partial (a^l)}{\partial w^l} \Big)\Bigg) && \frac{\partial (f \pm g)}{\partial x} = \frac{\partial (f)}{\partial x} \pm \frac{\partial (g)}{\partial x} \\ & = \frac{1}{n} \sum_x \Bigg( \frac{\partial (y-a^l)^2}{\partial (y-a^l)} \Big(0 - \frac{\partial (a^l)}{\partial w^l} \Big)\Bigg) && \frac{\partial (y)}{\partial w^l} = 0 \\ & = \frac{1}{n} \sum_x \Bigg( \frac{\partial (y-a^l)^2}{\partial (y-a^l)} \Big(- \frac{\partial (a^l)}{\partial w^l} \Big)\Bigg) \\ & = \frac{1}{n} \sum_x \Bigg( - \frac{\partial (y-a^l)^2}{\partial (y-a^l)} \frac{\partial (a^l)}{\partial w^l} \Bigg) && \text{Brought the } (-) \text{ up front} \\ & = \frac{1}{n} \sum_x \Bigg( -\frac{\partial (y-a^l)^2}{\partial (y-a^l)} \Big(\frac{\partial (\sigma(z^l))}{\partial z^l} \frac{\partial z^l}{\partial w^l} \Big)\Bigg) && \text{Chain Rule} \\ & = \frac{1}{n} \sum_x \Bigg( -\frac{\partial (y-a^l)^2}{\partial (y-a^l)} \frac{\partial (\sigma(z^l))}{\partial z^l} \frac{\partial z^l}{\partial w^l} \Bigg) \end{align*}

Evaluating differentials,

\begin{align*} \frac{\partial (y-a^l)^2}{\partial (y-a^l)} & = 2(y-a^l) \\ \frac{\partial (\sigma(z^l))}{\partial z^l} & = \sigma'(z^l) && \sigma'(x) = \sigma(x)(1-\sigma(x)) \\ \frac{\partial z^l}{\partial w^l} & = \frac{\partial (w^l a^{l-1} + b^l)}{\partial w^l} = a^{l-1} \end{align*}

Therefore, $$\frac{\partial C}{\partial w^l} = \frac{1}{n} \sum_x \Bigg( -\frac{\partial (y-a^l)^2}{\partial (y-a^l)} \frac{\partial (\sigma(z^l))}{\partial z^l} \frac{\partial z^l}{\partial w^l} \Bigg) = \frac{1}{n} \sum_x (-2(y-a^l)\sigma'(z^l)a^{l-1})$$

I accumulate a negative sign in front which makes me nervous because I did not see it at other places (though I do understand the choice of a half in front of the quadratic cost function but I chose not to add it).

So, is my derivation of $$\frac{\partial C}{\partial w^l}$$ correct?