How to prove Mean Squarred Error (MSE) I would like to prove this equation of Mean Squared Error (MSE):


*

*m is the number of training instances.

*X is a m × n matrix containing all the feature values (excluding
labels) of all instances in the dataset.

*X^T is the transpose of matrix X.

*w is the model’s parameter vector, containing the bias term w0 and the feature weights w1 to wn.

*y is the vector of target values containing y(1) to y(m) values (training instances).

The above function is the gradient of the minimum squared error loss function:

where,

*

*m is the number of training instances.

*hw is the hypothesis function, using the model parameters w.

*x(i) is a n × 1 vector of all the feature values (excluding the label) of the i-th instance in the dataset, and y(i) is its label (the desired output for this instance)

I would really appreciate any clue for a start at least or any relevant link where I could find more information.
 A: First, let's identify what we mean here:
$$\text{MSE}=\frac{1}{m}\sum (X_iW-Y_i)^2$$
And the gradient is a vector of partial derivatives:
$$\nabla_W\text{MSE}=\left[\frac{\partial \text{MSE}}{\partial W_j} \right]$$
$$\frac{\partial \text{MSE}}{\partial W_j}=
\frac{1}{m}\frac{\partial}{\partial W_j}\sum (X_iW-Y_i)^2=\\
\frac{1}{m}\sum 2(X_iW-Y_i)\frac{\partial}{\partial W_j}(X_iW-Y_i)
$$
Remember that $X_iW = \sum_k x_{ik}W_{k}$
This last derivative is then simply:
$$\frac{\partial}{\partial W_j}(X_iW-Y_i) = \frac{\partial}{\partial W_j}(\sum_k x_{ik}W_{k}-Y_i)=x_{ij}$$
Then, recognizing that the resulting sum is a vector product:
$$\frac{\partial \text{MSE}}{\partial W_j}=
\frac{1}{m}\sum 2(X_iW-Y_i)x_{ij} = \frac{2}{m} x_j^T(XW-Y)
$$
Where $x_j$ is the $j$-th column of $X$
Then reorganizing into the columns of the gradient we can recognize that it's a matrix product, arriving at the result:
$$\nabla_W\text{MSE}=\left[\frac{2}{m} x_j^T(XW-Y) \right]_j = \frac{2}{m}X^T(XW-Y)$$

Using matrix calculus:
$$\text{MSE}=\frac{1}{m}\epsilon^T\epsilon =\frac{1}{m} (XW-Y)^T(XW-Y)$$
$$\frac{\partial \text{MSE}}{\partial W}=
\frac{1}{m} \frac{\partial}{\partial W}(XW-Y)^T(XW-Y)=\\
\frac{1}{m} \left(\frac{\partial (XW-Y)^T}{\partial W}(XW-Y) + \left((XW-Y)^T\frac{\partial (XW-Y)}{\partial W}\right)^T\right)=\\
\frac{1}{m} \left(X^T(WX-Y) + ((WX-Y)^TX)^T\right) = \frac{2}{m} X^T(WX-Y)$$
