What should the divisor be in MSE? The mean squared error for two vectors is calculated using
$$mse(x, y) = \frac{1}{n}\sum_{i=1}^n(x_i-y_i)^2\text{, with }x, y \in \mathbb{R^n}\text{.}$$
That is straightforward, but I have a hard time figuring out the correct way to apply this formula to machine learning algorithms where you have to deal with multiple samples which would be stored in matrices such as $X, Y \in \mathbb{R^{n\times m}}$. In such a matrix there is a total of $m$ samples and each sample has $n$ features.
What would the correct definition of MSE be in this case? Do you divide by $m$ (the number of sample vectors)
$$mse(X, Y) = \frac{1}{m} \sum_{i=1}^m \sum_{j=1}^n (x_{ij} - y_{ij})^2\text{,}$$
or do you divide by $n \cdot m$ (the number of scalars in the matrix)
$$mse(X, Y) = \frac{1}{m\cdot n} \sum_{i=1}^m \sum_{j=1}^n (x_{ij} - y_{ij})^2\text{?}$$
I'm sorry if this is a silly question. It may very well be since statistics is not something I deal with regularly.
 A: MSE is Mean Square Error. So you first of all must compute your error (get different between vectors), compute it square and for total result compute your mean.
For example:
$$
Y = \begin{bmatrix}
1 & 2\\
3 & 4\\
2 & 0
\end{bmatrix}
$$
$$
\hat{Y} = \begin{bmatrix}
0.5 & 2.5\\
1 & 1\\
1 & 1
\end{bmatrix}
$$
Where $Y$ is your target result and $\hat{Y}$ is your output. Every matrix contains 3 samples and every sample has 2 features. SO the MSE for this situation would be
$$
MSE = \frac{1}{3 \cdot 2} \sum_{i=1}^3 \sum_{i=1}^2 (Y - \hat{Y}) ^ 2 = \frac{1}{6} \sum_{i=1}^3 \sum_{i=1}^2 \begin{bmatrix}
0.5 & -0.5\\
2 & 3\\
1 & -1
\end{bmatrix} ^ 2 = \frac{1}{6} \sum_{i=1}^3 \sum_{i=1}^2 \begin{bmatrix}
0.25 & 0.25\\
4 & 9\\
1 & 1
\end{bmatrix} = \frac{1}{6} \sum_{j=1}^3 \begin{bmatrix}
0.5\\
13\\
2
\end{bmatrix} = \frac{15.5}{6} \approx 2.583
$$
As for me, coefficient for which you divide total sum result didn't play a huge role. It exist to control your error result in normal bounds. If you didn't use it you will also get valid error result but for large datasets number would be really big.
If you try use this error in some neural networks you must understand that this fraction just a scalar which in training step would be multiply by step which is also a scalar. So in learning process this value control your step, but if you would remove this fraction and will decrease your step the learning result would be the same if you even didn't divide your total error sum by number of elements in matrix.
