# What should the divisor be in MSE?

The mean squared error for two vectors is calculated using

$$mse(x, y) = \frac{1}{n}\sum_{i=1}^n(x_i-y_i)^2\text{, with }x, y \in \mathbb{R^n}\text{.}$$

That is straightforward, but I have a hard time figuring out the correct way to apply this formula to machine learning algorithms where you have to deal with multiple samples which would be stored in matrices such as $X, Y \in \mathbb{R^{n\times m}}$. In such a matrix there is a total of $m$ samples and each sample has $n$ features.

What would the correct definition of MSE be in this case? Do you divide by $m$ (the number of sample vectors)

$$mse(X, Y) = \frac{1}{m} \sum_{i=1}^m \sum_{j=1}^n (x_{ij} - y_{ij})^2\text{,}$$

or do you divide by $n \cdot m$ (the number of scalars in the matrix)

$$mse(X, Y) = \frac{1}{m\cdot n} \sum_{i=1}^m \sum_{j=1}^n (x_{ij} - y_{ij})^2\text{?}$$

I'm sorry if this is a silly question. It may very well be since statistics is not something I deal with regularly.

• For vectors, each dimension usually means something and it wouldn't make sense to subtract the first dimension of one vector from the second dimension of a different vector (i.e. $x_1 - y_2$). In this case, is there anything wrong subtracting the first sample of one matrix from the second sample of a different matrix? (i.e. $x_{1j} - y_{2j}$) – Eric Farng May 16 '15 at 1:54
• @EricFarng I need $mse$ to return a single number that I can use as the neural network's error. It needs to be a scalar and not a vector so I can then run the back-propagation algorithm. – Paul Manta May 16 '15 at 6:09
• In that case, I believe the two are equivalent since the difference can be absorbed into the learning rate. – Eric Farng May 16 '15 at 12:15
• It doesn't really matter if you add some constant to the objective function or multiply it by a positive number since optimum values doesn't change The common practice is division by number of samples ("m") as it gives average over all samples. – yasin.yazici May 16 '15 at 16:53

MSE is Mean Square Error. So you first of all must compute your error (get different between vectors), compute it square and for total result compute your mean. For example: $$Y = \begin{bmatrix} 1 & 2\\ 3 & 4\\ 2 & 0 \end{bmatrix}$$
$$\hat{Y} = \begin{bmatrix} 0.5 & 2.5\\ 1 & 1\\ 1 & 1 \end{bmatrix}$$
Where $Y$ is your target result and $\hat{Y}$ is your output. Every matrix contains 3 samples and every sample has 2 features. SO the MSE for this situation would be $$MSE = \frac{1}{3 \cdot 2} \sum_{i=1}^3 \sum_{i=1}^2 (Y - \hat{Y}) ^ 2 = \frac{1}{6} \sum_{i=1}^3 \sum_{i=1}^2 \begin{bmatrix} 0.5 & -0.5\\ 2 & 3\\ 1 & -1 \end{bmatrix} ^ 2 = \frac{1}{6} \sum_{i=1}^3 \sum_{i=1}^2 \begin{bmatrix} 0.25 & 0.25\\ 4 & 9\\ 1 & 1 \end{bmatrix} = \frac{1}{6} \sum_{j=1}^3 \begin{bmatrix} 0.5\\ 13\\ 2 \end{bmatrix} = \frac{15.5}{6} \approx 2.583$$