49
$\begingroup$

Looking at the Wikipedia definitions of:

It looks to me that

$$\text{MSE} = \frac{1}{N} \text{RSS} = \frac{1}{N} \sum (f_i -y_i)^2$$

where $N$ is he number of samples and $f_i$ is our estimation of $y_i$.

However, none of the Wikipedia articles mention this relationship. Why? Am I missing something?

Improve this question
$\endgroup$
3
  • 8
    $\begingroup$ I know this seems unhelpful and kind of hostile, but they don't mention it because it is obvious. Also, you want to be a little careful, here. Usually, when you encounter a MSE in actual empirical work it is not $RSS$ divided by $N$ but $RSS$ divided by $N-K$ where $K$ is the number (including the intercept) of right-hand-side variables in some regression model. $\endgroup$
    – Bill
    Oct 23, 2013 at 14:49
  • 17
    $\begingroup$ @Bill: Well, it is exactly the kind of relationship that typically leads to articles being linked on Wikipedia. Your point regarding the degree of freedoms also shows that is not quite as obvious and definitely something worth mentioning. $\endgroup$
    – bluenote10
    Oct 29, 2015 at 11:18
  • 6
    $\begingroup$ @Bill: Agree, however obviousness is very subjective. The statistics/machine learning grey area is littered with notation hell and therefore it is good to be explicit. $\endgroup$
    – rnoodle
    Jul 26, 2018 at 14:24

3 Answers 3

Reset to default
43
$\begingroup$

Actually it's mentioned in the Regression section of Mean squared error in Wikipedia:

In regression analysis, the term mean squared error is sometimes used to refer to the unbiased estimate of error variance: the residual sum of squares divided by the number of degrees of freedom.

You can also find some informations here: Errors and residuals in statistics It says the expression mean squared error may have different meanings in different cases, which is tricky sometimes.

Improve this answer
$\endgroup$
9
$\begingroup$

But be aware that Sum of Squared Errors (SSE) and Residue Sum of Squares (RSS) sometimes are used interchangeably, thus confusing the readers. For instance, check this URL out.

Strictly speaking from statistic point of views, Errors and Residues are completely different concepts. Errors mainly refer to difference between actual observed sample values and your predicted values, and used mostly in the statistic metrics like Root Means Squared Errors (RMSE) and Mean Absolute Errors (MAE). In contrast, residues refer exclusively to the differences between dependent variables and estimations from linear regression.

Improve this answer
$\endgroup$
-1
$\begingroup$

I don´t think this is correct here if we consider MSE to be the sqaure of RMSE. For instance, you have a series of sampled data on predictions and observations, now you try to do a linear regresion: Observation (O)= a + b X Prediction (P). In this case, the MSE is the sum of squared difference between O and P and divided by sample size N.

But if you want to measure how linear regression performs, you need to calculate Mean Squared Residue (MSR). In the same case, it would be firstly calculating Residual Sum of Squares (RSS) that corresponds to sum of squared differences between actual observation values and predicted observations derived from the linear regression.Then, it is followed for RSS divided by N-2 to get MSR.

Simply put, in the example, MSE can not be estimated using RSS/N since RSS component is no longer the same for the component used to calculate MSE.

Improve this answer
$\endgroup$
2
  • 3
    $\begingroup$ I don't understand this answer. $\endgroup$
    – Michael R. Chernick
    Jun 15, 2019 at 18:53
  • $\begingroup$ Look, based on the mentioned example of sampled prediction and observed data values, the linear regression is established: Observation (O)= a + b X Prediction (P) (a, b are intercept and slope respectively). In this case, MSE = Σ(O-P)^2/n, where Σ(O-P)^2 is the Sum of Squared Erros (SSE) and n is the sample size. However, Mean Squared Residues (MSR) = Σ(O-O´)^2/n-2, where Σ(O-O´)^2 equals to Residue Sum of Squares (RSS) and O`=a+b X P. MSR and RSS are mainly used for testing overall significance of linear regression. Also note, SSE = Systematic Erros (SE) + RSS, where SE= Σ(P-O´)^2 $\endgroup$
    – Dr.CYY
    Jun 16, 2019 at 16:46

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.

Not the answer you're looking for? Browse other questions tagged or ask your own question.