33
$\begingroup$

Looking at the Wikipedia definitions of:

It looks to me that

$$\text{MSE} = \frac{1}{N} \text{RSS} = \frac{1}{N} \sum (f_i -y_i)^2$$

where $N$ is he number of samples and $f_i$ is our estimation of $y_i$.

However, none of the Wikipedia articles mention this relationship. Why? Am I missing something?

$\endgroup$
  • 6
    $\begingroup$ I know this seems unhelpful and kind of hostile, but they don't mention it because it is obvious. Also, you want to be a little careful, here. Usually, when you encounter a MSE in actual empirical work it is not $RSS$ divided by $N$ but $RSS$ divided by $N-K$ where $K$ is the number (including the intercept) of right-hand-side variables in some regression model. $\endgroup$ – Bill Oct 23 '13 at 14:49
  • 11
    $\begingroup$ @Bill: Well, it is exactly the kind of relationship that typically leads to articles being linked on Wikipedia. Your point regarding the degree of freedoms also shows that is not quite as obvious and definitely something worth mentioning. $\endgroup$ – bluenote10 Oct 29 '15 at 11:18
  • 2
    $\begingroup$ @Bill: Agree, however obviousness is very subjective. The statistics/machine learning grey area is littered with notation hell and therefore it is good to be explicit. $\endgroup$ – rnoodle Jul 26 '18 at 14:24
32
$\begingroup$

Actually it's mentioned in the Regression section of Mean squared error in Wikipedia:

In regression analysis, the term mean squared error is sometimes used to refer to the unbiased estimate of error variance: the residual sum of squares divided by the number of degrees of freedom.

You can also find some informations here: Errors and residuals in statistics It says the expression mean squared error may have different meanings in different cases, which is tricky sometimes.

$\endgroup$
4
$\begingroup$

But be aware that Sum of Squared Erros (SSE) and Residue Sum of Squares (RSS) sometimes are used exchangeablly, thus confusing the readers. For instance, check this Url: https://365datascience.com/sum-squares/ for more information on linear regression.

Strictly speaking from statistic point of views, Errors and Residues are completely different concepts. Errors mainly refer to difference between actual observed sample values and your predicted values, and used mostly in the statistic metrics like Root Means Squared Errors (RMSE) and Mean Absollute Errors (MAE). In contrast, residues refer exclusively to the differences between dependent variables and estimations from linear regression.

$\endgroup$
0
$\begingroup$

I don´t think this is correct here if we consider MSE to be the sqaure of RMSE. For instance, you have a series of sampled data on predictions and observations, now you try to do a linear regresion: Observation (O)= a + b X Prediction (P). In this case, the MSE is the sum of squared difference between O and P and divided by sample size N.

But if you want to measure how linear regression performs, you need to calculate Mean Squared Residue (MSR). In the same case, it would be firstly calculating Residual Sum of Squares (RSS) that corresponds to sum of squared differences between actual observation values and predicted observations derived from the linear regression.Then, it is followed for RSS divided by N-2 to get MSR.

Simply put, in the example, MSE can not be estimated using RSS/N since RSS component is no longer the same for the component used to calculate MSE.

$\endgroup$
  • 2
    $\begingroup$ I don't understand this answer. $\endgroup$ – Michael R. Chernick Jun 15 '19 at 18:53
  • $\begingroup$ Look, based on the mentioned example of sampled prediction and observed data values, the linear regression is established: Observation (O)= a + b X Prediction (P) (a, b are intercept and slope respectively). In this case, MSE = Σ(O-P)^2/n, where Σ(O-P)^2 is the Sum of Squared Erros (SSE) and n is the sample size. However, Mean Squared Residues (MSR) = Σ(O-O´)^2/n-2, where Σ(O-O´)^2 equals to Residue Sum of Squares (RSS) and O`=a+b X P. MSR and RSS are mainly used for testing overall significance of linear regression. Also note, SSE = Systematic Erros (SE) + RSS, where SE= Σ(P-O´)^2 $\endgroup$ – Dr.CYY Jun 16 '19 at 16:46

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.