0
$\begingroup$

I recently discovered that MSE can mean multiple things:

  1. MSE for predictor (mean of squared errors)
  2. MSE for estimator (variance+ bias^2)
  3. MSE in regression analysis (residual sum of squares divided by degrees of freedom)

My questions are, 1. Why these multiple interpretations? 2. Are there anymore interpretations of MSE?

$\endgroup$
1
$\begingroup$

MSE or mean squared error literally measures the average squared error for an estimator when estimating a parameter. The different interpretations rise from the estimation of different parameters.

  1. This is when the parameter being estimated is a future value (unobserved response) from current model.
  2. This is when the parameter being estimated is an attribute of a distribution, like the mean etc from statistic
  3. This is when the parameter being estimated is the mean regression function from the OLS line.

In general MSE = $\dfrac{1}{n}\sum$ (parameter - estimate)^2. So the underlying formula is always the same, just the parameters and estimates change from problem to problem.

In regression, sometime MSE refers to Residual sum of squares where then

$$MSE = \dfrac{1}{n-p} \sum(y - y_i)^2 $$

The denominator changes in order for MSE to be an unbiased estimated of the error variance. This is one example of a slight deviation from the general understanding of mean squared error.

$\endgroup$
  • $\begingroup$ could you explain more about why the difference in the formulae in each case? Please. $\endgroup$ – Bach Mar 23 '16 at 14:29
  • 2
    $\begingroup$ Helpful, but your formula needs a divisor. As it stands, it is SSE. (I'd prefer estimate to estimator personally.) $\endgroup$ – Nick Cox Mar 23 '16 at 14:29
  • $\begingroup$ @Bach The basic formula is always the same. But the formula takes a different form, from problem to problem. $\endgroup$ – Greenparker Mar 23 '16 at 14:35
  • $\begingroup$ oh, and why does that happen? because in case of regression, we are estimating from same data, so the denominator reduces by the number of parameters estimated and similarly for estimate and predictor? $\endgroup$ – Bach Mar 23 '16 at 14:36
  • 1
    $\begingroup$ Congratulations with 1000 reputation :) $\endgroup$ – Richard Hardy Mar 23 '16 at 14:41

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.