3
$\begingroup$

I am doing some tutoring for an AS-Level maths student and unfortunately for me they are doing statistics. This is not my strong point, mainly from the point of view of remembering all of the definitions, formulae and statistics. The workbook they had asked them to work out the Mean, the Variance, Standard Deviation, the Mean Squared Deviation and Root Mean Squared Deviation.

The Variance is defined on wikipedia as

$Var= \frac{\sum_{i=0}^n (x_i - \bar{x})^2}{n-1}$

The Means Squared Deviation is defined on wikipedia as

$MSD = \frac{\sum_{i=0}^n (x_i - \bar{x})^2}{n}$

except for $\bar{x}$ expected value as opposed to $\hat{y_i}$.

The Standard Deviation and Root Mean Squared Deviation would be the square roots of the above respectively.

Elsewhere on the internet the is some ambiguity. Even within the Variance wiki page the two formulae, MSD and Var, are referenced as types of variance.

The subtle difference of $n$ vs $n-1$ was not clearly defined within the student's notebook or textbook nor explained why there is a difference. The student asked me why there was a difference and I gave some "it's a sample vs population thing - go with it".

So, in summary, my final short questions are,

  1. What is the difference between Var and MSD? Are the above definitions correct?
  2. Is the MSD just another name for the population variance?
  3. Does Var imply using the sample variance?
  4. When would you use either MSE or Var over the other?
Improve this question
$\endgroup$
2
  • 1
    $\begingroup$ I don't think you are quite correct in putting the gap between "variance" and "MSD" (or MSE, Mean squared error). For both this and that can be computed as unbiased and as biased, when seen as estimates of the population dispersion. I think that variance and MSE (or MSD) are basically synonimic, with MSE being a wider term. We use "MSE" mostly in modeling with number of parameters to hold in the df equal or greater than just 1 (the mean). Therefore the unbiased general formula of MSE is SSE/(n-m) (like here). For usual "variance", m=1. $\endgroup$
    – ttnphns
    Oct 10, 2016 at 9:02
  • $\begingroup$ It is the school, or rather the syllabus, that are putting a gap between them. I think there is a bigger gap between them though in respect to what we use them for. As you say we use RMSE in modelling and machine learning with the difference from the dependant variable as opposed to the mean. I think the syllabus might be doing this to get around teaching population vs sample variance... As Tim said the RMSE is more descriptive. $\endgroup$
    – josh
    Oct 10, 2016 at 9:38

1 Answer 1

Reset to default
6
$\begingroup$

Squared difference divided by $n$ or by $n-1$ are both variance. The only difference is that in the second case it is an unbiased estimator of variance. Taking square root of it leads to estimating standard deviation.

I guess that mean squared deviation and root mean squared deviation are used more commonly in machine learning field where you have mean squared error and it's square root that are often used.

I also guess that some people prefer using mean squared deviation as a name for variance because it is more descriptive -- you instantly know from the name what someone is talking about, while for understanding what variance is you need to know at least elementary statistics.

Check the following threads to learn more:

Improve this answer
$\endgroup$
4
  • $\begingroup$ Thanks for clarifying the variance. Does this mean that machine learning makes an assumption and uses the population variance? Is there a reason behind this? $\endgroup$
    – josh
    Oct 10, 2016 at 8:51
  • $\begingroup$ @josh I guess machine learning people in general are not interested in things as population variance (it's statistics domain), I meant them calculating mean squared error. $\endgroup$
    – Tim ♦
    Oct 10, 2016 at 8:54
  • $\begingroup$ I would agree with you there. I use machine learning every day in my day job and certainly don't worry about variance. I guess I'm still struggling to see why they are teaching MSE as what should probably be population variance and variance as specifically sample variance. Maybe this is one for matheducators.stackexchange.com $\endgroup$
    – josh
    Oct 10, 2016 at 9:13
  • $\begingroup$ @josh one reason can be simply that not always unbiased estimator equals "better" estimator. Sometimes you may prefer biased estimator over the unbiased one. So non-unbiasness is not the reason for rejecting it. $\endgroup$
    – Tim ♦
    Oct 10, 2016 at 9:34

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.

Not the answer you're looking for? Browse other questions tagged or ask your own question.