What's the difference between the variance and the mean squared error? - Cross Validated most recent 30 from stats.stackexchange.com 2019-09-23T07:51:57Z https://stats.stackexchange.com/feeds/question/140536 https://creativecommons.org/licenses/by-sa/4.0/rdf https://stats.stackexchange.com/q/140536 24 What's the difference between the variance and the mean squared error? luciano https://stats.stackexchange.com/users/12492 2015-03-05T19:27:33Z 2017-10-05T23:00:33Z <p>I'm surprised this hasn't been asked before, but I cannot find the question on stats.stackexchange.</p> <p>This is the formula to calculate the variance of a normally distributed sample:</p> <p>$$\frac{\sum(X - \bar{X}) ^2}{n-1}$$</p> <p>This is the formula to calculate the mean squared error of observations in a simple linear regression:</p> <p>$$\frac{\sum(y_i - \hat{y}_i) ^2}{n-2}$$ </p> <p>What's the difference between these two formulas? The only difference I can see is that MSE uses $n-2$. So if that's the only difference, why not refer to them as both the variance, but with different degrees of freedom?</p> https://stats.stackexchange.com/questions/140536/-/140541#140541 25 Answer by Alexis for What's the difference between the variance and the mean squared error? Alexis https://stats.stackexchange.com/users/44269 2015-03-05T20:29:47Z 2015-03-07T15:11:15Z <p>The mean squared error as you have written it for OLS is hiding something:</p> <p>$$\frac{\sum_{i}^{n}(y_i - \hat{y}_i) ^2}{n-2} = \frac{\sum_{i}^{n}\left[y_i - \left(\hat{\beta}_{0} + \hat{\beta}_{x}x_{i}\right)\right] ^2}{n-2}$$</p> <p>Notice that the numerator sums over a function of both $y$ and $x$, so you lose a degree of freedom for each variable, hence $n-2$. In the formula for the sample variance, the numerator is a function of a single variable, so you lose just one degree of freedom in the denominator.</p> <p>However, you are on track in noticing that these are conceptually similar quantities. The sample variance measures the spread of the data around the mean (in squared units), while the MSE measures the vertical spread of the data around the regression line (in squared vertical units).</p> https://stats.stackexchange.com/questions/140536/-/306532#306532 0 Answer by Brajesh Kumar for What's the difference between the variance and the mean squared error? Brajesh Kumar https://stats.stackexchange.com/users/179585 2017-10-05T23:00:33Z 2017-10-05T23:00:33Z <p>In the variance formula, the sample mean approximates the population mean. The sample mean is calculated for a given sample with n data points. Knowing the sample mean leaves us with only n-1 independent data points as the nth data point is constrained by the sample mean, so (n-1) DOF in the denominator in the variance formula. To get the estimated value of y (= b0 + b1*x) in MSE formula, we need to estimate both b0 i.e. the intercept as well as b1 i.e. the slope so we lose 2 DOFs and so is the reason for (n-2) in the denominator in the MSE formula.</p>