# Using test MSE to compare models

I was reading through the definition of MSE and the formula I have found in all articles is the one shown in https://en.wikipedia.org/wiki/Mean_squared_error , which is the expected squared difference between the estimated values (y hat) and what is estimated (y). So far so good.

However, when looking at the MSE in a regression setting, I have seen various articles stating that the MSE is sometimes used to refer to the unbiased estimate of error variance, in which case the denominator is not the sample size n but rather the degrees of freedom.

I agree that the formula for the unbiased estimator of the error variance is the one divided with the d.f, but my question is when comparing regression models with the MSE should I use the formula that has as a denominator the sample size (n) or the d.f (n-p-1)? Also do you know if different packages in r use different formulas?

• What does "test MSE" mean? You write, "when comparing regression models with the test MSE". – Heteroskedastic Jim Oct 18 '18 at 0:05
• I mean the MSE computed in the test data set. – ALEX.VAMVAS Oct 18 '18 at 0:09
• So you do several regressions on training data then apply each model to test data and how do you compute the MSE for each model in the test data is your question? – Heteroskedastic Jim Oct 18 '18 at 0:11
• Yes correct. Which formula should i use and whether different r packages use different ones for regression – ALEX.VAMVAS Oct 18 '18 at 0:12
• You can use the information you have provided in the comments to modify your question so the question is clearer. I think one may use the traditional MSE formula (with n as divisor). The model selection is blind to the test data, there should be no need to consider the number of parameters. I do not know what R packages do. You can check their documentations. I hope you get a more informed answer. – Heteroskedastic Jim Oct 18 '18 at 0:30