I need to use the criterion


for a linear regression model to estimate the test MSE. Here $\beta$'s are the regression parameters and $V$ is the population covariance matrix. I have to split the data

• train (to estimate $\beta$'s)

• validation (to estimate hyperparameters) and

• test sets (to estimate Test MSE).

But the criterion given above does not include any term about the test set.

How could I use this criterion to estimate the test MSE?

  • $\begingroup$ It is strange to know the covariance of a random variable $\hat\beta$ but not to know its expectation: how does that work?? $\endgroup$
    – whuber
    Oct 30 '20 at 19:40
  • $\begingroup$ Why not inverse of $V$? That would give you a much more sensible measure (Mahalanobis distance). $\endgroup$ Mar 7 at 16:08

The MSE is the mean of the squared errors. So to compute the mean squared error on you test data you:

  1. do a prediction on your test set
  2. compute (for each observation) the difference between actual value and prediction
  3. square those differences
  4. take the mean of those squared differences.

The Wikipedia page is quite complete on this topic and includes the mathematical notation: https://en.wikipedia.org/wiki/Mean_squared_error

  • $\begingroup$ I know that definition of Test MSE. But I have to use the criterion in the question. There are some papers using this criterion and I have to do the same for a simulation study. $\endgroup$
    – mert
    Oct 22 '18 at 8:29
  • $\begingroup$ Is not on those papers how to proceed? $\endgroup$ Oct 30 '20 at 19:43

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.