I am trying to work on Bayesian linear regression. i have Classical and Bayesian regression estimates, now i want to find the Mean square error (MSE) for both approaches. Is the formula to find MSE will remain same i.e $MSE = \sum_i(y_i-\hat{y})^2/(n-k)$. k(parameters=2).


Yes and no. It will stay the same. MSE is MSE, the method of estimation you used does not matter. The only difference is that in classical approach you get a point estimate and in Bayesian you get a distribution of likely values and if you want to compare both approaches using MSE, you need to decide on some kind of point estimate as well (e.g. mean, median, or mode of posterior distribution).

  • 2
    $\begingroup$ From a decision theoretic perspective a quadratic loss function implies the posterior mean as a point estimate. $\endgroup$ – conjugateprior Dec 11 '14 at 15:59
  • $\begingroup$ Thank you very much Tim, we use posterior means of distribution of regression coefficients to find yhat. Is it ok? $\endgroup$ – Tahir Malik Dec 17 '14 at 9:44
  • $\begingroup$ I would rather estimate $\hat{y}$ in the model and then take averages of those estimates. $\endgroup$ – Tim Dec 17 '14 at 9:54

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.