# How do you / can you compare Bayesian vs. frequentist regression models?

I am working on a regression model to predict a target variable in a dataset with over 100 features. Three different regression models are defined and fit in order to compare their performance using $$R^2$$ and the errors: MAE, MSE, RMSE. E.g.:

R^2: 0.89
MSE: 0.13
RMSE: 0.3

I am working with Pyro and following their documentation where they evaluate the Bayesian model by calculating the posterior on predictive samples; but am not sure I understand how one would compare both models using a metric.

AFAIK it does not make sense to calculate an $$R^2$$ score in a Bayesian model so the questions are:

1. Is it possible to compare non-Bayesian vs. Bayesian regression models using the same metric?
2. What is the best way to do that?
• Welcome to the site. Questions about coding (as your second question) are off topic here. If you drop that, then I think you have a good question for this site. – Peter Flom Feb 26 at 12:11
• Please register your account (you can find information on how to do this in the My Account section of our help center), then you will be able to edit & comment on your own question. – gung Feb 26 at 19:20
• I edited your question for clarity. By "regressor", you seem to mean "regression model" (a regressor is a feature in a regression model). Please ensure that it reflects what you want to ask. – gung Feb 26 at 19:25
• You can use an information criterion like AIC to compare the models. – Alex Jul 2 at 2:12