is it okay if r-square and RMSE for the lasso be worse than normal linear regression?

Question

I am comparing three regression models, simple linear regression, Lasso and Bayesian Lasso then the R-Square and RSME for them are

r2 score for Bays model is 0.10171034421952285
rmse score for Bays model is 0.7888288293573854
r2 score for regression model is 0.3303114752643104
rmse score for regression model is 0.6811001935995461
r2 score for Lasso model is -0.009865062766038157
rmse score for Lasso model is 0.8363850259509928

You can see that for Baysian Lasso and Lasso the R-Square and rmse are not good. Can I still say the results for those two is valid or how can I argue them?

Answer 1

Sure, it just means that the model with the best fit is the OLS model.

We like alternatives to OLS, such as regularized models, because they can achieve better fits than OLS, but that better fit is not guaranteed. Your situation is one where the regularized models are not as good as the OLS model.

A potential remedy is to tune the amount of regularization.

Answer 2

If these are fits on training data, Lasso and Bayesian regression models are supposed to not fit the training data as well as simple linear regression. This is to avoid overfitting. However, a negative r-squared value on training data indicates that something has gone wrong, although I can't say what from the information provided.

If these are fits on test data, the results would indicate that Lasso/Bayes are underfitting the data, and you can easily find resources on how to avoid this.