Suppose, we have a normal linear model given regressor matrix $X$, can one expect a boosted linear model with the same regressor matrix to be have better performance?
I mean the following: I have a normal linear model trained on the train set with, say, $R^2 = 0.95$ and this model has also good predictive performance on the cross validation sets, so I believe is not overfitted. Would you expect the xgboost linear model to perform better? Personally, I don't expect this behavior.
So I tried and the CV score is even little worse. Then I thought, in principle, boosting with linear base learner is just sum of linear models, hence it is always linear model. Then it does not make much sense to me, to use linear model as a base learner.