Stepwise regression predicts as well as Bayesian model averaging and boosting, why? I used Bayesian model averaging and gamboostLSS for variable selection on real data with 23 covariates. A 10-fold CV was used to asses performance. I tested stepwise regression out of curiosity and found that it predicted almost as well as the other two methods. I know stepwise regression has a lot of short comings so I actually expected it do much worse. I'm not sure how to interpret this. Could there be something wrong with my codes? Is it reasonable to expect stepwise regression to do worse than BMA and gamboostLSS, which are much more solid methods? 
My data is relatively small with 660 observations over 11 years. 
Any thoughts on the matter? 
 A: I skip the part about the possibility of errors in your codes because you haven't shown the codes. Concerning predictive performance of stepwise regression, you're not the first one to find that in a real world application, it performs better than one would expect: see here. However, there are good theoretical reasons why stepwise regression should tend to overfit the training data set, resulting in poor predictive performance. Thus the fact that on some specific data sets it does a good work, may well be due to chance. Have you addressed the possible high variance of the cross-validation estimator, by using repeated cross-validation? Have you tried other ways to estimate the generalization error, for example by using bootstrap? 
Even if the predictive performance happens to be good on a specific data set, I don't see why one would want to use stepwise regression today, when we have LASSO as a great way to estimate a sparse linear regression model (note that gamboostLSS doesn't estimate the linear regression model, but GAMLSS which is a much more complicated model). You cannot reliably make inference with stepwise regression, because the p-values don't have the standard interpretation. You could of course compute perfectly valid p-values for stepwise regression using sample splitting, but you would lose power that way. Instead, we have a significance test for LASSO which uses all the data.
A: First, if there exists a decent predictive linear model to be found among your variables, then it's unsurprising one model could do about as well as model averaging or boosting. This seems to be the case for your data.
Next, if predictors in your dataset don't have too many large correlations among one another, stepwise regression has a high chance of finding a close-to-best predictive model. (You shouldn't trust the p-values for hypothesis testing etc., but for pure prediction it may be fine.) In the correlation matrix you posted, it seems you have mostly small correlations, so it's unsurprising that a stepwise model's performance would be about as good as any other.
Finally, cross-validation isn't ideal for model selection if you use a large training:testing ratio, as with 10-fold. If you really want to be sure you chose the "best" model, you'd have to train on smaller splits and test on larger ones, so that the difference between the best and near-best models' performances is measured more precisely. With 10-fold CV, it's unsurprising that all "decent" models show similar estimated performance.
In short, I see nothing wrong with stepwise doing about as well as model averaging or boosting in your case.
