Why isn't stepwise regression, like backward elimination, used for Bayesian models? What is generally used to find insignificant variables in bayesian methods? Or does one simply not worry about insignificant explanatory variables, when doing bayesian inference? if so, why?
If I could hazard a guess as to the unpopularity of stepwise regression among Bayesians.
- Different stepping patterns can produce inconsistent results.
- It can't explore the full space of configurations easily because of the one-at-a-time choice of variables.
- The choice of model is being explored, and given the model coefficients being estimated etc but the model choice is made ad hoc and not probabilistically as it would be in a fully Bayesian treatment.
- Bayes has gotten more popular with MCMC in the late 80s onwards. I believe stepwise regression is an older method and the bad points above were known at that time.
- Availability of alternatives, e.g. the Bayesian Lasso, sparse models.