1
$\begingroup$

Why isn't stepwise regression, like backward elimination, used for Bayesian models? What is generally used to find insignificant variables in bayesian methods? Or does one simply not worry about insignificant explanatory variables, when doing bayesian inference? if so, why?

$\endgroup$
1
  • $\begingroup$ Search for "spike and slab priors" and "Bayesian model averaging" - they are probably the closest methods to stepwise algorithms $\endgroup$ – probabilityislogic Aug 19 '16 at 13:37
1
$\begingroup$

If I could hazard a guess as to the unpopularity of stepwise regression among Bayesians.

  1. Different stepping patterns can produce inconsistent results.
  2. It can't explore the full space of configurations easily because of the one-at-a-time choice of variables.
  3. The choice of model is being explored, and given the model coefficients being estimated etc but the model choice is made ad hoc and not probabilistically as it would be in a fully Bayesian treatment.
  4. Bayes has gotten more popular with MCMC in the late 80s onwards. I believe stepwise regression is an older method and the bad points above were known at that time.
  5. Availability of alternatives, e.g. the Bayesian Lasso, sparse models.
$\endgroup$
2
  • 1
    $\begingroup$ On this site you will find unpopularity of stepwise models among Frequentists as well. $\endgroup$ – EdM Jun 9 '15 at 14:01
  • $\begingroup$ True, but the question was about Bayesians and why it isn't really discussed in that sort of literature. $\endgroup$ – conjectures Jun 9 '15 at 15:37

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.