1
$\begingroup$

I have a set of possible predictors for a binary outcome. In order to obtain the best model, I start from the zero model, and do a stepwise selection (in R) in order to obtain the best predictors. The way R (or any other program) does this is by minimizing the AIC. How come the final chosen subset can still contain insignificant predictors?

$\endgroup$
  • 2
    $\begingroup$ The tag info specifically states this is not for forward or backward selection and urges to use stepwise-regression instead. $\endgroup$ – Richard Hardy Mar 30 at 16:40
  • $\begingroup$ You'll need to give detail about the procedures you're using for this to be answerable. Please edit the question to describe the tests you're performing & the selection criteria you've chosen for forward & backward steps, & explain how you're calculating p-values in the final model. An example might help too. $\endgroup$ – Scortchi - Reinstate Monica Mar 30 at 16:42
  • 1
    $\begingroup$ in general a single parameter will be selected/kept via AIC (Delta-AIC is favorable) if its p<0.16, so I would be unsurprised to see variables with 0.05<p<0.16 in the final model $\endgroup$ – Ben Bolker Mar 30 at 18:48
2
$\begingroup$

You wrote:

In order to obtain the best model, I start from the zero model, and do a stepwise selection (in R) in order to obtain the best predictors.

As has been discussed here and elsewhere, this does not produce the "best" model for any sensible definition of "best". It produces a model where the parameter estimates are biased away from 0, the standard errors are biased toward 0, the final model is too complex and (most importantly) the data analyst (you) have been prohibited from thinking.

As to which variables are left in the final model and their (incorrectly estimated) p values in the final model, there's no mystery. You are selecting on AIC rather than p value. If you are using the stepAIC model, then it doesn't even let you select p values to use for entering and leaving the model.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.