8
$\begingroup$

This question already has an answer here:

I have a question about performing stepwise regression. I realize that there are issues with using stepwise methods, but I have about 30 or so predictors and have constructed an lm object.

m1 <- lm(LEADSforester ~ . , data=dat)
m2 <- lm(LEADSforester ~ 1 , data=dat)
step(m1, m2, direction = "backward")

However, when I run the following line of code, I get an error message.

backBIC <- step(m1, direction="backward", data=dat)

Error in step(m1, direction = "backward") : 
  AIC is -infinity for this model, so 'step' cannot proceed

The same problem occurs when I run the following:

m1 = lm(LEADS ~ ED + Fa + Pu + New + Gr + Vol + Dur + Boun + Visit + views + Nw + 
                Uniq + sits, data=dat)
step(m1, direction="backward")

Error in step(m1, direction = "backward") : 
  AIC is -infinity for this model, so 'step' cannot proceed

What am I doing wrong?

$\endgroup$

marked as duplicate by amoeba, John, whuber Dec 29 '14 at 20:28

This question has been asked before and already has an answer. If those answers do not fully address your question, please ask a new question.

  • 6
    $\begingroup$ @gung I think this may actually contain an underlying statistical problem, though the OP may not realize it. ATMathew: AIC is -infinity when the fit is perfect (has no error), because there's a $\log(s^2)$ term. $\endgroup$ – Glen_b Aug 22 '13 at 16:14
  • 5
    $\begingroup$ ATMathew - it would probably help other people find the cause of their problem if your title referred to the AIC being $-\infty$ rather than to R or the stepwise procedure (neither of which I'd think are specifically related to the direct cause of the problem). $\endgroup$ – Glen_b Aug 22 '13 at 16:22
  • 7
    $\begingroup$ Hmm... You are using 13 parameters (and therefore $2^{13}-1$, which is over 8000, models) to fit only 43 observations. Search our site for overfitting for some advice about that. $\endgroup$ – whuber Aug 22 '13 at 17:38
  • 3
    $\begingroup$ AIC = 2k - 2*log(L). K is the number of parameters in the model and L is the maximized value of the likelihood function. If AIC is infinite you have L << 1. In case of a least squares this can only happen if variance is zero. Probably you have one or a set of covariates that together are perfectly collinear with the outcome ... In other words, there is nothing left in the residuals. $\endgroup$ – mmgm Sep 21 '13 at 19:51
  • 1
    $\begingroup$ @mmgm why you did not to writing this comment as an answer elaborating a little bit more? $\endgroup$ – Tim Dec 12 '14 at 19:13
0
$\begingroup$

Why don't you use Forward feature selection?

min.model <- lm(y ~ 1, data=dat)
fwd.model <- step(min.model, direction = "forward", scope = (~ x1 + x2 + ... xn))

This way the model will only add predictors until it can.

$\endgroup$

Not the answer you're looking for? Browse other questions tagged or ask your own question.