4
$\begingroup$

I'm building a model that predicts rather rare event. The data set consists of about 240.000 examples from which only 1410 are positive. One of the variables from the data set, let's call it $X_{1}$ is a binary variable with value 1 in only about 1% examples. From the knowledge about the event I know that it is impossible to occur an event when $X_{1} = 1$. When I include $X_{1}$ in logistic regression model then it has highly negative coefficient, but it is also highly non significant. The coefficient is -12 and the standard error is 122.

Question:
Is it valid to keep the variable $X_{1}$ despite the low significance?

$\endgroup$
  • 2
    $\begingroup$ Partly relevant answer also here. $\endgroup$ – ttnphns Sep 17 '13 at 10:12
  • 2
    $\begingroup$ And see here re separation. If you're using maximum likelihood parameter estimates & you're judging significance by Wald standard errors they're badly wrong when you have separation. $\endgroup$ – Scortchi - Reinstate Monica Sep 17 '13 at 21:06
5
$\begingroup$

In principle it is perfectly OK to include non-significant variables. You are interested in the effect of a variable, and the fact that that effect is not significant is valueable information. The logical way to convey that information is to have it be part of your model and show that it is not significant.

However, in your case I would be worried about perfect separation. If that is the case the non-significance is not really informative and I would not include that variable or use one of the alternatives proposed in the link.

$\endgroup$
  • 1
    $\begingroup$ Removing the variable on which separation occurs when you have prior knowledge that it reflects a true impossibility (as the OP says) is not a good idea. $\endgroup$ – Scortchi - Reinstate Monica Sep 17 '13 at 21:09
  • 1
    $\begingroup$ IMHO removing the variable is not a good idea, although I'm a bit concerned that it might lead to some numerical instability. Firth LR seems to be a good solution and fortunatelly it is both R and SAS. Maarten thanks for the link, it helped me to found Firth LR! $\endgroup$ – Tomek Tarczynski Sep 18 '13 at 7:49
  • 1
    $\begingroup$ @Tomek: Note that Firth's penalization will change your "impossible" to "improbable". $\endgroup$ – Scortchi - Reinstate Monica Sep 18 '13 at 8:44
1
$\begingroup$

As you say, it seems that X1 is quite a good predictor for your response. It is not significant itself in Wald test maybe because the strange structure of the data set, or maybe some other variable can explain such effect. The Wald test can not tell us too much in such sense.

Maybe a Chi-square test about adding that X1 can tell us more, and also AIC, BIC.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.