4
$\begingroup$

I'm building a model that predicts rather rare event. The data set consists of about 240.000 examples from which only 1410 are positive. One of the variables from the data set, let's call it $X_{1}$ is a binary variable with value 1 in only about 1% examples. From the knowledge about the event I know that it is impossible to occur an event when $X_{1} = 1$. When I include $X_{1}$ in logistic regression model then it has highly negative coefficient, but it is also highly non significant. The coefficient is -12 and the standard error is 122.

Question:
Is it valid to keep the variable $X_{1}$ despite the low significance?

Improve this question
$\endgroup$
2
  • 2
    $\begingroup$ Partly relevant answer also here. $\endgroup$
    – ttnphns
    Sep 17, 2013 at 10:12
  • 2
    $\begingroup$ And see here re separation. If you're using maximum likelihood parameter estimates & you're judging significance by Wald standard errors they're badly wrong when you have separation. $\endgroup$
    – Scortchi - Reinstate Monica ♦
    Sep 17, 2013 at 21:06

2 Answers 2

Reset to default
5
$\begingroup$

In principle it is perfectly OK to include non-significant variables. You are interested in the effect of a variable, and the fact that that effect is not significant is valueable information. The logical way to convey that information is to have it be part of your model and show that it is not significant.

However, in your case I would be worried about perfect separation. If that is the case the non-significance is not really informative and I would not include that variable or use one of the alternatives proposed in the link.

Improve this answer
$\endgroup$
3
  • 1
    $\begingroup$ Removing the variable on which separation occurs when you have prior knowledge that it reflects a true impossibility (as the OP says) is not a good idea. $\endgroup$
    – Scortchi - Reinstate Monica ♦
    Sep 17, 2013 at 21:09
  • 1
    $\begingroup$ IMHO removing the variable is not a good idea, although I'm a bit concerned that it might lead to some numerical instability. Firth LR seems to be a good solution and fortunatelly it is both R and SAS. Maarten thanks for the link, it helped me to found Firth LR! $\endgroup$
    – Tomek Tarczynski
    Sep 18, 2013 at 7:49
  • 1
    $\begingroup$ @Tomek: Note that Firth's penalization will change your "impossible" to "improbable". $\endgroup$
    – Scortchi - Reinstate Monica ♦
    Sep 18, 2013 at 8:44
1
$\begingroup$

As you say, it seems that X1 is quite a good predictor for your response. It is not significant itself in Wald test maybe because the strange structure of the data set, or maybe some other variable can explain such effect. The Wald test can not tell us too much in such sense.

Maybe a Chi-square test about adding that X1 can tell us more, and also AIC, BIC.

Improve this answer
$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.

Not the answer you're looking for? Browse other questions tagged or ask your own question.