0
$\begingroup$

I have a logistic regression model with variables $V_1$ through $V_n$. When I build a full model with all of the variables, I find that $V_1$ and $V_2$ are significant. However, when I build a new model in which I remove $V_1$, I find that, in addition to $V_2$, $V_3$ is now significant. I assume this occurs because $V_1$ and $V_3$ are correlated, so that all the information $V_3$ is captured in $V_1$ in the full model. $V_3$ is of particular interest to me given my hypothesis, so I’d like to know what it’s reasonable to say about $V_3$ in terms of significance. Can I still report that $V_3$ is significant even though it’s masked by $V_1$? Are there further tests I can do that would be informative here?

$\endgroup$

1 Answer 1

1
$\begingroup$

It is important to have theoretical justification for including variables in a model to avoid "fishing" for significance. The first thing you should do is try to understand the underlying relationship between $V_1$, $V_3$, and $y$. Are $V_1$ and $V_3$ similar measures of the same thing, such that including both in the model is redundant, or does $V_1$ confound the relationship between $V_3$ and $y$?

If the former, you could (reasonably) justify dropping one of the covariates from your model, because there is no additional information gained by including both of them in your model as they are measures of the same thing. If the latter, you absolutely should NOT drop $V_1$, as failure to control for $V_1$ would lead to omitted variable bias, resulting in inconsistent parameter estimate for your main covariate of interest, $V_3$. Why would failing to control for $V_1$ bias your results in the second scenario? Suppose the true relationships are as follows:

  • $Corr(V_1,y)≠0$ (there is a relationship between $V_1$ and $y$)
  • $Corr(V_1,V_3)≠0$ (there is a relationship between $V_1$ and $V_3$)
  • $Corr(V_3,y)=0$ (there is NO relationship between $V_3$ and $y$)

If $V_1$ and $V_3$ are correlated and you do not control for $V_1$, the initial observed relationship between $V_3$ and $y$ actually reflects the (true) relationship between $V_1$ and $y$. The parameter estimate on $V_3$ is, in effect, a combination of the effect of $V_3$ on $y$ AND the partial effect of $V_1$ and $y$. Because in reality there is no relationship between $V_3$ and $y$, once you control for the factor that is "responsible" for the observed relationship (i.e., $V_1$), the coefficient on $V_3$ is no longer significant.

I would report that $V_3$ was significantly associated with $y$ in bivariate analysis, but that the relationship becomes non-significant after controlling for $V_1$ and discuss why you think this is. You may find it helpful to dig into the literature for insight on the relationship between $V_1$, $V_3$, and $y$.

$\endgroup$
2
  • $\begingroup$ Thanks! What's the best test of association between $V_3$ and $y$ that you'd recommend performing here? $\endgroup$
    – nan
    Apr 12, 2015 at 13:36
  • $\begingroup$ I want to clarify that when I say "true relationship," I mean the relationship at the population level. Very often, this is not observable to us, as we work only with a sample drawn from that population. We can't know what the true relationship is just from one study. This is why it's important to dig into the literature and see what others before you found. That said, the p-value in your regression model output is a formal test of the association between $V3$ and $y$. You can do a post-estimation Wald test, but the p-value will be roughly equal to the p-value in the model $\endgroup$ Apr 13, 2015 at 3:13

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.