Masked variables in logistic regression

Question

I have a logistic regression model with variables $V_1$ through $V_n$. When I build a full model with all of the variables, I find that $V_1$ and $V_2$ are significant. However, when I build a new model in which I remove $V_1$, I find that, in addition to $V_2$, $V_3$ is now significant. I assume this occurs because $V_1$ and $V_3$ are correlated, so that all the information $V_3$ is captured in $V_1$ in the full model. $V_3$ is of particular interest to me given my hypothesis, so I’d like to know what it’s reasonable to say about $V_3$ in terms of significance. Can I still report that $V_3$ is significant even though it’s masked by $V_1$? Are there further tests I can do that would be informative here?

Answer 1

It is important to have theoretical justification for including variables in a model to avoid "fishing" for significance. The first thing you should do is try to understand the underlying relationship between $V_1$, $V_3$, and $y$. Are $V_1$ and $V_3$ similar measures of the same thing, such that including both in the model is redundant, or does $V_1$ confound the relationship between $V_3$ and $y$?

If the former, you could (reasonably) justify dropping one of the covariates from your model, because there is no additional information gained by including both of them in your model as they are measures of the same thing. If the latter, you absolutely should NOT drop $V_1$, as failure to control for $V_1$ would lead to omitted variable bias, resulting in inconsistent parameter estimate for your main covariate of interest, $V_3$. Why would failing to control for $V_1$ bias your results in the second scenario? Suppose the true relationships are as follows:

• $Corr(V_1,y)≠0$ (there is a relationship between $V_1$ and $y$)
• $Corr(V_1,V_3)≠0$ (there is a relationship between $V_1$ and $V_3$)
• $Corr(V_3,y)=0$ (there is NO relationship between $V_3$ and $y$)

If $V_1$ and $V_3$ are correlated and you do not control for $V_1$, the initial observed relationship between $V_3$ and $y$ actually reflects the (true) relationship between $V_1$ and $y$. The parameter estimate on $V_3$ is, in effect, a combination of the effect of $V_3$ on $y$ AND the partial effect of $V_1$ and $y$. Because in reality there is no relationship between $V_3$ and $y$, once you control for the factor that is "responsible" for the observed relationship (i.e., $V_1$), the coefficient on $V_3$ is no longer significant.

I would report that $V_3$ was significantly associated with $y$ in bivariate analysis, but that the relationship becomes non-significant after controlling for $V_1$ and discuss why you think this is. You may find it helpful to dig into the literature for insight on the relationship between $V_1$, $V_3$, and $y$.