10
$\begingroup$

When I analyze my variables in two separate (univariate) logistic regression models, I get the following:

Predictor 1:    B= 1.049,    SE=.352,    Exp(B)=2.85,    95% CI=(1.43, 5.69),    p=.003
   Constant:    B=-0.434,    SE=.217,    Exp(B)=0.65,                            p=.046

Predictor 2:    B= 1.379,    SE=.386,    Exp(B)=3.97,    95% CI=(1.86, 8.47),    p<.001
   Constant:    B=-0.447,    SE=.205,    Exp(B)=0.64,                            p=.029

but when I enter them into a single multiple logistic regression model, I get:

Predictor 1:    B= 0.556,    SE=.406,    Exp(B)=1.74,    95% CI=(0.79, 3.86),    p=.171
Predictor 2:    B= 1.094,    SE=.436,    Exp(B)=2.99,    95% CI=(1.27, 7.02),    p=.012
   Constant:    B=-0.574,    SE=.227,    Exp(B)=0.56,                            p=.012

Both predictors are dichotomous (categorical). I have checked for multicollinearity.

I am not sure if I have given enough info, but I cannot understand why predictor 1 has gone from being significant to non-significant and why the odds ratios are so different in the multiple regression model. Can anyone provide a basic explanation of what is going on?

$\endgroup$
  • 2
    $\begingroup$ multivariate usually indicates multiple dependent variables - you meant multiple predictors, right? That would usually be referred to as multiple regression. $\endgroup$ – Macro Apr 27 '12 at 16:24
  • 1
    $\begingroup$ Also, $\beta$'s from different logistic regression models are usually not comparable. This is because the scale has changed - this is a subtle issue but the basic idea is that the total variance (on the latent scale that logistic regression naturally arises - see en.wikipedia.org/wiki/…) is not fixed across models, so you shouldn't expect the coefficients to be the same across models, although that wouldn't necessarily explain the change in statistical significance. How did you check for dependence between the two predictors? $\endgroup$ – Macro Apr 27 '12 at 16:34
  • $\begingroup$ ah, ok thank you. I checked collinearity diagnostics through linear regression on spss & checked the tolerance and VIF - is this correct? $\endgroup$ – Annie Apr 27 '12 at 16:45
  • $\begingroup$ Nice comment @Macro . I vaguely recall reading about ways to fix this issue about the scale, but I don't remember where. $\endgroup$ – Peter Flom Apr 27 '12 at 16:59
  • 1
    $\begingroup$ @PeterFlom, one thing you can do is scale the coefficients by the variance of the linear predictors (plus $\pi^{2}/3$, the variance of the standard logistic distribution) - this puts them on the same scale. Of course, they are no longer interpretable as odds ratios once you do this. $\endgroup$ – Macro Apr 27 '12 at 17:50
20
$\begingroup$

There are several reasons (none of which are specifically related to logistic regression, but may occur in any regression).

  1. Loss of degrees of freedom: when trying to estimate more parameters from a given dataset, you're effectively asking more of it, which costs precision, hence leads to lower t-statistics, hence higher p-values.
  2. Correlation of Regressors: Your regressors may be related to each other, effectively measuring something similar. Say, your logit model is to explain labor market status (working/not working) as a function of experience and age. Individually, both variables are positively related to the status, as more experienced/older (ruling out very old employees for the sake of the argument) employees find it easier to find jobs than recent graduates. Now, obviously, the two variables are strongly related, as you need to be older to have more experience. Hence, the two variables basically "compete" for explaining the status, which may, especially in small samples, result in both variables "losing", as none of the effects may be strong enough and sufficiently precisely estimated when controlling for the other to get significant estimates. Essentially, you are asking: what is the positive effect of another year of experience when holding age constant? There may be very few to no employees in your dataset to answer that question, so the effect will be imprecisely estimated, leading to large p-values.

  3. Misspecified models: The underlying theory for t-statistics/p-values requires that you estimate a correctly specified model. Now, if you only regress on one predictor, chances are quite high that that univariate model suffers from omitted variable bias. Hence, all bets are off as to how p-values behave. Basically, you must be careful to trust them when your model is not correct.

$\endgroup$
  • $\begingroup$ Thanks for your thorough and quick response. I will try to eliminate any multicollinearity first. I have run correlations between variables and found some, and will try running variance inflation factors as I have heard that is a good way to check for this as well. If it does turn out to just be a degrees of freedom issue, is there anything I can do about that? I can explain that this is happening, but it seems to compromise the integrity of the regression if the significance drops so severely. $\endgroup$ – Sam O'Brien Mar 18 '15 at 8:37
  • 3
    $\begingroup$ @SamO'Brien: Note that if your goal is really what you said it was - "to try to determine "which independent variables potentially cause a response" - , ignoring some just because they're correlated with others to "eliminate any multicollinearity" isn't going to help achieve it. $\endgroup$ – Scortchi Mar 18 '15 at 12:54
  • 1
    $\begingroup$ Is it possible to have it the other way around i.e. same predictor non-significant in simple regression but significant in multiple regression? $\endgroup$ – gkcn Jan 26 '17 at 15:34
8
$\begingroup$

There is no particular reason why this should not happen. Multiple regression asks a different question from simple regression. In particular, multiple regression (in this case, multiple logistic regression) asks about the relationship between the dependent variables and the independent variables, controlling for the other independent variables. Simple regression asks about the relationship between a dependent variable and a (single) independent variable.

If you add the context of your study (e.g., what are these variables?) it may be possible to give more specific responses. Also, given that all three variables in your case are dichotomies, you could present us with the data pretty easily... there are only 8 lines needed to summarize this:

\begin{array}{llll} DV &IV1 &IV2 &{\rm Count} \\ A &A &A &10 \\ A &A &B &20 \end{array}

etc.

$\endgroup$

Your Answer

By clicking "Post Your Answer", you acknowledge that you have read our updated terms of service, privacy policy and cookie policy, and that your continued use of the website is subject to these policies.

Not the answer you're looking for? Browse other questions tagged or ask your own question.