12
$\begingroup$

When I analyze my variables in two separate (univariate) logistic regression models, I get the following:

Predictor 1:    B= 1.049,    SE=.352,    Exp(B)=2.85,    95% CI=(1.43, 5.69),    p=.003
   Constant:    B=-0.434,    SE=.217,    Exp(B)=0.65,                            p=.046

Predictor 2:    B= 1.379,    SE=.386,    Exp(B)=3.97,    95% CI=(1.86, 8.47),    p<.001
   Constant:    B=-0.447,    SE=.205,    Exp(B)=0.64,                            p=.029

but when I enter them into a single multiple logistic regression model, I get:

Predictor 1:    B= 0.556,    SE=.406,    Exp(B)=1.74,    95% CI=(0.79, 3.86),    p=.171
Predictor 2:    B= 1.094,    SE=.436,    Exp(B)=2.99,    95% CI=(1.27, 7.02),    p=.012
   Constant:    B=-0.574,    SE=.227,    Exp(B)=0.56,                            p=.012

Both predictors are dichotomous (categorical). I have checked for multicollinearity.

I am not sure if I have given enough info, but I cannot understand why predictor 1 has gone from being significant to non-significant and why the odds ratios are so different in the multiple regression model. Can anyone provide a basic explanation of what is going on?

Improve this question
$\endgroup$
5
  • 3
    $\begingroup$ multivariate usually indicates multiple dependent variables - you meant multiple predictors, right? That would usually be referred to as multiple regression. $\endgroup$
    – Macro
    Apr 27, 2012 at 16:24
  • 1
    $\begingroup$ Also, $\beta$'s from different logistic regression models are usually not comparable. This is because the scale has changed - this is a subtle issue but the basic idea is that the total variance (on the latent scale that logistic regression naturally arises - see en.wikipedia.org/wiki/…) is not fixed across models, so you shouldn't expect the coefficients to be the same across models, although that wouldn't necessarily explain the change in statistical significance. How did you check for dependence between the two predictors? $\endgroup$
    – Macro
    Apr 27, 2012 at 16:34
  • $\begingroup$ ah, ok thank you. I checked collinearity diagnostics through linear regression on spss & checked the tolerance and VIF - is this correct? $\endgroup$
    – Annie
    Apr 27, 2012 at 16:45
  • $\begingroup$ Nice comment @Macro . I vaguely recall reading about ways to fix this issue about the scale, but I don't remember where. $\endgroup$
    – Peter Flom
    Apr 27, 2012 at 16:59
  • 1
    $\begingroup$ @PeterFlom, one thing you can do is scale the coefficients by the variance of the linear predictors (plus $\pi^{2}/3$, the variance of the standard logistic distribution) - this puts them on the same scale. Of course, they are no longer interpretable as odds ratios once you do this. $\endgroup$
    – Macro
    Apr 27, 2012 at 17:50

2 Answers 2

Reset to default
22
$\begingroup$

There are several reasons (none of which are specifically related to logistic regression, but may occur in any regression).

  1. Loss of degrees of freedom: when trying to estimate more parameters from a given dataset, you're effectively asking more of it, which costs precision, hence leads to lower t-statistics, hence higher p-values.

  2. Correlation of Regressors: Your regressors may be related to each other, effectively measuring something similar. Say, your logit model is to explain labor market status (working/not working) as a function of experience and age. Individually, both variables are positively related to the status, as more experienced/older (ruling out very old employees for the sake of the argument) employees find it easier to find jobs than recent graduates. Now, obviously, the two variables are strongly related, as you need to be older to have more experience. Hence, the two variables basically "compete" for explaining the status, which may, especially in small samples, result in both variables "losing", as none of the effects may be strong enough and sufficiently precisely estimated when controlling for the other to get significant estimates. Essentially, you are asking: what is the positive effect of another year of experience when holding age constant? There may be very few to no employees in your dataset to answer that question, so the effect will be imprecisely estimated, leading to large p-values.

  3. Misspecified models: The underlying theory for t-statistics/p-values requires that you estimate a correctly specified model. Now, if you only regress on one predictor, chances are quite high that that univariate model suffers from omitted variable bias. Hence, all bets are off as to how p-values behave. Basically, you must be careful to trust them when your model is not correct.

Improve this answer
$\endgroup$
6
  • $\begingroup$ Thanks for your thorough and quick response. I will try to eliminate any multicollinearity first. I have run correlations between variables and found some, and will try running variance inflation factors as I have heard that is a good way to check for this as well. If it does turn out to just be a degrees of freedom issue, is there anything I can do about that? I can explain that this is happening, but it seems to compromise the integrity of the regression if the significance drops so severely. $\endgroup$
    – Sam O'Brien
    Mar 18, 2015 at 8:37
  • 3
    $\begingroup$ @SamO'Brien: Note that if your goal is really what you said it was - "to try to determine "which independent variables potentially cause a response" - , ignoring some just because they're correlated with others to "eliminate any multicollinearity" isn't going to help achieve it. $\endgroup$
    – Scortchi - Reinstate Monica ♦
    Mar 18, 2015 at 12:54
  • 1
    $\begingroup$ Is it possible to have it the other way around i.e. same predictor non-significant in simple regression but significant in multiple regression? $\endgroup$
    – gkcn
    Jan 26, 2017 at 15:34
  • $\begingroup$ Assuming thet there is no model misspecification and 1 or 2 is happenong (or both), is there any restriction to fit more than one model (same response variable)? let's say one model with one set of predictors and another model with the rest of predictors. Is there a multiple comparison issue? Thanks! $\endgroup$
    – Rafael
    Aug 12, 2020 at 10:14
  • $\begingroup$ Yes, I would say that multiple testing then becomes an issue. Of course, depending on the causal mechanism you are interested in, it is of course not clear how many different models you can have that all satisfy the requirement that none of them is misspecified. $\endgroup$
    – Christoph Hanck
    Aug 12, 2020 at 10:37
8
$\begingroup$

There is no particular reason why this should not happen. Multiple regression asks a different question from simple regression. In particular, multiple regression (in this case, multiple logistic regression) asks about the relationship between the dependent variables and the independent variables, controlling for the other independent variables. Simple regression asks about the relationship between a dependent variable and a (single) independent variable.

If you add the context of your study (e.g., what are these variables?) it may be possible to give more specific responses. Also, given that all three variables in your case are dichotomies, you could present us with the data pretty easily... there are only 8 lines needed to summarize this:

\begin{array}{llll} DV &IV1 &IV2 &{\rm Count} \\ A &A &A &10 \\ A &A &B &20 \end{array}

etc.

Improve this answer
$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.