Logistic Regression with Multiple Independent Variables vs One Independent Variable

Question

Will the estimates and odds ratios change for an independent variable if it is by itself vs if there are other independent variables? I would think that it would change since thinking of it as an equation $y=x$ (one variable) is different than $y=x+z+q$ (3 variables).

Answer 1

The estimate of the effect of $x$ will certainly change if $z$ or $q$ (or both) have an effect on $y$ net of $x$. It will change even if $z$ and $q$ are orthogonal to $x$ as long as $z$ and $q$ explain any portion of $y$. This happens because adding new variables changes the scale in which the entire model is expressed. The logistic model given by:

(1)$$ \ln\bigg(\frac{1}{1-p_i}\bigg) = \beta_{0} + \beta_{1}x_{1i} $$

Is expressed in the latent variable formulation as:

(2)$$ y^* =\alpha_{0} + \alpha_{1}x_{1i} + \sigma \varepsilon $$

The total variance in the model is made of explained (modelled) and unexplained (residual/errors) variance. The logistic model forces the errors to have a fixed variance of 3.29 (and a logistical distribution). Hence, any changes in the amount of explained variance will force the total variance of $y*$ to change, causing its scale to change because the variance of the errors is fixed and cannot change. This affects the size of the coefficients because they now explain change in $y*$ in a different scale. The scaling factor is given by $\sigma$ in (2). The alphas in (2) are related to the betas in (1) by:

(3)$$ \beta_{j} = \frac{\alpha_{j}}{\sigma}\;\;j=1,...,J. $$

Adding more covariates that explain any portion of $y$ will reduce the amount of unexplained heterogeneity and will consequentially change the other coefficients in the model. You can refer to my answer here or to the following literature:

References

• Allison, P. D. (1999). Comparing Logit and Probit Coefficients Across Groups. Sociological Methods & Research, 28(2), 186–208.
• Mood, C. (2010). Logistic Regression: Why We Cannot Do What We Think We Can Do, and What We Can Do About It. European Sociological Review, 26(1), 67–82.