# Logistic Regression with Multiple Independent Variables vs One Independent Variable

Will the estimates and odds ratios change for an independent variable if it is by itself vs if there are other independent variables? I would think that it would change since thinking of it as an equation $y=x$ (one variable) is different than $y=x+z+q$ (3 variables).

• Yes, it would. You can verify yourself with any example. – Gregor May 8 '17 at 22:26
• It'd most likely be different due to other variables. – SmallChess May 9 '17 at 2:08
• Yes, it will change because more of the variation in the dependent variable y is now being attributed to x. – Michael Grogan May 9 '17 at 2:46

The estimate of the effect of $x$ will certainly change if $z$ or $q$ (or both) have an effect on $y$ net of $x$. It will change even if $z$ and $q$ are orthogonal to $x$ as long as $z$ and $q$ explain any portion of $y$. This happens because adding new variables changes the scale in which the entire model is expressed. The logistic model given by:

(1)$$\ln\bigg(\frac{1}{1-p_i}\bigg) = \beta_{0} + \beta_{1}x_{1i}$$

Is expressed in the latent variable formulation as:

(2)$$y^* =\alpha_{0} + \alpha_{1}x_{1i} + \sigma \varepsilon$$

The total variance in the model is made of explained (modelled) and unexplained (residual/errors) variance. The logistic model forces the errors to have a fixed variance of 3.29 (and a logistical distribution). Hence, any changes in the amount of explained variance will force the total variance of $y*$ to change, causing its scale to change because the variance of the errors is fixed and cannot change. This affects the size of the coefficients because they now explain change in $y*$ in a different scale. The scaling factor is given by $\sigma$ in (2). The alphas in (2) are related to the betas in (1) by:

(3)$$\beta_{j} = \frac{\alpha_{j}}{\sigma}\;\;j=1,...,J.$$

Adding more covariates that explain any portion of $y$ will reduce the amount of unexplained heterogeneity and will consequentially change the other coefficients in the model. You can refer to my answer here or to the following literature:

References

• Thanks for your information! Does that mean the more independent variables you include the better your model will be, thus, the better your output would be even if it changes some of the odds ratios a bit? – Mike May 25 '17 at 19:12
• Not necessarily. Usually it is better to have more of the relevant variables, but there are cases in which controlling for a variable will lead to biased estimates of other coefficients. One example of these is what is called collider effects in some fields, and is probably a matter for a different question here in this site (if there is not any already). – Kenji May 26 '17 at 21:11