# Logistic regression weights of uncorrelated predictors

I am a bit puzzled about the behavior of uncorrelated predictors in logistic regression. As in OLS, I thought that if two predictors (rv1 and rv2) are uncorrelated, then the regression weights of rv1 will not change from a regression that only includes rv1 to one that includes rv1 and rv2. However, it seems to be the case that this is not true in logistic regression and coefficients change between the two regression models, even if the predictors are uncorrelated.

I have pasted some R syntax below that demonstrates this behavior.

Why is this the case and how do the regression weights from the two regressions (the one with only rv1 and the other one with rv1 and rv2) relate to each other? Is there a way to know what the regression weight of rv1 will be if one knows the regression weight of rv1 in the regression that includes both predictors?

Thanks! P.S. This post is crossposted at another unrelated stat answer site.

library(MASS)

#generate lots of data (a little bit weird data handling, I know)
n <- 10000
rdta <- as.data.frame(mvrnorm(n=n,c(0,0),matrix(c(1,0,0,1),2,2),empirical=TRUE))
names(rdta) <- c("rv1","rv2")

#confirm that preds are uncorrelated
cov(rdta$rv1,rdta$rv2)

rv1 <- rdta$rv1 rv2 <- rdta$rv2

rv1ry <- 1
rv2ry <- 1

#generate binary data from known regression coefficients
ylinp <- (1 / (1+exp(-(-1 + rv1*rv1ry + rv2*rv2ry))))
y <- rbinom(n,1,ylinp)

#confirm that OLS regression works as expected (regression weights do not change)
rv1y <- .222
rv2y <- .333
y <- rv1y * rv1 + rv2y * rv2 + rnorm(n,0,.5)
lm(y~rv1+rv2)
lm(y~rv1)
lm(y~rv2)


I am not sure if it is expected to also paste relevant output here, but here goes: OLS results

lm(y~rv1+rv2)

Call:
lm(formula = y ~ rv1 + rv2)

Coefficients:
(Intercept)          rv1          rv2
0.001096     0.220051     0.333072


lm(y~rv1)

Call:
lm(formula = y ~ rv1)

Coefficients:
(Intercept)          rv1
0.001096     0.220051


lm(y~rv2)

Call:
lm(formula = y ~ rv2)

Coefficients:
(Intercept)          rv2
0.001096     0.333072


Logistic regression results

Call:  glm(formula = ry ~ rv1 + rv2, family = binomial(link = "logit"))

Coefficients:
(Intercept)          rv1          rv2
-1.001        1.916        2.469


Call:  glm(formula = ry ~ rv1, family = binomial(link = "logit"))

Coefficients:
(Intercept)          rv1
-0.5495       1.0535


Call:  glm(formula = ry ~ rv2, family = binomial(link = "logit"))

Coefficients:
(Intercept)          rv2
-0.6538       1.6140


I don't think this is peculiar and it is not restricted to logistic regression. It can happen with linear regression as well. The least squares estimates of the regression coefficients will depend on which variable you include to fit in the model. This will happen even if covariates $X_1$ and $X_2$ are independent or even just uncorrelated.