# Interpreting intercept in logistic regression when there is more than one categorical variable

mydata <- read.csv("https://stats.idre.ucla.edu/stat/data/binary.csv")
#create a binary variable just for the purpose of experimentation
mydata$bin = rbinom(nrow(mydata), p = 0.7, size = 1) mydata$rank <- factor(mydata$rank) mydata$bin <- factor(mydata$bin) #Run the model with just bin my.mod <- glm(admit ~ bin, data = mydata, family = "binomial") summary(my.mod)  Coefficients: Estimate Std. Error z value Pr(>|z|) (Intercept) -0.8434294 0.1966085 -4.28989 0.000017876 bin1 0.1121304 0.2347628 0.47763 0.63291 The intercept is simply logodds of admit == 1 when bin1 = 0 Check: x = subset(mydata, bin == "0") mean(x$admit)
log(0.3008130081/(1-0.3008130081))


Result:
-0.843429383 Tallied

Next, do the same with rank variable alone in the model

my.mod <- glm(admit ~ rank, data = mydata, family = "binomial")
summary(my.mod)


Coefficients: Estimate Std. Error z value Pr(>|z|)
(Intercept) 0.1643031 0.2569384 0.63946 0.522521
rank2 -0.7500300 0.3079693 -2.43540 0.014875
rank3 -1.3646980 0.3353867 -4.06903 0.000047210
rank4 -1.6867296 0.4093073 -4.12094 0.000037733

The intercept is simply logodds of admit == 1 when rank2 = 0 & rank3 = 0 & rank4 = 0; in other words rank == 1

Check:

x = subset(mydata, rank == "1" )
mean(x$admit) log(0.5409836066/(1-0.5409836066))  Result: 0.1643030515 Tallied Next, I add both the categorical variables in the model my.mod <- glm(admit ~ rank + bin, data = mydata, family = "binomial") summary(my.mod)  Coefficients: Estimate Std. Error z value Pr(>|z|) (Intercept) 0.23245129 0.32715773 0.71052 0.477383 rank2 -0.75673758 0.30869909 -2.45138 0.014231 rank3 -1.38523274 0.34119770 -4.05991 0.000049091 rank4 -1.69956417 0.41127752 -4.13240 0.000035899 bin1 -0.08308446 0.24667503 -0.33682 0.736255 Question: How to make sense of this intercept like above 2 examples? I don't think in this case intercept equals logodds of admit == 1 when (bin = 0 and rank = 1). Check:  x = subset(mydata, rank == "1" & bin == "0" ) mean(x$admit)
log(0.6363636364/(1-0.6363636364))


Result: 0.5596157881 Not tallied!

## 1 Answer

So, you have two binary predictors, say $$X$$ and $$Z$$, and the logistic model $$\DeclareMathOperator{\P}{\mathbb{P}} \P(Y=1 \mid X=x) = \frac{e^{\beta_0 + \beta_1 x}}{1+e^{\beta_0 + \beta_1 x}}$$ setting $$X=0$$ you can see that $$\beta_0$$ is the log odds of probability that $$Y=1$$ given $$X=0$$. Now adding the second predictor in the model: $$\P(Y=1 \mid X=x,Z=z) = \frac{e^{\beta_0 + \beta_1 x+\beta_2 z}}{1+e^{\beta_0 + \beta_1 x + \beta_2 z}}$$ and you can see that in this model $$\beta_0$$ is the log odds of the probability that $$Y=1$$ given both $$X=0$$ and $$Z=0$$ (in your code example you look at the conditioning $$X=0, Z=1$$).

• dummy variables for the categorical variable with rank = 1 as the baseline are all zero. bin1 is also zero for binary predictor for bin = 0. This will wipe out all the terms in the linear predictor except for the intercept - β0. Ex - > model.matrix(my.mod)[13,] (Intercept) bin1 rank2 rank3 rank4 1 0 0 0 0 ..I still cannot see the answer, some elaboration would surely help. Sep 9 '17 at 8:01
• > mydata[13,] ADMIT GRE GPA RANK bin rank 13 1 760 4 1 0 1 Sep 9 '17 at 8:11