# Correct interpretation of confidence interval for logistic regression?

In trying to understand logistic regression, I find it easiest to transform the coefficients into predicted probabilities. So, for a particular predictor value (x):

precicted probability = 1 / (1 + exp(-(intercept + slope * x)))


Using the 'predict' function, I'll exemplify as:

# Generate data
set.seed(42)
n <- 90
x <- sort(sample(seq(1, 90, by = 1), n, replace = T))
y <- rbinom(n, c(1, 0), c(seq(0, 1, length.out = n), seq(1, 0, length.out = n)))

############ Apply logistic regression ##########
m1 <- glm(y ~ x, family = 'binomial')

############ Interpreting logistic regression ##########
new <- data.frame(x = seq(1, 90, by = 1))
pred <- predict.glm(m1, newdata = new, type = "response")
plot(x, m1$$fitted.values, ylim = c(0, 1)) lines(new$$x, pred)


My question, then, is if I can interpret the confidence intervals around the coefficients for logistic regression in terms of predicted probability too. That is, like:

############ Interpreting confidence intervals ##########
confs <- confint(m1)
ll <- 1 / (1 + exp(-(confs[1, 1] + confs[2, 2] * new$$x))) ul <- 1 / (1 + exp(-(confs[1, 2] + confs[2, 1] * new$$x)))
lines(new$$x, ll, col = "red") lines(new$$x, ul, col = "green")


I believe the two colored lines (are not prediction intervals but) illustrate the limits of the relationship between the predictor and outcome variable we can be 95% confident in based on this data. Is this so?

• Comment about the graph: I know why you made both lines red, but it looks like they cross around 58. Perhaps consider using a different color for the upper and lower lines.
– Dave
Commented Nov 4, 2020 at 13:02
• I've edited the color for clarification. They lines do cross. The lines are supposed to represent the uncertainty that the relationship could be the red one, the green one, or anything in between. In linear regression, you can imagine that the uncertainty is that you can grab the regression line and "twist it". See for example rpubs.com/aaronsc32/regression-confidence-prediction-intervals for an illustration. Here, the lines are not actually a line, but a sigmoid, but (I believe!) the interpretation is the same. Commented Nov 4, 2020 at 13:14
• Another graphical comment: red/green colorblindness is not that common but common enough. I often use lime green (like yours) and a fairly dark blue, and people seem to be happy with graphs that I produce.
– Dave
Commented Nov 4, 2020 at 13:26

The problem is that you cannot use the confidence intervals for the coefficients in that way, for various reasons, including that it ignores dependence among the estimates. The fact that the lines cross, indicating a 95% confidence interval on a single value, is a clue to the mistake.

Instead, (i) find the logits and their standard errors (this involves finding the asymptotic variance of a linear combination of the coefficient estimates), (ii) find the 95% intervals for the true logits, and (iii) back-transform to get to the the probability scale, like this:

pred1 <- predict.glm(m1, newdata = new, type = "link", se.fit=TRUE)
logit =  pred1$$fit fit.prob = exp(logit)/(1+exp(logit)) upper.logit = logit + 1.96*pred1$$se.fit
lower.logit = logit - 1.96*pred1\$se.fit
upper.prob = exp(upper.logit)/(1+exp(upper.logit))
lower.prob = exp(lower.logit)/(1+exp(lower.logit))

lines(new$$x, lower.prob, col = "red") lines(new$$x, upper.prob , col = "green")


Now the picture makes more sense:

• plogis(x) is a little more compact (and numerically stable) than exp(x)/(1+exp(x)) Commented Nov 4, 2020 at 19:56
• Thank you for this correction and clarification! Commented Nov 5, 2020 at 8:46