# Different ways to produce a confidence interval for odds ratio from logistic regression

I am studying how to construct a 95% confidence interval for odds ratio from the coefficients obtained in the logistic regression. So, considering the logistic regression model,


such that $x = 0$ for control group and $x = 1$ for case group.

I have already read that the simplest way is to construct a 95% CI for $\beta$ then we applied the exponential function, that is,

$$\hat{\beta} \pm 1.96\times \se(\hat{\beta}) \rightarrow \exp\{\hat{\beta} \pm 1.96\times \se(\hat{\beta})\}$$

My questions are:

1. What is the theoretical reason that justifies this procedure? I know $\mbox{odds ratio} = \exp\{\beta\}$ and maximum likelihood estimators are invariant. However, I do not know the connection among these elements.

2. Should the delta method produce the same 95% confidence interval as the previous procedure? Using the delta method,

$$\exp\{\hat{\beta}\} \dot{\sim} N(\beta,\ \exp\{\beta\}^2 \var(\hat{\beta}))$$

Then,

$$\exp\{\hat{\beta}\} \pm 1.96\times \sqrt{\exp\{\beta\}^2 \var(\hat{\beta})}$$

If not, which is the best procedure?

• I like bootstrap for CI as well, if I have parameter values or training data of sufficient size. Apr 10 '16 at 1:32
• There is a better way of doing this, see stats.stackexchange.com/questions/5304/… for details May 10 '16 at 13:28

1. The justification for the procedure is the asymptotic normality of the MLE for $\beta$ and results from arguments involving the Central Limit Theorem.

2. The Delta method comes from a linear (i.e first order Taylor) expansion of the function around the MLE. Subsequently we appeal to the asymptotic normality and unbiasedness of the MLE.

Asymptotically both give the same answer. But practically, you would favor the one which looks more closely normal. In this example, I would favor the first one because the latter is likely to be less symmetric.

# A comparison of confidence intervals methods on an example from ISL

The book "Introduction to Statistical Learning" by Tibshirani, James, Hastie provides an example on page 267 of confidence intervals for polynomial logistic regression degree 4 on the wage data. Quoting the book:

## A open ended conclusion

A look at the Normal QQ plots for both the probabilities and the negative log odds show that neither are normally distributed. Could this explain the difference ?

Source:

For most purposes the simplest way is probably best, as discussed in the context of a log transform on this page. Think about your dependent variable as being analyzed in the logit scale, with statistical tests performed and confidence intervals (CI) defined on that logit scale. The back transformation to odds ratio is simply to put those results into a scale that a reader might more readily grasp. This is also done, for example, in Cox survival analysis, where the regression coefficients (and the 95% CI) are exponentiated to obtain hazard ratios and their CI.