Does it make sense to use Logistic regression with binary outcome and predictor?

Question

I have a binary outcome variable {0,1} and a predictor variable {0,1}. My thoughts are that it doesn't make sense to do logistic unless I include other variables and calculate the odds ratio.

With one binary predictor, wouldn't calculation of probability suffice vs odds ratio?

Answer 1

In this case you can collapse your data to $$ \begin{array}{c|cc} X \backslash Y & 0 & 1 \\ \hline 0 & S_{00} & S_{01} \\ 1 & S_{10} & S_{11} \end{array} $$ where $S_{ij}$ is the number of instances for $x = i$ and $y =j$ with $i,j \in \{0,1\}$. Suppose there are $n$ observations overall.

If we fit the model $p_i = g^{-1}(x_i^T \beta) = g^{-1}(\beta_0 + \beta_1 1_{x_i = 1})$ (where $g$ is our link function) we'll find that $\hat \beta_0$ is the logit of the proportion of successes when $x_i = 0$ and $\hat \beta_0 + \hat \beta_1$ is the logit of the proportion of successes when $x_i = 1$. In other words, $$ \hat \beta_0 = g\left(\frac{S_{01}}{S_{00} + S_{01}}\right) $$ and $$ \hat \beta_0 + \hat \beta_1 = g\left(\frac{S_{11}}{S_{10} + S_{11}}\right). $$

Let's check this is R.

n <- 54
set.seed(123)
x <- rbinom(n, 1, .4)
y <- rbinom(n, 1, .6)

tbl <- table(x=x,y=y)

mod <- glm(y ~ x, family=binomial())

# all the same at 0.5757576
binomial()$linkinv( mod$coef[1])
mean(y[x == 0])
tbl[1,2] / sum(tbl[1,])

# all the same at 0.5714286
binomial()$linkinv( mod$coef[1] + mod$coef[2])
mean(y[x == 1])
tbl[2,2] / sum(tbl[2,])

So the logistic regression coefficients are exactly transformations of proportions coming from the table.

The upshot is that we certainly can analyze this dataset with a logistic regression if we have data coming from a series of Bernoulli random variables, but it turns out to be no different than directly analyzing the resulting contingency table.

---

I want to comment on why this works from a theoretical perspective. When we're fitting a logistic regression, we are using the model that $Y_i | x_i \stackrel{\perp}{\sim} \text{Bern}(p_i)$. We then decide to model the mean as a transformation of a linear predictor in $x_i$, or in symbols $p_i = g^{-1}\left( \beta_0 + \beta_1 x_i\right)$. In our case we only have two unique values of $x_i$, and therefore there are only two unique values of $p_i$, say $p_0$ and $p_1$. Because of our independence assumption we have $$ \sum \limits_{i : x_i = 0} Y_i = S_{01} \sim \text{Bin} \left(n_0, p_0\right) $$ and $$ \sum \limits_{i : x_i = 1} Y_i = S_{11} \sim \text{Bin} \left(n_1, p_1\right). $$ Note how we're using the fact that the $x_i$, and in turn $n_0$ and $n_1$, are nonrandom: if this was not the case then these would not necessarily be binomial.

This means that $$ S_{01} / n_0 = \frac{S_{01}}{S_{00} + S_{01}} \to_p p_0 \hspace{2mm} \text{ and } \hspace{2mm} S_{11} / n_1 = \frac{S_{11}}{S_{10} + S_{11}} \to_p p_1. $$

The key insight here: our Bernoulli RVs are $Y_i | x_i = j \sim \text{Bern}(p_j)$ while our binomial RVs are $S_{j1} \sim \text{Bin}(n_j, p_j)$, but both have the same probability of success. That's the reason why these contingency table proportions are estimating the same thing as an observation-level logistic regression. It's not just some coincidence with the table: it's a direct consequence of the distributional assumptions we have made.

Answer 2

When you have more than one predictors and all the predictors are binary variables, you could fit a model using Logic Regression [1] (note it's "Logic" not "Logistic"). It's useful when you believe interaction effects among your predictors are prominent. There's an implementation in R (LogicReg package).

[1] Ruczinski, I., Kooperberg, C., & LeBlanc, M. (2003). Logic regression. Journal of Computational and graphical Statistics, 12(3), 475-511.