18
$\begingroup$

I have a binary outcome variable {0,1} and a predictor variable {0,1}. My thoughts are that it doesn't make sense to do logistic unless I include other variables and calculate the odds ratio.

With one binary predictor, wouldn't calculation of probability suffice vs odds ratio?

$\endgroup$
26
$\begingroup$

In this case you can collapse your data to $$ \begin{array}{c|cc} X \backslash Y & 0 & 1 \\ \hline 0 & S_{00} & S_{01} \\ 1 & S_{10} & S_{11} \end{array} $$ where $S_{ij}$ is the number of instances for $x = i$ and $y =j$ with $i,j \in \{0,1\}$. Suppose there are $n$ observations overall.

If we fit the model $p_i = g^{-1}(x_i^T \beta) = g^{-1}(\beta_0 + \beta_1 1_{x_i = 1})$ (where $g$ is our link function) we'll find that $\hat \beta_0$ is the logit of the proportion of successes when $x_i = 0$ and $\hat \beta_0 + \hat \beta_1$ is the logit of the proportion of successes when $x_i = 1$. In other words, $$ \hat \beta_0 = g\left(\frac{S_{01}}{S_{00} + S_{01}}\right) $$ and $$ \hat \beta_0 + \hat \beta_1 = g\left(\frac{S_{11}}{S_{10} + S_{11}}\right). $$

Let's check this is R.

n <- 54
set.seed(123)
x <- rbinom(n, 1, .4)
y <- rbinom(n, 1, .6)

tbl <- table(x=x,y=y)

mod <- glm(y ~ x, family=binomial())

# all the same at 0.5757576
binomial()$linkinv( mod$coef[1])
mean(y[x == 0])
tbl[1,2] / sum(tbl[1,])

# all the same at 0.5714286
binomial()$linkinv( mod$coef[1] + mod$coef[2])
mean(y[x == 1])
tbl[2,2] / sum(tbl[2,])

So the logistic regression coefficients are exactly transformations of proportions coming from the table.

The upshot is that we certainly can analyze this dataset with a logistic regression if we have data coming from a series of Bernoulli random variables, but it turns out to be no different than directly analyzing the resulting contingency table.


I want to comment on why this works from a theoretical perspective. When we're fitting a logistic regression, we are using the model that $Y_i | x_i \stackrel{\perp}{\sim} \text{Bern}(p_i)$. We then decide to model the mean as a transformation of a linear predictor in $x_i$, or in symbols $p_i = g^{-1}\left( \beta_0 + \beta_1 x_i\right)$. In our case we only have two unique values of $x_i$, and therefore there are only two unique values of $p_i$, say $p_0$ and $p_1$. Because of our independence assumption we have $$ \sum \limits_{i : x_i = 0} Y_i = S_{01} \sim \text{Bin} \left(n_0, p_0\right) $$ and $$ \sum \limits_{i : x_i = 1} Y_i = S_{11} \sim \text{Bin} \left(n_1, p_1\right). $$ Note how we're using the fact that the $x_i$, and in turn $n_0$ and $n_1$, are nonrandom: if this was not the case then these would not necessarily be binomial.

This means that $$ S_{01} / n_0 = \frac{S_{01}}{S_{00} + S_{01}} \to_p p_0 \hspace{2mm} \text{ and } \hspace{2mm} S_{11} / n_1 = \frac{S_{11}}{S_{10} + S_{11}} \to_p p_1. $$

The key insight here: our Bernoulli RVs are $Y_i | x_i = j \sim \text{Bern}(p_j)$ while our binomial RVs are $S_{j1} \sim \text{Bin}(n_j, p_j)$, but both have the same probability of success. That's the reason why these contingency table proportions are estimating the same thing as an observation-level logistic regression. It's not just some coincidence with the table: it's a direct consequence of the distributional assumptions we have made.

$\endgroup$
1
$\begingroup$

When you have more than one predictors and all the predictors are binary variables, you could fit a model using Logic Regression [1] (note it's "Logic" not "Logistic"). It's useful when you believe interaction effects among your predictors are prominent. There's an implementation in R (LogicReg package).

[1] Ruczinski, I., Kooperberg, C., & LeBlanc, M. (2003). Logic regression. Journal of Computational and graphical Statistics, 12(3), 475-511.

$\endgroup$
  • 1
    $\begingroup$ The question is specifically about one regressor, thus your answer would better serve as a comment. $\endgroup$ – Richard Hardy Mar 14 '18 at 9:18

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.