If I have a binary variable, say sex, and I want to test whether multiple other variables are associated with it. To do this, I run a logistic regression of the form

\begin{equation} logit(probability(sex = male)) = \beta X1 + \beta X2 ... + \beta Xk \end{equation}

Once I do this, I calculate the pseudo R-squared, and it is 0.45 meaning that the regression explained 45% of the variance in sex.

My question is, is it also fair/correct to interpret this as sex explained 45% of the variance in the regressors?

Likewise, if the odds ratio for X1 is 2.0, can one claim that being male increased the odds of X1 occurring by 100%?

Basically, can the equal sign in the regression equation truly be treated as an equal sign (i.e., bi-directional equivalence) if I have no claim to directionality?

  • $\begingroup$ The linear equation estimated by the model is an equality, but that doesn't apply to the inferences made by the model. $\endgroup$ – david25272 Aug 21 '17 at 0:58
  • $\begingroup$ Is there any specific reason you tagged "causality"? $\endgroup$ – Carlos Cinelli Aug 21 '17 at 6:00
  • 1
    $\begingroup$ Adding to other answers: we can talk about "explained variance" only when conducting linear regression. For GLMs $R^2$ has nothing to do with explained variance. $\endgroup$ – Tim Aug 21 '17 at 6:25
  • $\begingroup$ @carloscinelli I tagged causality because, if interferences are directionally constrained, that is a sort of causal inference. For example, if we can claim that X has an effect on Y but not the inverse, we at least know that Y doesn't cause X. $\endgroup$ – JRF1111 Aug 21 '17 at 22:49
  • $\begingroup$ @JRF1111 ok, that opens a whole lot of different answers. Causality is definitely directionally constrained, that is, for causal models the equal sign is not a literal equality, it's more like an assignment operator, and the relationship is asymmetric $y \leftarrow x$ . $\endgroup$ – Carlos Cinelli Aug 21 '17 at 23:19

You do have a claim to directionality, it is just implicit.

A more explicit statement of the logistic model is:

$$ y \mid x \sim \text{Bernoulli}(p = logit^{-1}(\beta_0 + \beta_1 x_1 + \beta x_2 ... + \beta x_k)) $$

Here, $\sim$ means "is distributed as".

You are making a distributional statement about $y$ conditional on $x$, and conditioning is not symmetric. Because of the conditional, the $x$'s are not considered random by the model, so $y$ cannot be thought of as explaining variance in the $x$'s (at least from the point of view of the model).

  • $\begingroup$ Great response. Perhaps this is best left to a new question, but do you know of some methods that allow for binary, categorical, and non-normal variables where the conditioning is symmetric or where the xs are conditional on y? $\endgroup$ – JRF1111 Aug 20 '17 at 22:44

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.