1
$\begingroup$

From what I've read, logistic regression assumes

  1. $y_i\sim \text{Bernoulli}(p)$
  2. $y_i \perp y_j$, for $i\ne j$ and $i,j \in \lbrace 1,..., n\rbrace$
  3. log-odds is linear in $X$, i.e. $\text{logit}(p)=X\beta$
  4. no multicollinearity among predictors
  5. $n$ is large enough

However, I'm confused about assumptions 3 and 4 in the context where all my predictors are binary.

What plot or test should I run to assess if the linearity assumption is satisfied?

For multicollinearity, I know the way we define correlation matters. For instance, if your using ordinal data, Spearman correlation is appropriate. For nominal, polychoric correlation is appropriate. So, when doing a multicollinearity test, is it still reasonable to usual tests like VIF, condition numbers, and pair-wise correlation to evaluate multicollinearity? If not, what tools are appropriate? Would a pair-wise correlation with tetrachoric be appropriate? In the numerical case, I thought this was flawed because not all multicollinearity occurs just between pairs but can be groups of several predictors correlated with each other. Hence, VIF and condition numbers are used. Wouldn't this issue be present in binary data? If so, what is the binary equivalent of VIF and condition numbers?

$\endgroup$

1 Answer 1

1
$\begingroup$

Assumption 2 states that the observations are independent from each other. This would be violated if individuals shared an unmeasured quality based on their location, for example. It is commonly violated when students are nested with school, cities within states, patients within hospitals, etc. There are a few ways to control for dependence, which include using mixed models or generalized estimating equations.

Assumption 3 just means that you have the model form correct. This is an unverifiable assumption. It may be possible to see whether there are some egregiously wrong predictions, which would indicate that your functional form is incorrect, and you can use other methods to prune a larger model (e.g., lasso), but the assumption of linearity just means that the logistic regression model you specified correctly represents the relationship between the predictors and the outcome.

Multicollinearity is automatically satisfied when you use automated software to fit logistic regression. It is not a statement about how correlated covariates are with each other; its a statement about the rank of the design matrix. Logistic regression is valid with variables that are correlated at .99 (however you measure that), but unless your sample size is massive, there will be extreme imprecision in the estimates and the design matrix may not be invertible. This is a computational problem, though, not related to the assumptions of logistic regression. With binary variables, this amounts to not including both levels of the binary predictor and an intercept in the model. You wouldn't do this anyway, so you don't need to worry about it. Including perfectly redundant variables is another way to violate this assumption, but most software will figure that out and remove the offending variables for you.

You don't need to think about how you are measuring the association between predictors to figure out if this assumption has been met. The VIF measures how much additional uncertainty is caused by having correlated predictors, but this has nothing to do with the assumptions of logistic regression. It's better to have independent predictors to maximize the precision of the coefficient estimates, but estimates are still consistent even if the predictors are highly correlated (however you measure that).

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.