0
$\begingroup$

Is there any benefit to doing a for loop so that a logistic regression model is fit separately for each independent variable (i.e. 100 models all using the same dependent variable)? As opposed to fitting one model with all 100 independent variables and the dependent variable.

$\endgroup$
3
  • 2
    $\begingroup$ Why do you think there would be an advantage? The disadvantages are numerous: Increased omitted variable bias, no way to incorporate interactions, no way to prevent confounding from any of the 99 remaining explanatory variables. $\endgroup$ Oct 30 '19 at 13:23
  • $\begingroup$ @FransRodenburg I don’t know that there would be an advantage. That’s why I was asking the question. $\endgroup$
    – Insu Q
    Oct 30 '19 at 13:48
  • 1
    $\begingroup$ Depending on how much data you have, 100 variables may be more than you have enough information for (cf, Logistic regression with small number of cases). $\endgroup$ Oct 31 '19 at 19:02
1
$\begingroup$

There is a major disadvantage to fitting individual predictors separately in regression. This page points out a major difficulty in general and a difficulty specific to generalized models like logistic regression.

Even in ordinary least-squares regression, omitting a predictor that is both correlated to outcome and to the included predictors leads to omitted-variable bias in the estimates of the coefficients of the included predictors. In logistic regression, omitting any predictor associated with outcome, even one not correlated with the included predictors, leads to omitted-variable bias.

Thus your coefficient estimates from the single-predictor regressions would be biased in difficult-to-know ways. And as a comment points out, the single-predictor route does not allow for examining interactions. I can think of no offsetting benefit.

$\endgroup$
0
$\begingroup$

That would be similar to what naive bayes classifier is doing. There is no advantage for prediction. The only advantage is that this can be much faster to fit, while Ib singe situation not having too much worse performance.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.