I use univariate analysis to select variables for a multiple logistic regression. However, one of the categorical independent variables has a non significant dummy (one of the three). Should I include this variable in my multiple logistic regression?

And how many variables can I include in my multivariable model? My dependent variable is being informed, with 129 respondents being informed and 105 being uninformed.

  • $\begingroup$ You should not be doing univariate analysis at all. This has come up quite a few times on the site, and I would encourage you to search for some previous questions to understand why it is a bad idea. $\endgroup$ – Demetri Pananos Jun 19 '20 at 17:52

You should try to include any predictor associated with outcome in a logistic regression, but you should consider a different approach to selecting predictors.

Selecting predictors for multiple regression models based on their individual associations with outcome is not a very good idea in general. Even in standard linear regression, if you omit any predictor that is correlated both with outcome and with the included predictors you run a risk of omitted-variable bias. You might thus ignore a predictor whose importance is only clear when other predictors are taken into account.

That approach is particularly problematic in logistic regression, as then you risk bias even if an omitted predictor isn't correlated with the included predictors. So you need to take a more general approach.

The usual rule of thumb for logistic regression is that you can evaluate about 1 predictor for every 15 members of the minority class. (Categorical variables count as 1 predictor for every level beyond the first.) So with 105 in the smallest class you should be safe including half a dozen or so predictors in your multiple logistic regression.

If you have more than that many predictors, you have a few ways to proceed. First is to use your knowledge of the subject matter to choose predictors for inclusion. That's usually the wisest approach.

If you're just a bit beyond half a dozen or so predictors, you might consider limited backward selection from a full model. In general, automated selection of predictors isn't a good idea, but it can be done in the interest of parsimony as part of a careful analysis approach.

If you have a lot more than half a dozen or so candidate predictors and your main interest is in predicting future cases you could consider ridge regression, which includes all predictors but penalizes their regression coefficients to provide better performance on new data. LASSO will select predictors in an all-or-none fashion, but it can be tricky with multiple-level categorical predictors and the particular predictors it selects can depend heavily on the data set at hand; they aren't necessarily the "most important" predictors in any useful sense.


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.