Is there some definite relationship between number of covariates and the sample size in logistic regression? (e.g. larger the number of covariates, larger the sample size needed, etc.)
Thank you,
Is there some definite relationship between number of covariates and the sample size in logistic regression? (e.g. larger the number of covariates, larger the sample size needed, etc.)
Thank you,
A useful rule of thumb for standard logistic regression is that you should have 10-20 cases in the lower-prevalence category per parameter whose value you are trying to estimate. A useful reference on this is section 4.4 of Harrell's Regression Modeling Strategies (second edition) or of his course notes.
A few additional notes are in order, however.
First, to expand on a comment from @whuber, logistic regression has an inherent omitted variable bias. Unlike linear regression, omitting a covariate that is related to outcome can bias all the estimates in your model, even if the omitted covariate is uncorrelated to those that you include. So it is really important to try to include "all the covariates that matter," as he suggests
Second, even if you follow the suggestion for 10-20 cases per parameter you might still end up with perfect separation in which some combination of predictors precisely separates your classes. Although this might be a correct finding, the standard algorithm for estimating regression coefficients will not converge and there is a danger that such a result would not repeat in another sample from the population.
Third, you can work around both the limit on the number of parameters and perfect separation by using a regularization approach like ridge regression instead of standard regression. Regularization places penalties on the parameter-value estimates that effectively decrease the number of parameters that are estimated, and it also helps solve the perfect separation problem.