0
$\begingroup$

This question already has an answer here:

Is there some definite relationship between number of covariates and the sample size in logistic regression? (e.g. larger the number of covariates, larger the sample size needed, etc.)

Thank you,

$\endgroup$

marked as duplicate by whuber regression Jan 25 at 22:45

This question has been asked before and already has an answer. If those answers do not fully address your question, please ask a new question.

  • $\begingroup$ When we have other confounding variables or covariates that we need to adjust for. $\endgroup$ – Penguin_Knight Jan 25 at 21:39
  • $\begingroup$ but even when there are confounding variables, is the best model in terms of the power analysis always the model that includes least number of covariates (including the confounders)? $\endgroup$ – HDC Jan 25 at 21:41
  • 1
    $\begingroup$ Presuming there's no penalty for model complexity or data collection costs, the best model is found among those that have all the covariates that matter and only those. The problem is that in many cases nobody knows what they are! $\endgroup$ – whuber Jan 25 at 21:53
  • $\begingroup$ see Harrell Regression Modeling Strategies chapter 4. For clinical biostatistical applications, rule of thumb is that n/p ~ 10-20, where n is min(number of successes, number of failures) and p is number of parameters to estimate. $\endgroup$ – Ben Bolker Jan 25 at 22:25
3
$\begingroup$

A useful rule of thumb for standard logistic regression is that you should have 10-20 cases in the lower-prevalence category per parameter whose value you are trying to estimate. A useful reference on this is section 4.4 of Harrell's Regression Modeling Strategies (second edition) or of his course notes.

A few additional notes are in order, however.

First, to expand on a comment from @whuber, logistic regression has an inherent omitted variable bias. Unlike linear regression, omitting a covariate that is related to outcome can bias all the estimates in your model, even if the omitted covariate is uncorrelated to those that you include. So it is really important to try to include "all the covariates that matter," as he suggests

Second, even if you follow the suggestion for 10-20 cases per parameter you might still end up with perfect separation in which some combination of predictors precisely separates your classes. Although this might be a correct finding, the standard algorithm for estimating regression coefficients will not converge and there is a danger that such a result would not repeat in another sample from the population.

Third, you can work around both the limit on the number of parameters and perfect separation by using a regularization approach like ridge regression instead of standard regression. Regularization places penalties on the parameter-value estimates that effectively decrease the number of parameters that are estimated, and it also helps solve the perfect separation problem.

$\endgroup$
  • $\begingroup$ Hello, thank you for your reply. The purpose of my studies is actually determining the sample size from power analysis. Would it be a good idea to first conduct ridge logistic regression on the response against every possible covariates, and see which parameters shrinks to zero, and then fit the logistic regression on the response again, but this time only against the set of covariates that didn't shrink to 0 in ridge regression stage? Thank you $\endgroup$ – HDC Jan 25 at 23:23
  • $\begingroup$ @HDC One-variable-at-a-time is not a good approach. Simulation is a good way to analyze power for logistic regression. If you already have a pilot study providing standard errors for regression coefficients, note that standard errors scale with the square root of the number of cases, and you need 2.8 standard errors for 80% power at 5% type-I error. If the study is complete, then post-hoc power analysis is inappropriate. $\endgroup$ – EdM Jan 26 at 17:56

Not the answer you're looking for? Browse other questions tagged or ask your own question.