1
$\begingroup$

I have a problem with logistic regression. I had found out (here) that one of the assumptions of logistic regression model should be min. of for example 50 observations per predictor. But if I had created dummy variables in Stata using "i." operator, does every dummy category count as new predictor?

example:

  • i.maritalstatus
  • 1=Ref.
  • 2
  • 3 ...

Thank You very much.

$\endgroup$
5
$\begingroup$

It's not an assumption; it's a rule of thumb to avoid over-fitting. You'd have to ask the author of the document you link to in exactly what circumstances they envisaged it applying, but it clearly isn't going to work if you have a large number of categories per predictor, or if the number of observations in one response category is very small.

More common rules of thumb state that for each coefficient you're estimating (bar the intercept) you should have at least 10–20 observations in the least common response category. But you should check for over-fitting using bootstrap validation or cross-validation in any case when it's a concern.

$\endgroup$
  • 1
    $\begingroup$ And you need 96 observations just to estimate the intercept (that will yield a margin of error of 0.1 in estimating the overall probability). $\endgroup$ – Frank Harrell Feb 2 '14 at 16:38
  • $\begingroup$ But does this mean I need min. 50 observations per category in crosstabulation of single independent variable and dependent variable. Or there should also be at least 50 obs. per category in crosstabulation of every single independent variable with other independent variables? Thank You very much! $\endgroup$ – lvm3n Feb 2 '14 at 17:18
  • $\begingroup$ Count the number of coefficients for all predictors in your model, including any interaction & non-linear terms. If you haven't at least 10 to 20 times that number of observations in the minority response class you may have serious over-fitting. Having more doesn't guarantee that you don't, or that there aren't other problems, or that you can estimate coefficients as precisely as you might wish (see @Frank's comment). $\endgroup$ – Scortchi Feb 2 '14 at 19:55
  • $\begingroup$ Iam using binary variables and my least common response category in independent variable has 116 observations (0=2800obs; 1=116obs.). Is it potentionally enough or this minimal mumber of observations apply for 2x2 tables, where I have only 14 observations in my least common category (1 and 1) after crosstabulation? Thanks. $\endgroup$ – lvm3n Feb 4 '14 at 14:00
  • $\begingroup$ I think you're asking about the number of observations in each category of a categorical predictor, which affects the precision of your estimates. Are you fitting a model with interactions? If not, it's less of an issue, except insofar as it indicates confounding. $\endgroup$ – Scortchi Feb 4 '14 at 16:00
2
$\begingroup$

Further to Scortchi's comment, clearly the more categories that are created for the same number of observations, the more prone to overfitting the model will be. Worst case scenario is a separate category for each observation.

$\endgroup$
2
$\begingroup$

Adding to what others have said; if the logic isn't clear, just imagine a categorical variable "Subject name". This would result in a perfectly overfit model and an exact (but totally spurious) fit.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.