For linear regression, I understand that there is a 1 in 10 rule. For example, if I have a continuous dependent variable and 100 observations then at most I could add 10 binary or continuous independent variables in the multivariable model.

For logistic regression, if I have a nominal dependent variable such as eye colour: blue (N=50), green(N=30) and brown(N=20), then I could at most have 2 independent variables in the model, i.e. the number of observations in the category with the lowest N divided by 10(i.e. 20/10=2).

For ordinal logistic regression(proportional odds), for example, the ordinal dependent variable is Tobacco use frequency: daily(N=50), regularly(N=30) and occasionally or no use(N=20). I wonder how many independent variables could be included? Is it the same as logistic regression with nominal outcomes? e.g. 20/10=2?

Some other questions are: What would be the consequences if I over fitted a ordinal logistic model? Is there a method that I could test the overfitting quantitatively? Would there be a difference in sample size requirements between an explanatory ordinal logistic regression and a predictive ordinal logistic regression?

Any help would be much appreciated!

  • $\begingroup$ These rules of thumb are not actually rules. In practice, they may not be so helpful. One way to avoid overfitting if the goal is prediction is to have some data separate and test the performance of the model on the separate data relative to other models. Search "cross validation". If explanation, regularization approaches like ridge regression or Bayesian approaches can help you explain the data while being skeptical about the effects you expect to find, because you can shrink the effects towards zero. $\endgroup$ – Heteroskedastic Jim Jul 29 '18 at 9:31

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.