I have an OLS regression with around 1000 observations and created a dummy variable for at least 15 different categories (catholic, muslim, hindu etc). One of the created dummy variables (spiritual) indicates 1 for only a few observations (3). My question is, is it better to leave it in the regression or to merge with another category, e.g. "other religion" which has more observations (30)? Does it matter at all for my regression when I use these dummies as controls?

  • $\begingroup$ If a predictor (not only dummy one) is very much skewed it is suspicious for possible outliers. For example, your dummy with 997 and 3 counts - who will bet that those 3 are not slip of a pen and are real? Another reason to despise highly skewed/unbalanced factors is that they entangle main effects with interactions and also they lower statistical power of tests of the effects. $\endgroup$
    – ttnphns
    Commented Mar 10, 2014 at 7:39
  • $\begingroup$ I have around 15 categories, some have as many as 100 obs and others few like 3, 6 etc. I am doing a multilevel regression. Strangely, after grouping the variables into "other" my significance level and R^2 drops slightly. $\endgroup$
    – CodeTrek
    Commented Mar 10, 2014 at 7:59
  • $\begingroup$ You could keep them, but try regularization, like the fused lasso. $\endgroup$ Commented Nov 2, 2020 at 0:55


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Browse other questions tagged or ask your own question.