# Categorical variable with very few observations

I'm intending to do a bayesian zero-inflated regression, where some of my regressors are categorical variables, like this one, the country regions:

>table(db\$region)
REGIAO1  REGIAO2  REGIAO3  REGIAO4  REGIAO5  REGIAO6  REGIAO7  REGIAO8
4851     2293     5471      410     1157      626     3683     1712
REGIAO9 REGIAO10 REGIAO11 REGIAO12 REGIAO13 REGIAO14 REGIAO15 REGIAO16
3550     1719    48208     3647     5293      716      545     2115
REGIAO17 REGIAO18 REGIAO19 REGIAO20 REGIAO21 REGIAO22 REGIAO23 REGIAO24
318    13884     1759     1048     3518      162     2899      559
REGIAO25 REGIAO26 REGIAO27 REGIAO28 REGIAO29 REGIAO30 REGIAO31 REGIAO32
574      407      326      110      194      367       85       36
REGIAO33 REGIAO34 REGIAO35 REGIAO36 REGIAO37 REGIAO38 REGIAO39 REGIAO40
70       11       33      478      388     1758     1185      160
REGIAO41 REGIAO00
116        4


What makes me worried is that we have few observation for some of these variables. So, far I considered three solutions:

1. Ignore the problem and use all the observations anyway.
2. Remove the categories where we have very few observations like REGIAO00.
3. Do my regression considering only the observations where region=11.

The questions are:

• (1) If I choose option 2, what is theoretical motivation I'd have to explain it in my article?
• (2) If choose number 1, would this cause problems in the estimation of the other categories and/or other regressor? (3) I've heard that bayesian inference has less problems with unbalanced designs, so would this be the case?

One might say that the number of observations there are not that small, but what would you say if I have a category with only one observation (this is also truly present in my dataset):

ANO_MODELO1994 ANO_MODELO1995 ANO_MODELO1996 ANO_MODELO1997 ANO_MODELO1998
420           1676           9327          12843          17984
ANO_MODELO1999 ANO_MODELO2000 ANO_MODELO2001 ANO_MODELO2002 ANO_MODELO2003
20812          17055          24397          10036           1894
ANO_MODELO1992
1


(4) In this situations, should I remove this category ANO_MODELO1992?

• Why would one not simply use all the data? – Glen_b Mar 31 '13 at 2:11
• @Glen_b I thought that this seems reasonable for most of them, but what about extreme cases like REGIAO00 which only have 4 observations and ANO_MODELO1992 that only have 1? I just wonder if using them or not would significantly affect the other estimates. – random_user Mar 31 '13 at 2:30
• If you have flat priors and no shrinkage, smoothing, random effects or other regularization on the relevant variable, the one with a single observation could be a concern. – Glen_b Mar 31 '13 at 3:34
• That makes sense! Unfortunately, this is the case, a simple cross-section with no mutually informative priors at all and in fact, at least for now, I'm only using uniform flat priors for everything. – random_user Mar 31 '13 at 4:53
• Even so, posteriors should be computable... but if you're not going to bring in any regularization at all, it might be worth leaving out the categories for which you can't form reasonable estimates. – Glen_b Mar 31 '13 at 6:35