I have a dataset with several categorical variables. I have been running some regressions and used dummy coding for these categorical variables. The problem is that some specifications lead to perfect collinearity between a few vars when including all (but one) levels of a categorical variable of interest. Thus I need to omit two levels instead of one.
For instance, suppose variable $x_1$ is categorical and has four levels. Including the first three levels as dummies leads to perfect collinearity between the third level and variable $x_2$. Thus, the third level of $x_1$ is dropped for modeling purposes. Then variable $x_1$ has two instead of one reference levels.
This leads to statistically significant results, but I am not sure it is correct to do this at all. If so, is there any proper way to interpret the results?