-1
$\begingroup$

I want to construct a model with 5 categorical variables(no continuous variable), and all of them have more than 2 levels. Should I use ANOVA or other methods to set up my model? And how to express my model in a equation form? Besides, how can I add some interaction terms into model? The response variable is continuous.

$\endgroup$

1 Answer 1

1
$\begingroup$

Of course you can. The question remains, however, what the model will tell you. If you have one categorical variable with say 3 levels, you would use dummy coding, i.e. add two dummy variables which indicate whether two of the levels are taken or not. If both dummy variables are zero in a observation, then necessarily the other level has to be "true". This is what is called the reference level (for the reference level there will not be a coefficient $\beta_i$ in the model. The reference level is solely expressed in terms of the absence of the other two levels (and naturally all other variables in the model).

If you plan on using more than one categorical variable, then you can use this standard procedure. Technically you could also simply code the reference categories as additional dummy variables in order to bring all of them into the model. However, you will then end up having severe multicollinearities in the model. This is due to the fact that if for each level of a categorical variable a dummy variable is created, these dummy variables will some up to $\mathbf{1}$, the vector of ones. They are, hence, linearly dependent. However, one usually wants to omit multicollinearities in a model.

One other way would be to create several models for each categorical variable to avoid multicollinearities. Depending on the type of your categorical variables it might also be worth considering effects coding or contrast coding rather than dummy coding.

$\endgroup$
3
  • $\begingroup$ The suggestion at the end, that a collection of univariate regressions could substitute for the desired multiple regression, is misleading: that's a completely different approach and ordinarily will produce different answers than the intended model. Moreover, it is offered to solve a problem that might not exist: there isn't necessarily any collinearity introduced when more than one categorical independent variable is used in a regression. $\endgroup$
    – whuber
    Commented Apr 19, 2018 at 12:57
  • $\begingroup$ I did not mean that there necessarily is a multicollinearity. What I mean that there will always be one if one does dummy code all levels of a variable. $\endgroup$
    – YukiJ
    Commented Apr 19, 2018 at 13:55
  • $\begingroup$ Thank you for your asking! But I'm still wondering that if I use dummy variables, I may use 20+ variables in my model. And as you said, it may cause severe multicollinearities, thus should I use both dummy coding way and ANOVA to figure out which model is better? $\endgroup$
    – Emma Wang
    Commented Apr 19, 2018 at 14:17

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.