1
$\begingroup$

I want to perform a simply regression of a variable y on several independent variables (x1, x2, x3) and additionally on dummy variable(s) for each of 10 categories.

I read about dummy variables, so I created one for each of the 10 categories, whereby one of them is the base, i.e. consists of only zeros. Next I performed the regression including all 10 dummy variables but there was an error: Coefficients: (1 not defined because of singularities). I think this is because of perfect multicollinearity of a dummy variable with the intercept.

So from what I read, I can fix this error by removing the base dummy; this will be covered by the intercept instead.

First question: Is this correct? If so, it is ok, but when I look at the output I'd prefer to have sthg there which is not labelled as 'intercept', but rather as the base dummy which it refers to.

So here is my main question: Could I also drop the intercept and include the base dummy instead? I have tested this, but it does not seem to work. I believe if I dropped the intercept I would have to include a new dummy for the base category which also includes ones and not only zeros. Is this the right way?

Thanks for any hints. The sources on the internet were not really clear about this.

Improve this question
$\endgroup$
  • $\begingroup$ Every statistics package (SAS, SPSS, R for sure) will do this for you, usually offering several options of how to do it. Why do you want to do it by hand? $\endgroup$ – Peter Flom Jan 21 '17 at 13:38
  • $\begingroup$ Hmm I would like to conceptually understand the process of making dummies - you cannot just simply trust the packages :) But also because of my data structure I found it easier to do things myself. $\endgroup$ – Rian Jan 21 '17 at 13:44
  • 1
    $\begingroup$ To understand the process, here is a good link: methodology.psu.edu/media/techreports/12-120.pdf There are lots of others. $\endgroup$ – Peter Flom Jan 21 '17 at 13:51
  • $\begingroup$ Thank you, I will try this out. Nontheless I would be happy if someone could answer my questions, which are rather specific and haven't been answered before afaik. As I said I have read a few sources already - the problem is more how to set up the regression and this is never explained in detail unfortunately. $\endgroup$ – Rian Jan 21 '17 at 13:54
  • $\begingroup$ Here is a brief note that discusses your question: maartenbuis.nl/publications/ref_cat.html $\endgroup$ – Maarten Buis Jan 21 '17 at 20:03
1
$\begingroup$

When you run your regression model you get estimates of the coefficients that minimize deviations. But if some of your vectors are collinear to other, then there are infinite number of coefficients possible values that solve LS task that cause the error (just read about linear equations theory). When you add both constant and dummy variables, then your constant is collinear to these dummies (and vice versa according to linear algebra). So in order to avoid collinearity you should to avoid constant or one of your dummy variables. Finally vector of zeros is collinear to any other vector so never include it in your model. So your idea about multicollinearity was absolutely correct.

Improve this answer
$\endgroup$
  • $\begingroup$ Alright thanks. Just to clarify: So using either the intercept OR all dummies for every category (with '1' for the associated categories, i.e. no base category, with only '0') should lead to the same result, correct? $\endgroup$ – Rian Jan 22 '17 at 15:40
  • $\begingroup$ The difference is only in interpretation. If you use constant, then you omit one dummy variable. So any coefficient of the other of these dummy variables is how much bigger (smaller, depending on sing) effect of this dummy then the effect of one you have omitted. When you use all your dummy variales, then you assume that it is no constant effect that is, in most cases, seems not very good practies so researches are always include constant. Predictions in both models will be the same as they both are identical, only interpretation changes. $\endgroup$ – Bogdan Jan 22 '17 at 20:33

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.