5
$\begingroup$

I have a confusion in multiple regression about dummy variable trap, so far I had seen tutorials explaining about dummy variable trap and multicollinearity but I'm unable to understand it fully.

Improve this question
$\endgroup$
2
  • $\begingroup$ I can't see that you have a new question here. Search the forum for "dummy variable trap". I've specified a good thread as duplicated, which has more general pertinence, which is here a good thing. $\endgroup$
    – Nick Cox
    Jun 26, 2019 at 11:10
  • 2
    $\begingroup$ @NickCox That's not of my understanding level I'm a beginner and want a simple way to get the concept right. $\endgroup$
    – Sahil Silare
    Jun 26, 2019 at 11:40

1 Answer 1

Reset to default
6
$\begingroup$

Let's say you have a binary variable, like sex. You create two dummy variables to reflect that in your model. Let's say you have six individuals $(M,F,F,M,M,F)$. Your dummy variables look like:

  • $X_1=(0,1,1,0,0,1)$
  • $X_2=(1,0,0,1,1,0)$

But now $X_{i1}+X_{i2} = 1$ for every possible $i$ so you have a case of perfect multicolinearity. The model will not distinguish between an effect caused by a high $X_1$ or a low $X_2$ and vice-versa.

The way to avoid this trap is to get rid of one of those variables. but this implies taking one of the groups as a "reference" which is kind of an arbitraty choice.

More importantly, when considering multiple factors simultaneously, it may be the case that some of the dummy variables reach perfect multicolinearity due to the way your individuals are distributed among the groups.

Imagine, for example, you also have data like "taller than 170 cm/shorter than 170 cm" and you get $(T,S,S,T,T,S)$ (which is not rare to expect) You will be facing a similar problem to that we had when considering $X_1$ and $X_2$

Improve this answer
$\endgroup$
10
  • $\begingroup$ But let say I have 3 categorical variables ${X_{1}, X_{2} , X_{3} }$ such that if we neglect $X_{3}$ we get a equation that for some $i$ and $j$ , $X_{i1}+X_{i2}=X_{j1}+X_{j2}$ then isn't this called collinearity too? We'll then have to neglect another variable I guess. $\endgroup$
    – Sahil Silare
    Jun 26, 2019 at 11:38
  • 1
    $\begingroup$ I mean to say that if we have 3 variables and even after neglecting one we get the same results as perfect collinearity then do we have to remove more variables? $\endgroup$
    – Sahil Silare
    Jun 26, 2019 at 11:43
  • 1
    $\begingroup$ @SahilSilare Yes! We must remove variables until multicolinearity disappears, but multicolinearity appears when the equality you showed is true for all possible $i$ and $i=j$ and, not just when there are some $i$ and $j$ that make it true $\endgroup$
    – David
    Jun 26, 2019 at 11:46
  • 4
    $\begingroup$ This answer might be made clearer by explicitly pointing out it is assuming a constant term is included in the regression: only in this way can we understand your examples of collinearity. This would also make it apparent there is another way to eliminate the collinearity: remove the constant term. This solves the problem of designating a reference class, too. $\endgroup$
    – whuber ♦
    Jun 26, 2019 at 14:50
  • 1
    $\begingroup$ Sahil Yes, that's right, assuming--as David has cautioned in a comment--you have only one categorical regressor in the model. $\endgroup$
    – whuber ♦
    Jun 26, 2019 at 16:34

Not the answer you're looking for? Browse other questions tagged or ask your own question.