2
$\begingroup$

My dataset contains two numerical variables (n1, n2) and six indicator variables. The first three indicator variables specify the location of a property (i1=north, i2=center, i3=south), the next three indicator variables specify the color of walls (j1=blue, j2=red, j3=other). Let's say I want to predict the property price (y). I want to run an ordinary least squares regression. I have two questions:

  1. Should I include all indicator variables or should I exclude one per group? And why?

y = b1 * n1 + b2 * n2 + b3 * i1 + b4 * i2 + b5 * i3 + b6 * j1 + b7 * j2 + b8 * j3

vs

y = b1 * n1 + b2 * n2 + b3 * i1 + b4 * i2 + b6 * j1 + b7 * j2

  1. What is the difference between treating the i and j variables as two distinct groups of indicator variables (hence excluding one for each group) or treating them as the group of indicator variables (hence excluding just one)?
$\endgroup$

1 Answer 1

1
$\begingroup$

For part 1, yes you should exclude one of the dummies per group.

To be more specific, for every set of dummy variables that split the data into mutually exclusive and collectively exhaustive categories, you should omit one of the dummy variables to avoid the “dummy variable trap”.

The “dummy variable trap” refers to the perfect multicolinearity that arises when one of your independent variables can be perfectly predicted by the other variables in your regression. For example, if $j1$ and $j2$ are both equal to zero, then you already know for a fact that $j3$ will be equal to 1.

More formally, take your first example regression equation.

$$y = \beta_1 n_1 + \beta_2 n_2 + \beta_3 i_1 + \beta_4 i_2 + \beta_5 i_3 + \beta_6 j_1 + \beta_7 j_2 + \beta_8 j_3 + \beta_9 + \epsilon$$

(Note that I’ve added on an $\epsilon$ and a constant term $\beta_9$.)

Note also that by the definition of a dummy variable, since for any individual observation, only one of the $j$ variables can be equal to 1, you can write $$j_3 = 1 - j_2 - j_1$$ Therefore, $j_3$ is completely determined by the values of $j_1$ and $j_2$.

Now, substitute this back into the original equation: $$y = \beta_1 n_1 + \beta_2 n_2 + \beta_3 i_1 + \beta_4 i_2 + \beta_5 i_3 + \beta_6 j_1 + \beta_7 j_2 + \beta_8 (1-j_2-j_1) + \beta_9 + \epsilon$$

After some algebra, you could rewrite the equation as:

$$y = \beta_1 n_1 + \beta_2 n_2 + \beta_3 i_1 + \beta_4 i_2 + \beta_5 i_3 + (\beta_6-\beta_8) j_1 + ( \beta_7-\beta_8) j_2 + (\beta_9+\beta_8)+ \epsilon$$

Thus, you can rewrite the whole regression equation without $j_3$. So instead of including all of them, which will make your estimates of $\beta_6$, $\beta_7$, and $\beta_9$ incorrect.

Instead, you omit the group and interpret the coefficients on $j_1$ and $j_2$ as the changes in the outcome relative to the omitted group. So in your example, if $j_3$ is omitted, then $\beta_6$ represents the average increase in price that is associated with a house being blue, rather than “other”.

Note that this interpretation would not be possible if you included $j_3$ as a regressor. In that case, you would be comparing blue houses to some category of house that doesn’t exist, and so your coefficients would be essentially meaningless.

This same logic applies to the $i$ variables.

As for part 2, the $i$ and $j$ variables are distinct sets of dummy variables in the example you’ve given. That’s because the color of the house should not be perfectly correlated with its location (unless you have a really unusual dataset).

So you can’t uniquely determine the location of a house from its color, and vice versa.

In general, a “set of dummy variables” should represent a set of categories that are both:

  1. Mutually exclusive: no more than one of the variables can ever be equal to one for any observation
  2. Collectively exhaustive: one of the variables is equal to one for every observation

So the color example is perfect. A color can be either blue, red, or “other”. But it can’t be both blue and red, or blue and other. Thus, these categories are mutually exclusive. Further, since “other” captures everything that is not blue or red, every house must fall into one of these categories. So, the $j$ variables are also collectively exhaustive, and so one of them should be omitted.

Hope this helps.

$\endgroup$
2
  • $\begingroup$ Thank you for the excellent explanation. Why does b6 represent the average increase in price that is associated with a house being blue, rather than “other”? Shouldn't this be shown by the coefficient (b6 - b8)? $\endgroup$
    – NC520
    Sep 16, 2022 at 9:55
  • 1
    $\begingroup$ @NC520 yes you are correct. The point is that if you were to omit the “other” category, then there would be no b8 parameter to worry about. Instead you would be estimating only a parameter on j1 (which I was referring to as just b6) that would actually represent (b6-b8), but you could only identify the difference (b6-b8), and could not identify b6 and b8 separately. $\endgroup$
    – phil
    Sep 17, 2022 at 4:57

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.