How do you calculate a confidence interval for the difference between 2 dummy variables?

Question

I think I understand how to calculate the confidence interval for the difference in means of 2 groups when there's one dummy variable. You just add or substract 1.96 x (the standard error of the dummy coefficient) to the value of the dummy coefficient, eg b1 +- 1.96 x SE(b1) for a 95% confidence intercal. However as I understand it this only works when one of the groups you want to compare is a reference group (and doesn't have its own dummy variable). What if instead you want a confidence interval of the difference in means between 2 groups, both of which have a dummy? Say the regression is b0 + b1 x X1 + b2 x X2, with both X1 and X2 dummy variables. How do you find the confidence interval of the difference in means between X1 and X2?

Answer 1

If I'd want to know the 95% confidence interval of the difference in means between brown haired and other colours I'd just calculate it as b1+-1.96 x SE(b1), if I understood my handbook correctly. How do I make a similar confidence interval for the difference between blonde and brown haired people?

Solution 1

The difference between the height of people with brown coloured hair and blond coloured hair is $b_2-b_1$ and the standard error can be computed with the following rule for the variance of a sum of variables

$$Var(b_2-b_1) = Var(b_2) + Var(b_1) + 2 Cov(b_2, b_1)$$

So for this, you also need to know the covariance. Some software and packages for programming languages can give you this covariance. We can also compute it. The covariance is $\sigma^2 (X^TX)^{-1}$ and this $X$ contains an intercept and the dummy variables whether the hair is blue or brown so you get

$$X^TX = \begin{bmatrix} n_{total} & n_{blue} & n_{brown} \\ n_{blue} & n_{blue} & 0 \\ n_{brown} & 0 & n_{brown} \\ \end{bmatrix}$$

with the inverse

$$(X^TX)^{-1} = \frac{1}{det(X^TX)} \begin{bmatrix} n_{blue}n_{brown} & -n_{blue}n_{brown} & -n_{blue}n_{brown} \\ -n_{blue}n_{brown} & n_{brown} (n_{total}-n_{brown}) & -n_{blue}n_{brown} \\ -n_{blue}n_{brown} & -n_{blue}n_{brown} & n_{blue} (n_{total}-n_{blue}) \\ \end{bmatrix} $$

The correlation will then be

$$\rho = \frac{Cov(b_1,b_2)}{\sqrt{Var(b_1)Var(b_2)}} = \frac{-\sqrt{n_{blue}n_{brown}}}{\sqrt{(n_{total}-n_{blue})(n_{total}-n_{brown})}}$$

or

$$Cov(b_1,b_2) = \rho \sqrt{Var(b_1)Var(b_2)}$$

and finally

$$\begin{array}{} Var(b_2-b_1) &=& Var(b_2) + Var(b_1) + 2 Cov(b_1,b_2) \\ &=& Var(b_2) + Var(b_1) - \sqrt{{Var(b_1)Var(b_2)}}\sqrt{\frac{{n_{blue}n_{brown}}}{(n_{total}-n_{blue})(n_{total}-n_{brown})}} \end{array}$$

Solution 2

Much easier is to estimate without the intercept. So you have a model like

$$Height = b_{other} X_{other} + b_{blue} X_{blue} + b_{brown} X_{brown} $$

and then there won't be a correlation between the estimates $b_{blue}$ and $b_{brown}$ and you can compute it as

$$\begin{array}{} Var(b_{blue}-b_{brown}) &=& Var(b_{blue}) + Var(b_{brown}) + 2 \overbrace{Cov(b_{blue}, b_{brown}) }^{=0}\\&=& Var(b_{blue}) + Var(b_{brown}) \end{array}$$

Answer 2

Sextus Empiricus' answer is of course correct, but there is a simpler way to do it.

Consider the original equation:

$$y_i = \beta_0 + \beta_1 x_{1i} + \beta_2 x_{2i} + e_i$$

If $\beta_1 = \beta_2$. then this equation can be reduced to:

$$y_i = \beta_0 + \beta_1 (x_{1i} + x_{2i}) + e_i$$

Now we add another term $\gamma x_{2i}$ to the equation:

$$y_i = \beta_0 + \beta_1 (x_{1i} + x_{2i}) + \gamma x_{2i} + e_i$$

We can rearrange terms to get:

$$y_i = \beta_0 + \beta_1 x_{1i} + (\beta_1 + \gamma) x_{2i} + e_i$$

and by comparing with the original expression we see that $\beta_2 = \beta_1 + \gamma$, or, alternatively, $\gamma = \beta_2 - \beta_1$. This is what you want to estimate.

Consequently, estimating the parameters of:

$$y_i = \beta_0 + \beta_1 z_i + \gamma x_{2i} + e_i, \,\,z_i = x_{1i} + x_{2i} $$

allows us to test $\beta_2 - \beta_1 = 0$ by testing the equivalent $\gamma = 0$ using the standard results displayed by regression packages, and similarly for constructing confidence intervals.