What is the correct way to test for significant differences between coefficients?

Question

I'm hoping someone can help straighten out a point of confusion for me. Say I want to test whether 2 sets of regression coefficients are significantly different from each other, with the following set up:

• $y_i = \alpha + \beta x_i + \epsilon_i$, with 5 independent variables.
• 2 groups, with roughly equal sizes $n_1, n_2$ (though this may vary)
• Thousands of similar regression will be done simultaneously, so some kind of multiple hypothesis correction has to be done.

One approach that was suggested to me is to use a Z-test:

$Z = \frac{b_1 - b_2}{\sqrt(SEb_1^2 + SEb_2^2)}$

Another I've seen suggested on this board is to introduce a dummy variable for grouping and rewrite the model as:

$y_i = \alpha + \beta x_i + \delta(x_ig_i) + \epsilon_i$, where $g$ is the grouping variable, coded as 0, 1.

My question is, how are these two approaches different (e.g. different assumptions made, flexibility)? Is one more appropriate than the other? I suspect this is pretty basic, but any clarification would be greatly appreciated.

Answer 1

The two approaches do differ.

Let the estimated standard errors of the two regressions be $s_1$ and $s_2$. Then, because the combined regression (with all coefficient-dummy interactions) fits the same coefficients, it has the same residuals, whence its standard error can be computed as

$$s = \sqrt{\frac{(n_1-p) s_1^2 + (n_2-p) s_2^2)}{n_1 + n_2 - 2 p}}.$$

The number of parameters $p$ equals $6$ in the example: five slopes and an intercept in each regression.

Let $b_1$ estimate a parameter in one regression, $b_2$ estimate the same parameter in the other regression, and $b$ estimate their difference in the combined regression. Then their standard errors are related by

$$SE(b) = s \sqrt{(SE(b_1)/s_1)^2 + (SE(b_2)/s_2)^2}.$$

If you haven't done the combined regression, but only have statistics for the separate regressions, plug in the preceding equation for $s$. This will be the denominator for the t-test. Evidently it is not the same as the denominator presented in the question.

The assumption made by the combined regression is that the variances of the residuals are essentially the same in both separate regressions. If this is not the case, however, the z-test isn't going to be good, either (unless the sample sizes are large): you would want to use a CABF test or Welch-Satterthwaite t-test.

Answer 2

The most direct way to test for a difference in the coefficient between two groups is to include an interaction term into your regression, which is almost what you describe in your question. The model you would run is the following:

$y_i = \alpha + \beta x_i + \gamma g_i + \delta (x_i \times g_i) + \varepsilon_i$

Note that I have included the group variable as a separate regressor in the model. With this model, a $t$-test with the null hypothesis $H_0: \delta = 0$ is a test of the coefficients being the same between the two groups. To see this, first let $g_i = 0$ in the above model. Then, we get the following equation for group 0:

$y_i = \alpha + \beta x_i + \varepsilon_i$

Now, if $g_i = 1$, then we have:

$y_i = (\alpha + \gamma) + (\beta + \delta) x_i + \varepsilon_i$

Thus, when $\delta$ is 0, then two groups have the same coefficient.