34
$\begingroup$

This seems to be a basic issue, but I just realized that I actually don't know how to test equality of coefficients from two different regressions. Can anyone shed some light on this?

More formally, suppose I ran the following two regressions: $$ y_1 = X_1\beta_1 + \epsilon_1 $$ and $$ y_2 = X_2\beta_2 + \epsilon_2 $$ where $X_i$ refers to the design matrix of regression $i$, and $\beta_i$ to the vector of coefficients in regression $i$. Note that $X_1$ and $X_2$ are potentially very different, with different dimensions etc. I am interested in for instance whether or not $\hat\beta_{11} \neq \hat\beta_{21}$.

If these came from the same regression, this would be trivial. But since they come from different ones, I am not quite sure how to do it. Does anyone have an idea or can give me some pointers?

My problem in detail: My first intuition was to look at the confidence intervals, and if they overlap, then I would say they are essentially the same. This procedure does not come with the correct size of the test, though (i.e. each individual confidence interval has $\alpha=0.05$, say, but looking at them jointly will not have the same probability). My "second" intuition was to conduct a normal t-test. That is, take

$$ \frac{\beta_{11}-\beta_{21}}{sd(\beta_{11})} $$

where $\beta_{21}$ is taken as the value of my null hypothesis. This does not take into account the estimation uncertainty of $\beta_{21}$, though, and the answer may depend on the order of the regressions (which one I call 1 and 2).

My third idea was to do it as in a standard test for equality of two coefficients from the same regression, that is take $$ \frac{\beta_{11}-\beta_{21}}{sd(\beta_{11}-\beta_{21})} $$

The complication arises due to the fact that both come from different regressions. Note that

$$ Var(\beta_{11}-\beta_{21}) = Var(\beta_{11}) + Var(\beta_{21}) -2 Cov(\beta_{11},\beta_{21}) $$ but since they are from different regressions, how would I get $Cov(\beta_{11},\beta_{21})$?

This led me to ask this question here. This must be a standard procedure / standard test, but I cound not find anything that was sufficiently similar to this problem. So, if anyone can point me to the correct procedure, I would be very grateful!

$\endgroup$
  • 1
    $\begingroup$ This seems to relate to structural/simultanous equation modeling. One way of solving this problem is fitting both equations simultanously, e.g. with maximum likelihood, and then use a likelihood ratio test of a constrained (equal parameter model) against an unconstrained model. Practically this can be done with SEM software (Mplus, lavaan etc.) $\endgroup$ – tomka Apr 12 '14 at 13:11
  • 2
    $\begingroup$ Do you know about Seemingly Unrelated Regression (SUR)? $\endgroup$ – Dimitriy V. Masterov Apr 12 '14 at 16:01
  • 1
    $\begingroup$ I think the question your raise, i.e. how to get the cov of both coefficients, is solved by SEM, which would give you the var-cov matrix of all coefficients. Then you could possibly use a Wald test in the way you suggested instead of a LRT test. Furthermore you could also use re-sampling / bootstrap, which may be more direct. $\endgroup$ – tomka Apr 12 '14 at 16:44
  • 2
    $\begingroup$ Yes, you are right about that, @tomka. In a SUR model (which you can loosely speaking consider a special case of SEM models), I can get the appropriate test. Thanks for pointing me in that direction! I think I did not think about it because it seems a little bit like shooting a sparrow with a cannon, but I can indeed not think of a better way. If you write up an answer, I will mark it as correct. Otherwise, I will write it up myself soon, with a quick theoretical explanation and potentially with an example. $\endgroup$ – coffeinjunky Apr 12 '14 at 18:25
  • 1
    $\begingroup$ SUR is pretty easy to implement. Here's one example with Stata. With R, you want systemfit. $\endgroup$ – Dimitriy V. Masterov Apr 12 '14 at 19:24
25
$\begingroup$

Although this isn't a common analysis, it really is one of interest. The accepted answer fits the way you asked your question, but I'm going to provide another reasonably well accepted technique that may or may not be equivalent (I'll leave it to better minds to comment on that).

This approach is to use the following Z test:

$Z = \frac{\beta_1-\beta_2}{\sqrt{(SE\beta_1)^2+(SE\beta_2)^2}}$

Where $SE\beta$ is the standard error of $\beta$.

This equation is provided by Clogg, C. C., Petkova, E., & Haritou, A. (1995). Statistical methods for comparing regression coefficients between models. American Journal of Sociology, 100(5), 1261-1293. and is cited by Paternoster, R., Brame, R., Mazerolle, P., & Piquero, A. (1998). Using the correct statistical test for equality of regression coefficients. Criminology, 36(4), 859-866. equation 4, which is available free of a paywall. I've adapted Peternoster's formula to use $\beta$ rather than $b$ because it is possible that you might be interested in different DVs for some awful reason and my memory of Clogg et al. was that their formula used $\beta$. I also remember cross checking this formula against Cohen, Cohen, West, and Aiken, and the root of the same thinking can be found there in the confidence interval of differences between coefficients, equation 2.8.6, pg 46-47.

$\endgroup$
  • $\begingroup$ See also: stats.stackexchange.com/questions/55501/… $\endgroup$ – russellpierce Sep 24 '14 at 19:59
  • $\begingroup$ Awesome answer! A follow-up question: does this also apply to linear combinations of $\beta_1$ from Model 1 and $\beta_2$ from Model 2? Like,$$Z=\frac{A\beta_1-B\beta_2}{\sqrt{(\text{SE}A\beta_1)^2+(\text{SE}B\beta_2)^2}}$$ $\endgroup$ – Sibbs Gambling May 27 '16 at 6:41
  • $\begingroup$ Also I notice the paper discusses the case where one model is nested inside the other, and DV's of two models are the same. What if these two conditions are not met? Instead, I have design matrices of the two models are the same, but they have different DV's. Does this formula still apply? Thanks a lot! $\endgroup$ – Sibbs Gambling May 27 '16 at 9:59
  • $\begingroup$ @SibbsGambling: You might want to make that a question in its own right to draw more attention. $\endgroup$ – russellpierce Jun 2 '16 at 13:49
10
$\begingroup$

For people with a similar question, let me provide a simple outline of the answer.

The trick is to set up the two equations as a system of seemingly unrelated equations and to estimate them jointly. That is, we stack $y_1$ and $y_2$ on top of each other, and doing more or less the same with the design matrix. That is, the system to be estimated is:

$\left(\array{y_1 \\ y_2}\right) = \left(\array{X_1 \ \ 0 \\ 0 \ \ X_2}\right)\left(\array{\beta_1 \\ \beta_2 }\right) + \left(\array{e_1 \\ e_2 }\right) $

This will lead to a variance-covariance matrix that allows to test for equality of the two coefficients.

$\endgroup$
  • 11
    $\begingroup$ I implemented the way you suggested and compared it with the way above. I found the key difference is whether the assumption that the error variance is the same or not. Your way assumes that the error variance is the same and the way above doesn't assume it. $\endgroup$ – KH Kim Nov 26 '14 at 8:14
  • 2
    $\begingroup$ This worked well for me. In Stata, I did something like: expand =2, generate(indicator); generate y = cond(indicator, y2, y1); regress y i.indicator##c.X, vce(cluster id); Using clustered standard errors accounts for the fact that e1 and e2 are not independent for the same observation after stacking the dataset. $\endgroup$ – wkschwartz Oct 22 '17 at 20:11

protected by Glen_b Jun 24 '17 at 12:43

Thank you for your interest in this question. Because it has attracted low-quality or spam answers that had to be removed, posting an answer now requires 10 reputation on this site (the association bonus does not count).

Would you like to answer one of these unanswered questions instead?

Not the answer you're looking for? Browse other questions tagged or ask your own question.