Testing equality of coefficients from two different regressions

This seems to be a basic issue, but I just realized that I actually don't know how to test equality of coefficients from two different regressions. Can anyone shed some light on this?

More formally, suppose I ran the following two regressions: $$y_1 = X_1\beta_1 + \epsilon_1$$ and $$y_2 = X_2\beta_2 + \epsilon_2$$ where $X_i$ refers to the design matrix of regression $i$, and $\beta_i$ to the vector of coefficients in regression $i$. Note that $X_1$ and $X_2$ are potentially very different, with different dimensions etc. I am interested in for instance whether or not $\hat\beta_{11} \neq \hat\beta_{21}$.

If these came from the same regression, this would be trivial. But since they come from different ones, I am not quite sure how to do it. Does anyone have an idea or can give me some pointers?

My problem in detail: My first intuition was to look at the confidence intervals, and if they overlap, then I would say they are essentially the same. This procedure does not come with the correct size of the test, though (i.e. each individual confidence interval has $\alpha=0.05$, say, but looking at them jointly will not have the same probability). My "second" intuition was to conduct a normal t-test. That is, take

$$\frac{\beta_{11}-\beta_{21}}{sd(\beta_{11})}$$

where $\beta_{21}$ is taken as the value of my null hypothesis. This does not take into account the estimation uncertainty of $\beta_{21}$, though, and the answer may depend on the order of the regressions (which one I call 1 and 2).

My third idea was to do it as in a standard test for equality of two coefficients from the same regression, that is take $$\frac{\beta_{11}-\beta_{21}}{sd(\beta_{11}-\beta_{21})}$$

The complication arises due to the fact that both come from different regressions. Note that

$$Var(\beta_{11}-\beta_{21}) = Var(\beta_{11}) + Var(\beta_{21}) -2 Cov(\beta_{11},\beta_{21})$$ but since they are from different regressions, how would I get $Cov(\beta_{11},\beta_{21})$?

This led me to ask this question here. This must be a standard procedure / standard test, but I cound not find anything that was sufficiently similar to this problem. So, if anyone can point me to the correct procedure, I would be very grateful!

• This seems to relate to structural/simultanous equation modeling. One way of solving this problem is fitting both equations simultanously, e.g. with maximum likelihood, and then use a likelihood ratio test of a constrained (equal parameter model) against an unconstrained model. Practically this can be done with SEM software (Mplus, lavaan etc.) Commented Apr 12, 2014 at 13:11
• Do you know about Seemingly Unrelated Regression (SUR)? Commented Apr 12, 2014 at 16:01
• I think the question your raise, i.e. how to get the cov of both coefficients, is solved by SEM, which would give you the var-cov matrix of all coefficients. Then you could possibly use a Wald test in the way you suggested instead of a LRT test. Furthermore you could also use re-sampling / bootstrap, which may be more direct. Commented Apr 12, 2014 at 16:44
• Yes, you are right about that, @tomka. In a SUR model (which you can loosely speaking consider a special case of SEM models), I can get the appropriate test. Thanks for pointing me in that direction! I think I did not think about it because it seems a little bit like shooting a sparrow with a cannon, but I can indeed not think of a better way. If you write up an answer, I will mark it as correct. Otherwise, I will write it up myself soon, with a quick theoretical explanation and potentially with an example. Commented Apr 12, 2014 at 18:25
• SUR is pretty easy to implement. Here's one example with Stata. With R, you want systemfit. Commented Apr 12, 2014 at 19:24

Although this isn't a common analysis, it really is one of interest. I'm going to provide a reasonably well accepted technique that may or may not be equivalent (I'll leave it to better minds to comment on that).

This approach is to use the following Z test:

$$Z = \frac{\beta_1-\beta_2}{\sqrt{(SE\beta_1)^2+(SE\beta_2)^2}}$$

Where $$SE\beta$$ is the standard error of $$\beta$$.

This equation is provided by Clogg, C. C., Petkova, E., & Haritou, A. (1995). Statistical methods for comparing regression coefficients between models. American Journal of Sociology, 100(5), 1261-1293. and is cited by Paternoster, R., Brame, R., Mazerolle, P., & Piquero, A. (1998). Using the correct statistical test for equality of regression coefficients. Criminology, 36(4), 859-866. equation 4, which is available free of a paywall. I've adapted Peternoster's formula to use $$\beta$$ rather than $$b$$ because it is possible that you might be interested in different DVs for some awful reason and my memory of Clogg et al. was that their formula used $$\beta$$. I also remember cross checking this formula against Cohen, Cohen, West, and Aiken, and the root of the same thinking can be found there in the confidence interval of differences between coefficients, equation 2.8.6, pg 46-47.

• Commented Sep 24, 2014 at 19:59
• Awesome answer! A follow-up question: does this also apply to linear combinations of $\beta_1$ from Model 1 and $\beta_2$ from Model 2? Like,$$Z=\frac{A\beta_1-B\beta_2}{\sqrt{(\text{SE}A\beta_1)^2+(\text{SE}B\beta_2)^2}}$$ Commented May 27, 2016 at 6:41
• Also I notice the paper discusses the case where one model is nested inside the other, and DV's of two models are the same. What if these two conditions are not met? Instead, I have design matrices of the two models are the same, but they have different DV's. Does this formula still apply? Thanks a lot! Commented May 27, 2016 at 9:59
• @SibbsGambling: You might want to make that a question in its own right to draw more attention. Commented Jun 2, 2016 at 13:49
• On a quick glance, this looks like a special case of the SUR solution hinted at in the answer by coffeinjunky. It is a special case because the covariance between the estimators of $\beta_1$ and $\beta_2$ is implicitly assumed to be zero. I wonder if it is generally justifiable. To be safe, I would go for the more general solution by coffeinjunky instead. Which leaves me wondering why this is the accepted answer with clearly the most votes. Commented Oct 17, 2019 at 12:57

For people with a similar question, let me provide a simple outline of the answer.

The trick is to set up the two equations as a system of seemingly unrelated equations and to estimate them jointly. That is, we stack $y_1$ and $y_2$ on top of each other, and doing more or less the same with the design matrix. That is, the system to be estimated is:

$\left(\array{y_1 \\ y_2}\right) = \left(\array{X_1 \ \ 0 \\ 0 \ \ X_2}\right)\left(\array{\beta_1 \\ \beta_2 }\right) + \left(\array{e_1 \\ e_2 }\right)$

This will lead to a variance-covariance matrix that allows to test for equality of the two coefficients.

• I implemented the way you suggested and compared it with the way above. I found the key difference is whether the assumption that the error variance is the same or not. Your way assumes that the error variance is the same and the way above doesn't assume it. Commented Nov 26, 2014 at 8:14
• This worked well for me. In Stata, I did something like: expand =2, generate(indicator); generate y = cond(indicator, y2, y1); regress y i.indicator##c.X, vce(cluster id); Using clustered standard errors accounts for the fact that e1 and e2 are not independent for the same observation after stacking the dataset. Commented Oct 22, 2017 at 20:11
• @KHKim You can get around that assumption by using a robust covariance estimator. Commented Jul 25, 2022 at 19:18
• When the regressions come from two different samples, you can assume: $$Var(\beta_1-\beta_2)=Var(\beta_1)+Var(\beta_2)$$ which leads to the formula provided in another answer.

• But your question was precisely related to the case when $$covar(\beta_1,\beta_2) \neq 0$$. In this case, seemingly unrelated equations seems the most general case. Yet it will provide different coefficients from the ones from the original equations, which may not be what you are looking for.

• (Clogg, C. C., Petkova, E., & Haritou, A. (1995). Statistical methods for comparing regression coefficients between models. American Journal of Sociology, 100(5), 1261-1293.) presents an answer in the special case of nested equations (ie. to get the second equation, consider the first equation and add a few explanatory variables) They say it is easy to implement.

• If I well understand it, in this special case, a Haussman test can also be implemented. The key difference is that their test considers as true the second (full) equation, while the Haussman test considers as true the first equation.

• Note that Clogg et al (1995) is not suited for panel data. But their test has been generalized by (Yan, J., Aseltine Jr, R. H., & Harel, O. (2013). Comparing regression coefficients between nested linear models for clustered data with generalized estimating equations. Journal of Educational and Behavioral Statistics, 38(2), 172-189.) with a package provided in R: geepack See: https://www.jstor.org/stable/pdf/41999419.pdf?refreqid=excelsior%3Aa0a3b20f2bc68223edb59e3254c234be&seq=1

And (for the R-package): https://cran.r-project.org/web/packages/geepack/index.html

Using some data, here is how you could use the Clifford Clogg et al. (1995) paper cited by Ray Paternoster et al. (1998). I have a small script, which can be improved to do that.

This assumes that you are using the R language and that you have two sets of regression coefficients that you have extracted from your model into two dataframes, like below. I have truncated the outputs to only those germane to this illustration:

df1 = model1$$coefficients df2 = model2$$coefficients

df1 = data.frame(
estimate = c(15.2418519, 2.2215987, 0.3889724, 0.5289710),
std.error = c(1.0958919, 0.2487793, 0.1973446, 0.1639074),
row.names = c('(Intercept)', 'psychoticism', 'extraversion', 'neuroticism')
); df1

df2 = data.frame(
estimate = c(17.2373874, 0.8350460, -0.3714803, 1.0382513),
std.error = c(1.0987151, 0.2494201, 0.1978530, 0.1643297),
row.names = c('(Intercept)', 'psychoticism', 'extraversion', 'neuroticism')
); df2



The next step is to iterate over the columns and compare the coefficients. The function is also assuming that you are comparing all the coefficients between two models. If that is not the case, you can modify the script as needed.

The calculations in the function are done in a step-by-step manner so it's easy to follow along with the formula provided by Clogg et al. (1995). As well, I have commented it liberally, so it's easy to follow along. Anyway, below is the script

compare_coefs <- function(.data1, .data2){

# imports map_dbl() and pluck() functions from purrr library
import::here(map_dbl, pluck, .from = purrr)

# extract the relevant data
b1 = map_dbl(.data1[-1, 1], pluck)     #get reg. coefs for model 1
se1 = map_dbl(.data1[-1, 2], pluck)    #get std errors for model 1

b2 = map_dbl(.data2[-1, 1], pluck)     #get reg. coefs for model 2
se2 = map_dbl(.data2[-1, 2], pluck)    #get std. errors for model 2

# Clogg et al. (1995) formula as cited by Ray Paternoster et al. (1998)
b = b1 - b2
s1 = se1^2
s2 = se2^2
sc = s1 + s2
v = b / sqrt(sc)

data.frame(diff=b, zdiff=v, p-value=format(2*pnorm(-abs(v)), scientific=FALSE))

}


Note

In this example, I am comparing the effects of personality characteristics on two indicators of criminality. Specifically, I wanted to investigate if the effects (the regression coefficients) of personality characteristics are the same across those two indicators.