Testing equality of coefficients from two different regressions

Question

This seems to be a basic issue, but I just realized that I actually don't know how to test equality of coefficients from two different regressions. Can anyone shed some light on this?

More formally, suppose I ran the following two regressions: $$ y_1 = X_1\beta_1 + \epsilon_1 $$ and $$ y_2 = X_2\beta_2 + \epsilon_2 $$ where $X_i$ refers to the design matrix of regression $i$, and $\beta_i$ to the vector of coefficients in regression $i$. Note that $X_1$ and $X_2$ are potentially very different, with different dimensions etc. I am interested in for instance whether or not $\hat\beta_{11} \neq \hat\beta_{21}$.

If these came from the same regression, this would be trivial. But since they come from different ones, I am not quite sure how to do it. Does anyone have an idea or can give me some pointers?

My problem in detail: My first intuition was to look at the confidence intervals, and if they overlap, then I would say they are essentially the same. This procedure does not come with the correct size of the test, though (i.e. each individual confidence interval has $\alpha=0.05$, say, but looking at them jointly will not have the same probability). My "second" intuition was to conduct a normal t-test. That is, take

$$ \frac{\beta_{11}-\beta_{21}}{sd(\beta_{11})} $$

where $\beta_{21}$ is taken as the value of my null hypothesis. This does not take into account the estimation uncertainty of $\beta_{21}$, though, and the answer may depend on the order of the regressions (which one I call 1 and 2).

My third idea was to do it as in a standard test for equality of two coefficients from the same regression, that is take $$ \frac{\beta_{11}-\beta_{21}}{sd(\beta_{11}-\beta_{21})} $$

The complication arises due to the fact that both come from different regressions. Note that

$$ Var(\beta_{11}-\beta_{21}) = Var(\beta_{11}) + Var(\beta_{21}) -2 Cov(\beta_{11},\beta_{21}) $$ but since they are from different regressions, how would I get $Cov(\beta_{11},\beta_{21})$?

This led me to ask this question here. This must be a standard procedure / standard test, but I cound not find anything that was sufficiently similar to this problem. So, if anyone can point me to the correct procedure, I would be very grateful!

Answer 1

Although this isn't a common analysis, it really is one of interest. I'm going to provide a reasonably well accepted technique that may or may not be equivalent (I'll leave it to better minds to comment on that).

This approach is to use the following Z test:

$$Z = \frac{\beta_1-\beta_2}{\sqrt{(SE\beta_1)^2+(SE\beta_2)^2}}$$

Where $SE\beta$ is the standard error of $\beta$.

This equation is provided by Clogg, C. C., Petkova, E., & Haritou, A. (1995). Statistical methods for comparing regression coefficients between models. American Journal of Sociology, 100(5), 1261-1293. and is cited by Paternoster, R., Brame, R., Mazerolle, P., & Piquero, A. (1998). Using the correct statistical test for equality of regression coefficients. Criminology, 36(4), 859-866. equation 4, which is available free of a paywall. I've adapted Peternoster's formula to use $\beta$ rather than $b$ because it is possible that you might be interested in different DVs for some awful reason and my memory of Clogg et al. was that their formula used $\beta$. I also remember cross checking this formula against Cohen, Cohen, West, and Aiken, and the root of the same thinking can be found there in the confidence interval of differences between coefficients, equation 2.8.6, pg 46-47.

Answer 2

For people with a similar question, let me provide a simple outline of the answer.

The trick is to set up the two equations as a system of seemingly unrelated equations and to estimate them jointly. That is, we stack $y_1$ and $y_2$ on top of each other, and doing more or less the same with the design matrix. That is, the system to be estimated is:

$\left(\array{y_1 \\ y_2}\right) = \left(\array{X_1 \ \ 0 \\ 0 \ \ X_2}\right)\left(\array{\beta_1 \\ \beta_2 }\right) + \left(\array{e_1 \\ e_2 }\right) $

This will lead to a variance-covariance matrix that allows to test for equality of the two coefficients.

Answer 3

• When the regressions come from two different samples, you can assume: $Var(\beta_1-\beta_2)=Var(\beta_1)+Var(\beta_2)$ which leads to the formula provided in another answer.

• But your question was precisely related to the case when $covar(\beta_1,\beta_2) \neq 0$. In this case, seemingly unrelated equations seems the most general case. Yet it will provide different coefficients from the ones from the original equations, which may not be what you are looking for.

• (Clogg, C. C., Petkova, E., & Haritou, A. (1995). Statistical methods for comparing regression coefficients between models. American Journal of Sociology, 100(5), 1261-1293.) presents an answer in the special case of nested equations (ie. to get the second equation, consider the first equation and add a few explanatory variables) They say it is easy to implement.

• If I well understand it, in this special case, a Haussman test can also be implemented. The key difference is that their test considers as true the second (full) equation, while the Haussman test considers as true the first equation.

• Note that Clogg et al (1995) is not suited for panel data. But their test has been generalized by (Yan, J., Aseltine Jr, R. H., & Harel, O. (2013). Comparing regression coefficients between nested linear models for clustered data with generalized estimating equations. Journal of Educational and Behavioral Statistics, 38(2), 172-189.) with a package provided in R: geepack See: https://www.jstor.org/stable/pdf/41999419.pdf?refreqid=excelsior%3Aa0a3b20f2bc68223edb59e3254c234be&seq=1

And (for the R-package): https://cran.r-project.org/web/packages/geepack/index.html

Answer 4

Using some data, here is how you could use the Clifford Clogg et al. (1995) paper cited by Ray Paternoster et al. (1998). I have a small script, which can be improved to do that.

This assumes that you are using the R language and that you have two sets of regression coefficients that you have extracted from your model into two dataframes, like below. I have truncated the outputs to only those germane to this illustration:

df1 = model1$coefficients
df2 = model2$coefficients

df1 = data.frame(
  estimate = c(15.2418519, 2.2215987, 0.3889724, 0.5289710),
  std.error = c(1.0958919, 0.2487793, 0.1973446, 0.1639074),
  row.names = c('(Intercept)', 'psychoticism', 'extraversion', 'neuroticism')
  ); df1

df2 = data.frame(
  estimate = c(17.2373874, 0.8350460, -0.3714803, 1.0382513),
  std.error = c(1.0987151, 0.2494201, 0.1978530, 0.1643297),
  row.names = c('(Intercept)', 'psychoticism', 'extraversion', 'neuroticism')
); df2

The next step is to iterate over the columns and compare the coefficients. The function is also assuming that you are comparing all the coefficients between two models. If that is not the case, you can modify the script as needed.

The calculations in the function are done in a step-by-step manner so it's easy to follow along with the formula provided by Clogg et al. (1995). As well, I have commented it liberally, so it's easy to follow along. Anyway, below is the script

compare_coefs <- function(.data1, .data2){
  
  # imports map_dbl() and pluck() functions from purrr library
  import::here(map_dbl, pluck, .from = purrr)  
  
  # extract the relevant data
  b1 = map_dbl(.data1[-1, 1], pluck)     #get reg. coefs for model 1
  se1 = map_dbl(.data1[-1, 2], pluck)    #get std errors for model 1
  
  b2 = map_dbl(.data2[-1, 1], pluck)     #get reg. coefs for model 2
  se2 = map_dbl(.data2[-1, 2], pluck)    #get std. errors for model 2
  
  # Clogg et al. (1995) formula as cited by Ray Paternoster et al. (1998)
  b = b1 - b2
  s1 = se1^2
  s2 = se2^2
  sc = s1 + s2
  v = b / sqrt(sc)
  
  data.frame(diff=b, zdiff=v, `p-value`=format(2*pnorm(-abs(v)), scientific=FALSE))
  
  }

Note

In this example, I am comparing the effects of personality characteristics on two indicators of criminality. Specifically, I wanted to investigate if the effects (the regression coefficients) of personality characteristics are the same across those two indicators.