# Compare coefficients of two independent variable from two regressions models with the same dependent variable

I have the two following regression: $$y=X_1\beta_1+\varepsilon_1$$ and $$y=X_2\beta_2+\varepsilon_2$$

$x_1$ and $x_2$ are indicator variables and very similar. One expert claim that the two coefficients are the same. I want to know what kind of test I can perform to see if $β_1=β_2$.

• Something like this? $Z=((β_1-β_2)/√(S_(β_1)^2+S_(β_2) ))^2$ – user150086 Feb 21 '17 at 0:03
• Note that you can wrote $S_{\beta_1}^2$ in MathJax. Right-click on this notation, then chose "Show Math As" from the menu, then choose "TeX Commands", and you will see the code. – Michael Hardy Feb 21 '17 at 3:10
• You refer to $X_1$ and also to $x_1.$ Are those two different things? – Michael Hardy Feb 21 '17 at 3:20
• So far I'm the only one who's up-voted this question, although someone else has answered. That tends to be neglected. – Michael Hardy Feb 21 '17 at 4:44
• X1 and x1 are the same. – user150086 Feb 21 '17 at 15:20

If the two predictors are orthogonal, an optimal test is to compare the model

$$y_i \sim \beta_0 + \beta_1 X_1 + \beta_2 X_2$$

with the submodel

$$y_i \sim \beta_0 + \beta_1 (X_1 + X_2)$$

with the likelihood ratio test. The null hypothesis is that $\beta_1 = \beta_2$.

• My main problem is that the two variable are highly correlated and I want to test if they are different and pick one of them in my model. – user150086 Feb 21 '17 at 0:50
• If they're highly correlated, you may just reduce it to a single variable with PCA. There's no need to do what you're asking. – SmallChess Feb 21 '17 at 1:23
• @StudentT, Not sure there's any point in doing PCA on two binary variables. The combination can only take on four values. – whatisleverage Feb 21 '17 at 2:47
• @user150086, if they are that closely related, the difference in effect size probably isn't large, so you will have low power to detect a statistically significant difference between their effects no matter what you do. You could just pick the one with a larger effect size, or the one that optimizes some predictive criteria. – whatisleverage Feb 21 '17 at 2:50
• Yes @MichaelHardy. I assumed we were talking about the normal errors linear model, where you could use a LRT or an F test. – whatisleverage Feb 21 '17 at 4:31

You are analyzing a single model $y=X\beta+\varepsilon$. The regression model will return $y=\beta_0+X_1\beta_1$ with $\beta_0$ being the intercept. To determine the difference between the two models you need to look at both the slope $\beta_1$ and the intercept $\beta_0$. You can use the indicator variable as a regressor by itself and in combination with your other independent variables. If the indicator variable is $d$ it will be $d=0$ for the $X_1$s you include above and $d=1$ for the $X_2$s. The model becomes $y\sim d, X, (X*d)$. The regression will return $y=\beta_0+\beta_1X+\beta_2d+\beta_3Xd$. When $d=0$ the coefficients $\beta_2$ and $\beta_3$ vanish. When $d=1$ the intercept is $(\beta_0+\beta_2)$ and the slope is $(\beta_1+\beta_3)X$. Keep the $\beta$s with significant p-values and you end up with a model that tells exactly what and where is the difference between the two groups. See Ways of comparing linear regression interepts and slopes?.