# Combining Regression Equations

Suppose we are interested in English scores ($E_{ij}$) and Math scores ($M_{ij}$) in students in various classes. The scores are in different scales from each other. We perform two linear regressions. The regression with the English scores has $X$ as the covariates. The regression with the Math scores has $Z$ and $W$ as the covariates. Is there a way to combine the two regressions into one regression? Note that we use GEE linear regression with an exchangeable working variance covariance matrix.

So we have:

$$E(E_{ij}) = \beta_{0}+ \beta_{1}X_{ij}$$ $$E(M_{ij}) = \gamma_{0}+ \gamma_{1} Z_{ij} + \gamma_{2}W_{ij}$$

Could we somehow combine the covariates? For example, would it make sense to consider:

$$E(M_{ij}-E_{ij}) = (\gamma_{0}-\beta_{0})+(\gamma_{1}Z_{ij}-\beta_{1}X_{ij})+ \gamma_{2} W_{ij}$$

• What is the story you are going to tell? If you wish to explain a greater skill in Math than in English then $M-E$ looks good, but if you are interested in overall skill, why not $M+E$? Commented May 9, 2014 at 5:47

$E_{ij}=\beta_0+\beta_1X_{ij}+\epsilon_E$

$M_{ij}=\gamma_0+\gamma_1Z_{ij}+\gamma_2W_{ij}+\epsilon_M$

$M_{ij}-E_{ij}=\gamma_0+\gamma_1Z_{ij}+\gamma_2W_{ij}+\epsilon_M-\beta_0-\beta_1X_{ij}-\epsilon_E$

$(M_{ij}-\gamma_0-\gamma_1Z_{ij}-\gamma_2W_{ij})-(E_{ij}-\beta_0-\beta_1X_{ij})=\epsilon_M-\epsilon_E\sim N(0,\sigma_E^2+\sigma_M^2)$

$\sum(\epsilon_M-\epsilon_E)^2=\sum[(M_{ij}-\gamma_0-\gamma_1Z_{ij}-\gamma_2W_{ij})-(E_{ij}-\beta_0-\beta_1X_{ij})]^2=\sum(M_{ij}-\gamma_0-\gamma_1Z_{ij}-\gamma_2W_{ij})^2+\sum(E_{ij}-\beta_0-\beta_1X_{ij})^2-2\sum(M_{ij}-\gamma_0-\gamma_1Z_{ij}-\gamma_2W_{ij})(E_{ij}-\beta_0-\beta_1X_{ij})$

After the least square estimation we have

$\sum(\epsilon_M-\epsilon_E)^2=\sum(M_{ij}-\hat{M_{ij}})^2+\sum(E_{ij}-\hat{E_{ij}})^2-2\sum(M_{ij}-\hat{M_{ij}})(E_{ij}-\hat{E_{ij}})$

If and only if you have $\epsilon_M$ and $\epsilon_E$ to be independent, which you have $Cov(\epsilon_M,\epsilon_E)=0$, you have what you want.

Otherwise you've got new regression model for the combined result.