# Combining regression estimates by summing

I want to know if one can combine regression estimates from panel regressions when the new dependent variable is a sum of the dependent variables from previously estimated regressions.

To be concrete:

Regression 1:

$$Y_{it}^1 = \alpha_i^1 + \gamma_t^1 + X_{it}\beta_1 + \epsilon_{it}^1$$

Regression 2:

$$Y_{it}^2 = \alpha_i^2 + \gamma_t^2 + X_{it}\beta_2 + \epsilon_{it}^2$$

Regression 3:

$$Y_{it}^{sum} = \alpha_i^3 + \gamma_t^3 + X_{it}\beta_3 + \epsilon_{it}^3$$

where $$Y_{it}^{sum} = Y_{it}^{1} + Y_{it}^{2}$$ and $$\alpha_i$$ and $$\gamma_t$$ are person and time-fixed effects.

So, my question is: Is there a connection between $$\beta_3$$ and $$\beta_1$$, $$\beta_2$$?

By definition of linear regression we should have $$Y_1= \mathcal{N}(X\beta_1,\sigma_1^2)$$, $$Y_2= \mathcal{N}(X\beta_2,\sigma_2^2)$$, and hence $$Y_{sum}= \mathcal{N}(X\beta_1+ X\beta_2,\sigma_1^2+\sigma_2^2)$$.

So, then should we have $$\beta_3=\beta_1+ \beta_2$$? The empirical results don't agree with that. What am I missing? Do person and time fixed effects complicate things?

Also, how about the case when it is a poisson regression instead of a linear regression?

Any help is greatly appreciated!

• At the moment you have posed your two initial models so that they have the same coefficients (except for the one on the explanatory variable) and they even have the same error terms! I am guessing that this is not what you intended. So, to clarify, do you intend for your two initial models to have the same parameters $\alpha_i$ and $\gamma_t$, and do you intend them to have the same error terms $\epsilon_{it}$?
– Ben
Jul 14, 2019 at 6:13
• Yes. You're right. I thought that was implied. The error terms and the $\alpha_i$, $\gamma_t$ are different for the 3 set of equations. Jul 14, 2019 at 12:25
• Most (all?) of the answers to this question so far are assuming that $\epsilon_{it}^1$ and $\epsilon_{it}^2$ are uncorrelated, but it seems to me that may be a stretch in this context. I'd be interested in seeing an answer that addresses this Jul 15, 2019 at 16:55
• As I think I (now) provide appropriate answers to all the above questions, and as I am quite interested in the bounty, please do not hesitate to say if something remains unclear. ;) Jul 16, 2019 at 20:51

Based on your clarification in the comments, and your amended notation, you want to use a Gaussian linear model where the parameters and error terms in each of the initial regressions are different. You therefore have the model:

$$\begin{matrix} Y_{it}^{(1)} = \alpha_i^{(1)} + \gamma_t^{(1)} + X_{it}\beta^{(1)} + \epsilon_{it}^{(1)} & & & \epsilon_{it}^{(1)} \sim \text{N}(0, \sigma_1^2), \\[6pt] Y_{it}^{(2)} = \alpha_i^{(2)} + \gamma_t^{(2)} + X_{it}\beta^{(2)} + \epsilon_{it}^{(2)} & & & \epsilon_{it}^{(2)} \sim \text{N}(0, \sigma_1^2). \\[6pt] \end{matrix}$$

Taking the sum of the response variables gives the linear regression model:

\begin{aligned} Y_{it}^\text{sum} &\equiv Y_{it}^{(1)} + Y_{it}^{(2)} \\[6pt] &= \alpha_i^{(1)} + \gamma_t^{(1)} + X_{it} \beta^{(1)} + \epsilon_{it}^{(1)} + \alpha_i^{(2)} + \gamma_t^{(2)} + X_{it} \beta^{(2)} + \epsilon_{it}^{(2)} \\[6pt] &= (\alpha_i^{(1)} + \alpha_i^{(2)}) + (\gamma_t^{(1)} + \gamma_t^{(2)}) + X_{it} (\beta^{(1)} + \beta^{(2)}) + (\epsilon_{it}^{(1)} + \epsilon_{it}^{(2)}) \\[6pt] &= \alpha_i^\text{sum} + \gamma_t^\text{sum} + X_{it} \beta^\text{sum} + \epsilon_{it}^\text{sum}, \\[6pt] \end{aligned}

where we have the parameters and error terms given by:

$$\begin{matrix} \alpha_i^\text{sum} = \alpha_i^{(1)} + \alpha_i^{(2)} & & & \gamma_t^\text{sum} + \gamma_t^{(2)} = \gamma_t^{(1)} + \gamma_t^{(2)} \\[6pt] \beta_\text{sum} = \beta_1 + \beta_2 \ \quad & & & \epsilon_{it}^\text{sum} \sim \text{N}(0,\sigma_1^2+\sigma_2^2). \\[6pt] \end{matrix}$$

This shows you that the true parameters in the regression model for the summed response are equal to the sums of the parameters in the individual regressions. Note that this does not imply that the parameters estimates will obey this relationship, and in general, they will not. If you use OLS estimation to estimate each of the underlying parameters, and you do the same for the summed model, you will find that the the sum of the estimated parameters for the smaller models will not be equal to the estimated parameter for the summed model.

• Thanks! This makes sense. However, a part of the question was regarding the Poisson model i.e. would such a summative relation hold for Poisson model also, if not what can we say about the relation between the coefficients. Jul 14, 2019 at 13:51

$$Y_1 \sim \mathcal{N}(X\beta_1,\sigma_1^2)$$ ==> $$Y_1=X\beta_1+\epsilon_1$$ and $$\epsilon_1\sim \mathcal{N}(0,\sigma_1^2)$$

$$Y_2 \sim \mathcal{N}(X\beta_2,\sigma_2^2)$$ ==> $$Y_2=X\beta_2+\epsilon_2$$ and $$\epsilon_2\sim \mathcal{N}(0,\sigma_2^2)$$

Assume $$\epsilon_1$$ and $$\epsilon_2$$ are independent, then

$$Y_1 + Y_2 \sim \mathcal{N}(X(\beta_1+\beta_2),\sigma_1^2 + \sigma_2^2)$$ or $$Y_1+ Y_2=X(\beta_1+\beta_2)+\epsilon_3$$ and $$\epsilon_3 = \epsilon_1 + \epsilon_2 \sim \mathcal{N}(0,\sigma_1^2+\sigma_2^2)$$

Now let check the estimate.

$$\hat \beta_1 = (X'X)^{-1}Y_1$$

$$\hat \beta_2 = (X'X)^{-1}Y_2$$

$$\widehat {\beta_1 + \beta_2}= (X'X)^{-1}(Y_1+Y_2) = (X'X)^{-1}Y_1 + (X'X)^{-1}Y_2 = \hat \beta_1 + \hat \beta_2$$

About "The empirical results don't agree with that", the real data and model are needed for further discussion.

This conclusion is not correct for Poisson regression, because log link function is used in Poisson regression.

• This shows the empirical estimates should actually be the same - the data generating process is irrelevant. The OLS formulas have this decomposition, as you show. There must be something else going on in the analysis Blade Runner carries out Jul 15, 2019 at 8:45

1. In the described theoretical setting, you are right to expect $$\beta_3 = \beta_1+\beta_2$$ in the linear setting. The time- and individual- FE does not change it.
2. In the linear setting, $$\beta_3$$ can differ from $$\beta_1+\beta_2$$ for instance due to data limitations. (please look at the bottom of the posted answer)
3. In a Poisson setting, there is no reason that the first two coefficients would sum to the third one. This is also not due to the time- and individual- FE.

1 & 3/**Note that on this dataset (without any missig value) I get that $$\beta_3$$ is **exactly the sum of $$\beta1+\beta_2$$ when using linear fixed-effects equations, but are far from being equal when using fixed-effect Poisson equations.

• Fixed-effects are a distraction here. Consider here they are just dummy variables. I rewrite the linear equations: $$(Reg1): Y_1 = X\beta_1 + \epsilon_1$$ $$(Reg2): Y_2 = X\beta_2 + \epsilon_2$$ $$(Reg3): Y_3 = X\beta_3 + \epsilon_3$$ There are explicit solutions/estimates for the various $$\beta$$: $$\beta_1 = (X'X)^{-1}X'Y_1$$ $$\beta_2 = (X'X)^{-1}X'Y_2$$ $$\beta_3 = (X'X)^{-1}X'Y_3$$ One indeed gets: $$\beta_3 = \beta_1+\beta_2$$ when using $$Y_3=Y_1+Y_2$$

No assumption is needed regarding independence between $$\epsilon_1$$ and $$\epsilon_2$$.

Save random numerical approximations when computing inverse of the matrix (X'X), the equality should perfectly hold in the linear case (just try it).

• What about the Poisson case ? Here, your model writes (I index your explanatory variables with letters): $$E(Y_i|X) = e^{X\beta_i}=e^{x_a\beta_{i,a}+...+x_p\beta_{i,p}}$$ How to interpret the $$\beta$$ coefficients ? A increase by $$dx_a$$ will turn:

• $$E(Y_1|X)$$ into $$e^{dx_a \times \beta_{1,a}} E(Y_1|X)$$

• $$E(Y_2|X)$$ into $$e^{dx_a \times \beta_{2,a}} E(Y_2|X)$$
• $$E(Y_3|X)$$ into $$e^{dx_a \times \beta_{3,a}} E(Y_3|X)$$

In order to have $$Y_3=Y_1+Y_2$$, you expect: $$(1): e^{dx_a \times \beta_{3,a}} (E(Y_1|X)+E(Y_2|X)) = e^{dx_a \times \beta_{1,a}} E(Y_1|X) +e^{dx_a \times \beta_{2,a}} E(Y_2|X)$$

I do not know a more general relationship between $$\beta_1$$ and $$\beta_2$$ but this clearly shows that in general you do not have: $$\beta_{3,a} = \beta_{1,a}+\beta_{2,a}$$ .

(To do so, consider the trivial case where $$\beta_{1,a}=\beta_{2,a}$$. In this case, equation (1) becomes: $$(1_{trivial}): e^{dx_a \times \beta_{3,a}} (E(Y_1|X)+E(Y_2|X)) = e^{dx_a \times \beta_{1,a}} (E(Y_1|X)+E(Y_2|X))$$ Save $$E(Y_1|X)+E(Y_2|X)=0$$, this implies: $$\beta_3 = \beta_1 =\beta_2$$. Hence, when it is different from zero, it is different from $$\beta_1+\beta_2$$ )

2/ But there are probably some missing values in your dataset. More precisely, perhaps some observations are missing for some individual $$i$$ (or time $$t$$) for $$Y_{1,i,t}$$, while some other observations are missing for some individual $$j$$ (or time $$u$$) for $Y_{1,j,u}. Hence, your are: • Estimating $$\beta_1$$ on the individuals for which $$Y_1$$ is not lacking. • Estimating $$\beta_2$$ on the individuals for which $$Y_2$$ is not lacking. • Estimating $$\beta_3$$ on the individuals for which $$Y_1$$ and $$Y_2$$ are not lacking. This implies you are comparing estimates fom three different datasets. In this case, it is possible that you do not have $$\beta_3 = \beta_1+\beta_2$$, even in the linear setting. • Some silly mistakes withdrawn. Jul 15, 2019 at 8:31 • I think I got it, please see 2/. @CloseToC thanks for the tip provided in another answer ! Jul 15, 2019 at 16:41 • @Blade Runner: to chek my hypothesis, you can restrict your three estimations on the subsample for which both$Y_1$and$Y_2\$ are non-missing. Jul 15, 2019 at 18:38

I'm no expert, so interpret with caution. Isn't the model that you fit for the sum supposed to be $$Y^{sum} = 2 \alpha_i + 2\gamma_t + \beta_3 X + 2\epsilon_{it},$$ thus changing what you should expect. This assumes that your parameters $$\alpha_i$$ and $$\gamma_t$$ are the same for the two models, which I think is what you meant. Correct me if I'm wrong.
Also, concerning the claim: $$\mathbb{Var}(Y^1 + Y^2) = \mathbb{Var}(Y^1) + \mathbb{Var}(Y^2) = \sigma_1^2 + \sigma_2^2$$. It seems to me that you rather have the following: $$\mathbb{Var}(Y^1 + Y^2) = \sigma_1^2 + \sigma_2^2 + 2 \mathbb{Cov}(Y^1,Y^2)$$ where $$\mathbb{Cov}(Y^1,Y^2) \neq 0$$, since both variables depend on $$X$$.