Combining regression estimates by summing

Question

I want to know if one can combine regression estimates from panel regressions when the new dependent variable is a sum of the dependent variables from previously estimated regressions.

To be concrete:

Regression 1:

$Y_{it}^1 = \alpha_i^1 + \gamma_t^1 + X_{it}\beta_1 + \epsilon_{it}^1$

Regression 2:

$Y_{it}^2 = \alpha_i^2 + \gamma_t^2 + X_{it}\beta_2 + \epsilon_{it}^2$

Regression 3:

$Y_{it}^{sum} = \alpha_i^3 + \gamma_t^3 + X_{it}\beta_3 + \epsilon_{it}^3$

where $Y_{it}^{sum} = Y_{it}^{1} + Y_{it}^{2}$ and $\alpha_i$ and $\gamma_t$ are person and time-fixed effects.

So, my question is: Is there a connection between $\beta_3$ and $\beta_1$, $\beta_2$?

By definition of linear regression we should have $Y_1= \mathcal{N}(X\beta_1,\sigma_1^2)$, $Y_2= \mathcal{N}(X\beta_2,\sigma_2^2)$, and hence $Y_{sum}= \mathcal{N}(X\beta_1+ X\beta_2,\sigma_1^2+\sigma_2^2)$.

So, then should we have $\beta_3=\beta_1+ \beta_2$? The empirical results don't agree with that. What am I missing? Do person and time fixed effects complicate things?

Also, how about the case when it is a poisson regression instead of a linear regression?

Any help is greatly appreciated!

Answer 1

Based on your clarification in the comments, and your amended notation, you want to use a Gaussian linear model where the parameters and error terms in each of the initial regressions are different. You therefore have the model:

$$\begin{matrix} Y_{it}^{(1)} = \alpha_i^{(1)} + \gamma_t^{(1)} + X_{it}\beta^{(1)} + \epsilon_{it}^{(1)} & & & \epsilon_{it}^{(1)} \sim \text{N}(0, \sigma_1^2), \\[6pt] Y_{it}^{(2)} = \alpha_i^{(2)} + \gamma_t^{(2)} + X_{it}\beta^{(2)} + \epsilon_{it}^{(2)} & & & \epsilon_{it}^{(2)} \sim \text{N}(0, \sigma_1^2). \\[6pt] \end{matrix}$$

Taking the sum of the response variables gives the linear regression model:

$$\begin{equation} \begin{aligned} Y_{it}^\text{sum} &\equiv Y_{it}^{(1)} + Y_{it}^{(2)} \\[6pt] &= \alpha_i^{(1)} + \gamma_t^{(1)} + X_{it} \beta^{(1)} + \epsilon_{it}^{(1)} + \alpha_i^{(2)} + \gamma_t^{(2)} + X_{it} \beta^{(2)} + \epsilon_{it}^{(2)} \\[6pt] &= (\alpha_i^{(1)} + \alpha_i^{(2)}) + (\gamma_t^{(1)} + \gamma_t^{(2)}) + X_{it} (\beta^{(1)} + \beta^{(2)}) + (\epsilon_{it}^{(1)} + \epsilon_{it}^{(2)}) \\[6pt] &= \alpha_i^\text{sum} + \gamma_t^\text{sum} + X_{it} \beta^\text{sum} + \epsilon_{it}^\text{sum}, \\[6pt] \end{aligned} \end{equation}$$

where we have the parameters and error terms given by:

$$\begin{matrix} \alpha_i^\text{sum} = \alpha_i^{(1)} + \alpha_i^{(2)} & & & \gamma_t^\text{sum} + \gamma_t^{(2)} = \gamma_t^{(1)} + \gamma_t^{(2)} \\[6pt] \beta_\text{sum} = \beta_1 + \beta_2 \ \quad & & & \epsilon_{it}^\text{sum} \sim \text{N}(0,\sigma_1^2+\sigma_2^2). \\[6pt] \end{matrix}$$

This shows you that the true parameters in the regression model for the summed response are equal to the sums of the parameters in the individual regressions. Note that this does not imply that the parameters estimates will obey this relationship, and in general, they will not. If you use OLS estimation to estimate each of the underlying parameters, and you do the same for the summed model, you will find that the the sum of the estimated parameters for the smaller models will not be equal to the estimated parameter for the summed model.

Answer 2

$Y_1 \sim \mathcal{N}(X\beta_1,\sigma_1^2)$ ==> $Y_1=X\beta_1+\epsilon_1$ and $\epsilon_1\sim \mathcal{N}(0,\sigma_1^2) $

$Y_2 \sim \mathcal{N}(X\beta_2,\sigma_2^2)$ ==> $Y_2=X\beta_2+\epsilon_2$ and $\epsilon_2\sim \mathcal{N}(0,\sigma_2^2) $

Assume $\epsilon_1$ and $\epsilon_2$ are independent, then

$Y_1 + Y_2 \sim \mathcal{N}(X(\beta_1+\beta_2),\sigma_1^2 + \sigma_2^2)$ or $Y_1+ Y_2=X(\beta_1+\beta_2)+\epsilon_3$ and $\epsilon_3 = \epsilon_1 + \epsilon_2 \sim \mathcal{N}(0,\sigma_1^2+\sigma_2^2) $

Now let check the estimate.

$\hat \beta_1 = (X'X)^{-1}Y_1$

$\hat \beta_2 = (X'X)^{-1}Y_2$

$\widehat {\beta_1 + \beta_2}= (X'X)^{-1}(Y_1+Y_2) = (X'X)^{-1}Y_1 + (X'X)^{-1}Y_2 = \hat \beta_1 + \hat \beta_2$

About "The empirical results don't agree with that", the real data and model are needed for further discussion.

This conclusion is not correct for Poisson regression, because log link function is used in Poisson regression.

Answer 3

Short answer:

1. In the described theoretical setting, you are right to expect $\beta_3 = \beta_1+\beta_2$ in the linear setting. The time- and individual- FE does not change it.
2. In the linear setting, $\beta_3$ can differ from $\beta_1+\beta_2$ for instance due to data limitations. (please look at the bottom of the posted answer)
3. In a Poisson setting, there is no reason that the first two coefficients would sum to the third one. This is also not due to the time- and individual- FE.

Longer answer:

• Some (Stata) code showing it for illustrative purposes is available here: https://sites.google.com/view/acazenave-lacroutz/stack_ansa

1 & 3/**Note that on this dataset (without any missig value) I get that $\beta_3$ is **exactly the sum of $\beta1+\beta_2$ when using linear fixed-effects equations, but are far from being equal when using fixed-effect Poisson equations.

• Fixed-effects are a distraction here. Consider here they are just dummy variables. I rewrite the linear equations: $$(Reg1): Y_1 = X\beta_1 + \epsilon_1 $$ $$(Reg2): Y_2 = X\beta_2 + \epsilon_2 $$ $$(Reg3): Y_3 = X\beta_3 + \epsilon_3 $$ There are explicit solutions/estimates for the various $\beta$: $$\beta_1 = (X'X)^{-1}X'Y_1 $$ $$\beta_2 = (X'X)^{-1}X'Y_2 $$ $$\beta_3 = (X'X)^{-1}X'Y_3 $$ One indeed gets: $\beta_3 = \beta_1+\beta_2$ when using $Y_3=Y_1+Y_2$

No assumption is needed regarding independence between $\epsilon_1$ and $\epsilon_2$.

Save random numerical approximations when computing inverse of the matrix (X'X), the equality should perfectly hold in the linear case (just try it).

• What about the Poisson case ? Here, your model writes (I index your explanatory variables with letters): $$E(Y_i|X) = e^{X\beta_i}=e^{x_a\beta_{i,a}+...+x_p\beta_{i,p}} $$ How to interpret the $\beta$ coefficients ? A increase by $dx_a$ will turn:

• $E(Y_1|X)$ into $e^{dx_a \times \beta_{1,a}} E(Y_1|X)$

• $E(Y_2|X)$ into $e^{dx_a \times \beta_{2,a}} E(Y_2|X)$
• $E(Y_3|X)$ into $e^{dx_a \times \beta_{3,a}} E(Y_3|X)$

In order to have $Y_3=Y_1+Y_2$, you expect: $$(1): e^{dx_a \times \beta_{3,a}} (E(Y_1|X)+E(Y_2|X)) = e^{dx_a \times \beta_{1,a}} E(Y_1|X) +e^{dx_a \times \beta_{2,a}} E(Y_2|X)$$

I do not know a more general relationship between $\beta_1$ and $\beta_2$ but this clearly shows that in general you do not have: $\beta_{3,a} = \beta_{1,a}+\beta_{2,a}$ .

(To do so, consider the trivial case where $\beta_{1,a}=\beta_{2,a}$. In this case, equation (1) becomes: $$(1_{trivial}): e^{dx_a \times \beta_{3,a}} (E(Y_1|X)+E(Y_2|X)) = e^{dx_a \times \beta_{1,a}} (E(Y_1|X)+E(Y_2|X))$$ Save $E(Y_1|X)+E(Y_2|X)=0$, this implies: $\beta_3 = \beta_1 =\beta_2 $. Hence, when it is different from zero, it is different from $ \beta_1+\beta_2$ )

2/ But there are probably some missing values in your dataset. More precisely, perhaps some observations are missing for some individual $i$ (or time $t$) for $Y_{1,i,t}$, while some other observations are missing for some individual $j$ (or time $u$) for $Y_{1,j,u}. Hence, your are:

• Estimating $\beta_1$ on the individuals for which $Y_1$ is not lacking.

• Estimating $\beta_2$ on the individuals for which $Y_2$ is not lacking.

• Estimating $\beta_3$ on the individuals for which $Y_1$ and $Y_2$ are not lacking.

This implies you are comparing estimates fom three different datasets. In this case, it is possible that you do not have $\beta_3 = \beta_1+\beta_2$, even in the linear setting.

Answer 4

**** Edits to the question made this answer obsolete ****

I'm no expert, so interpret with caution. Isn't the model that you fit for the sum supposed to be $$ Y^{sum} = 2 \alpha_i + 2\gamma_t + \beta_3 X + 2\epsilon_{it}, $$ thus changing what you should expect. This assumes that your parameters $\alpha_i$ and $\gamma_t$ are the same for the two models, which I think is what you meant. Correct me if I'm wrong.

Also, concerning the claim: $\mathbb{Var}(Y^1 + Y^2) = \mathbb{Var}(Y^1) + \mathbb{Var}(Y^2) = \sigma_1^2 + \sigma_2^2$. It seems to me that you rather have the following: $$ \mathbb{Var}(Y^1 + Y^2) = \sigma_1^2 + \sigma_2^2 + 2 \mathbb{Cov}(Y^1,Y^2) $$ where $\mathbb{Cov}(Y^1,Y^2) \neq 0$, since both variables depend on $X$.