How does average treatment effect relate to averages of individual treatment effects? [closed]

Question

I have a balanced panel with two groups, A and B, and run a standard fixed effects regression. I want to derive how the $\beta$ coefficients of these two regressions are related: \begin{eqnarray} y_{it} &=& \beta x_{it} & + \alpha_i + \epsilon_{it} \\ y_{it} &=& \mathbb{I}(i=A) \beta_A x_{it} + \mathbb{I}(i=B) \beta_B x_{it} & +\alpha_i + \epsilon_{it} \end{eqnarray}

In my application, the case arises that the estimate of the average treatment effect is very far from the unweighted averages of the interacted treatment effect estimates, i.e. $\hat{\beta} \neq \frac{\hat{\beta}_A + \hat{\beta}_B}{2}$.

So I want to understand how OLS does the "weighting" of the two groups when computing $\hat{\beta}$

What I would ideally like to find is a formula that relates $\beta$, $\beta_i$ and the covariances of my $x_{it}$s.

Answer 1

Let $\tilde{y}_{it}$ and $\tilde{x}_{it}$ denote the group-demeaned variables. Now we run three bivariate regressions: \begin{eqnarray} \tilde{y}_{it} &=& \beta \tilde{x}_{it} & + \epsilon_{it} \\ \tilde{y}_{it} &=& {\beta}_A \tilde{x}_{it} & + \epsilon_{it} & \text{ if } i=A \\ \tilde{y}_{it} &=& {\beta}_B \tilde{x}_{it} & + \epsilon_{it} & \text{ if } i=B \\ \end{eqnarray}

Now write the estimate of each coefficient as a function of expected values: \begin{eqnarray} \hat{\beta}_{A} = \frac{Cov(\tilde{x}_{At},\tilde{y}_{At})}{Var(\tilde{x}_{At})} = \frac{E(\tilde{x}_{At} \times \tilde{y}_{At}) - E(\tilde{x}_{At})E(\tilde{y}_{At})}{ E(\tilde{x}_{At}^2) - E(\tilde{x}_{At})^2} = \frac{E(\tilde{x}_{At} \times \tilde{y}_{At})}{ E(\tilde{x}_{At}^2)} \end{eqnarray}

where the last equality follows by definition because the tilde variables have been de-meaned by group. Proceeding analogously for group and then $\beta$, I can write my overall coefficient as:

\begin{eqnarray} \hat{\beta} =& \left(\frac{E(\tilde{x}_{At}^2)}{E(\tilde{x}_{At}^2) + E(\tilde{x}_{Bt}^2)}\right)\hat{\beta}_A + \left(\frac{E(\tilde{x}_{Bt}^2)}{E(\tilde{x}_{At}^2) + E(\tilde{x}_{Bt}^2)}\right)\hat{\beta}_B \\ = & \left(\frac{Var(\tilde{x}_{At})}{Var(\tilde{x}_{At}) + Var(\tilde{x}_{Bt})}\right)\hat{\beta}_A + \left(\frac{Var(\tilde{x}_{Bt})}{Var(\tilde{x}_{At}) + Var(\tilde{x}_{Bt})}\right)\hat{\beta}_B \end{eqnarray}

Thus, the "average treatment effect" is a weighted average of the individual treatment effects, where the weight is proportional to the variance of the explanatory variable within the group.

Numerical Example

require(dplyr); n = 10^6; set.seed(1)
sim.l <- data.frame(t = 1:n) %>% mutate(
  e.common = rnorm(n),
  x.A = 5 + e.common + rnorm(n),
  x.B = (e.common*4 + rnorm(n))/15,
  y.A = x.A + rnorm(n),
  y.B = x.A + rnorm(n)
)
# most of the variation in x.B is driven by common shock, but var x.B << var x.A
round(cov(sim.l[,-c(1:2)]), digits = 2)

sim.p <- sim.l %>% melt(id.var = 't') %>% mutate(
  var = str_extract(variable,'^(.)'),
  group = str_extract(variable,'(.)$')
) %>% dcast(t + group ~  var, value.var = 'value');
lm(y ~ x:group + group, data=sim.p)
lm(y ~ x + group, data=sim.p)

This reproduces the case where the Average treatment effect (shown in column 2) is almost exactly equal to the treatment effect of group A shown in column 1:

===========================================
             Model 1         Model 2       
-------------------------------------------
(Intercept)       -0.00           -0.46 ***
                  (0.00)          (0.00)   
groupB             5.00 ***        5.46 ***
                  (0.00)          (0.00)   
x:groupA           1.00 ***                
                  (0.00)                   
x:groupB           3.53 ***                
                  (0.00)                   
x                                  1.09 ***
                                  (0.00)   
-------------------------------------------
R^2                0.49            0.41    
Adj. R^2           0.49            0.41    
Num. obs.    2000000         2000000       
RMSE               1.24            1.33    
===========================================
*** p < 0.001, ** p < 0.01, * p < 0.05

Based on the formula derived above, we can exactly reproduce these coefficients:

      x.A    x.B    y.A    y.B
x.A 2.0032 0.2669 2.0034 2.0028
x.B 0.2669 0.0756 0.2670 0.2667
y.A 2.0034 0.2670 3.0050 2.0038
y.B 2.0028 0.2667 2.0038 3.0045

beta_A = 2.0028/2.0032
beta_B = 0.2667/0.0756
beta   = (0.0756*beta_b   + 2.0028*beta_a)/(2.0028 + 0.0756)