STATA - Mean differences between treated and control groups after matching

Question

1. Relevant Files: https://www.dropbox.com/sh/0jnj3txf4stb2q8/AAD58COnAUysul58qG2p5emwa?dl=0

import excel "merge_db_environment_korea_kospi200_findel.xlsx", firstrow
keep if ksic1 != "K"
gen CO2Sale = co2_sale_gr_1000
keep if year<=2019
keep if year>=2011


global control_lag = "lag_size lag_cf_vol lag_capex lag_leverage lag_roa lag_ln_firm_age lag_rnd_sale"
global ESG_lag "lag_E_num lag_G_num lag_S_num"
global d_control_lag = "d_size d_cf_vol d_capex d_leverage d_roa d_ln_firm_age d_rnd_sale"
global d_ESG_lag "d_E_num d_G_num d_S_num"

**teffects nnmatch (CO2Sale ${control_lag} ${ESG_lag} ${d_control_lag} ${d_ESG_lag} i.ksic1_num i.year) (CarbonOffset), gen(match)**

2. I want to conduct a "t-difference mean test between treated and control groups after matching".

3. For instance, I can calculate a "t-difference mean test between treated and control groups" before matching like this:

ttest CO2Sale,by(CarbonOffset) level(99) unequal

4. And I can also calculate an Average Treatment Effect (ATE) like this as well:

teffects nnmatch (CO2Sale ${control_lag} ${ESG_lag} ${d_control_lag} ${d_ESG_lag} i.ksic1_num i.year) (CarbonOffset), gen(match)

5. However, I don't know how to conduct a "t-difference mean test between treated and control groups after matching". If there is someone who can conduct a similar test, I really want to know how to do it.

Answer 1

This is a bad idea, so I am adding a solution that demonstrates that.

In this example, the ATE of maternal smoking on birthweight is biased down by ~40%: The ATE from teffects nnmatch is -240 grams, but the corresponding difference from the t-test you propose is only -153 grams.

. /* Setup Data */
. webuse cattaneo2, clear
(Excerpt from Cattaneo (2010) Journal of Econometrics 155: 138–154)

. keep bweight mage mbsmoke prenatal1 mmarried fbaby

. gen id = _n

. order id

. 
. /* Store Duplicate Data */
. tempfile copy

. save `copy'
file /var/folders/62/51hy7j9958xd03txmytygwwc0000gn/T//S_57805.000001 saved as .dta format

. 
. /* Matching Step */
. teffects nnmatch (bweight mage prenatal1 mmarried fbaby) (mbsmoke), nn(1) gen(match) level(99)

Treatment-effects estimation                   Number of obs      =      4,642
Estimator      : nearest-neighbor matching     Matches: requested =          1
Outcome model  : matching                                     min =          1
Distance metric: Mahalanobis                                  max =        139
----------------------------------------------------------------------------------------
                       |              AI robust
               bweight | Coefficient  std. err.      z    P>|z|     [99% conf. interval]
-----------------------+----------------------------------------------------------------
ATE                    |
               mbsmoke |
(Smoker vs Nonsmoker)  |  -240.3306   28.43006    -8.45   0.000    -313.5616   -167.0997
----------------------------------------------------------------------------------------

. 
. /* Get Data Into T-Test Format */
. reshape long match, i(id) j(match_num)
(j = 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 
> 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 
> 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 
> 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 1
> 23 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139)

Data                               Wide   ->   Long
-----------------------------------------------------------------------------
Number of observations            4,642   ->   645,238     
Number of variables                 146   ->   9           
j variable (139 values)                   ->   match_num
xij variables:
             match1 match2 ... match139   ->   match
-----------------------------------------------------------------------------

. drop if missing(match)
(575,801 observations deleted)

. sort id match_num

. rename (id mbsmoke bweight mage prenatal1 mmarried fbaby) =_original

. rename  match id

. merge m:1 id using `copy', nogen keep(match)
(variable id was long, now double to accommodate using data's values)
(label YesNo already defined)
(label mmarried already defined)
(label mbsmoke already defined)

    Result                      Number of obs
    -----------------------------------------
    Not matched                             0
    Matched                            69,437  
    -----------------------------------------

. rename (id mbsmoke bweight mage prenatal1 mmarried fbaby) =_matched

. sort id_original

. 
. /* Average Over Birthweight in Case There Are Ties */
. /* Or To Keep Only the First Match Uncomment the Next Line */
. // keep if match_num == 1
. collapse (mean) bweight_matched, by(*_original)

. 
. /* Hypothesis Test */
. gen effect = cond(mbsmoke_original == "Smoke":mbsmoke, bweight_original - bweight_matched, bweig
> ht_matched - bweight_original)
(value label dereference "Smoke":mbsmoke not found)

. ttest effect = 0, level(99)

One-sample t test
------------------------------------------------------------------------------
Variable |     Obs        Mean    Std. err.   Std. dev.   [99% conf. interval]
---------+--------------------------------------------------------------------
  effect |   4,642   -153.7954    9.362789    637.9077   -177.9223   -129.6686
------------------------------------------------------------------------------
    mean = mean(effect)                                           t = -16.4262
H0: mean = 0                                     Degrees of freedom =     4641

    Ha: mean < 0                 Ha: mean != 0                 Ha: mean > 0
 Pr(T < t) = 0.0000         Pr(|T| > |t|) = 0.0000          Pr(T > t) = 1.0000

---

Code:

/* Setup Data */
webuse cattaneo2, clear
keep bweight mage mbsmoke prenatal1 mmarried fbaby
gen id = _n
order id

/* Store Duplicate Data */
tempfile copy
save `copy'

/* Matching Step */
teffects nnmatch (bweight mage prenatal1 mmarried fbaby) (mbsmoke), nn(1) gen(match) level(99)

/* Get Data Into T-Test Format */
reshape long match, i(id) j(match_num)
drop if missing(match)
sort id match_num
rename (id mbsmoke bweight mage prenatal1 mmarried fbaby) =_original
rename  match id
merge m:1 id using `copy', nogen keep(match)
rename (id mbsmoke bweight mage prenatal1 mmarried fbaby) =_matched
sort id_original

/* Average Over Birthweight in Case There Are Ties */
/* Or To Keep Only the First Match Uncomment the Next Line */
// keep if match_num == 1
collapse (mean) bweight_matched, by(*_original)

/* Hypothesis Test */
gen effect = cond(mbsmoke_original == "Smoke":mbsmoke, bweight_original - bweight_matched, bweight_matched - bweight_original)
ttest effect = 0, level(99)

Answer 2

This doesn't make sense beceause teffects doesn't create a matched sample. You can't just extract the matched units and run a t-test on them. teffects creates pairs and imputes the missing potential outcomes in one group with the paired units in the other group. No units are discarded. If you want the treatment effect estimate, you need to use the ATE output provided by teffects. See my answer here and the links therein for more information on the differences between matching imputation (what teffects does) and matching as nonparametric preprocessing (what almost all papers about matching do). If you want to do matching as nonparametric preprocessing, this is not possible in Stata. The MatchIt package in R and PROC PSMATCH in SAS do allow this, though.