At first glance, both the propensity score matching DID (PSM DID) and the inverse probability weighting (IPW DID) that you want to do are sensible ways to approach this: matching and weighting will take care of the selection into treatment based on observables, and the DID will deal with selection on unobservables as long as the bias from it is time-invariant, conditional on covariates. Matching is just an attempt to approximate what weighting is doing directly. If you're just worried about covariate imbalance, you can include them as controls, as long as they are variables that affect both treatment and the outcome, and are not affected by the treatment itself or by anticipation of treatment.
Also, as an aside, you can never test the parallel paths or trends/growth assumptions, even if you had more data. All you can say is that is seems to hold in the pre-treatment period, so you would be more willing to believe it holds in the post-treatment future. But is is still an assumption.
In the example below, your IPW approach will produce estimates that match what Stata's own teffects ipw would give you for the ATET/ATT and ATE. I will also demonstrate some alternative matching methods that you might find useful as a robustness check. Why? With IPW, things can go south if the estimated propensity scores are near zero or 1 since you would be dividing by small numbers. This is not the case in this example, and so the estimates are fairly similar. Asymptotically, both PSM and IPW should give you the same answer (though I cannot recall where I saw this result).
Below I will employ a user-written command that can do two-period Kernel PSM DID called diff. I will also do a regression version with the kernel-based weights from the command. Finally, you can also trick matching estimators, like user-written psmatch2 or Stata's own teffects ipw, to do DID with two periods, or you can also do a regression version of ipw.
Here's the annotated example:
. set more off
. estimates clear
. use http://fmwww.bc.edu/repec/bocode/c/CardKrueger1994.dta, clear
(Dataset from Card&Krueger (1994))
. /* Need to balance the panels so that all the methods use the same sample */
. drop if missing(fte)
(19 observations deleted)
. bys id (t): keep if _N==2
(23 observations deleted)
.
. /* (1) -diff- to get ATT */
. diff fte, treated(treated) period(t) id(id) cov(bk kfc roys) kernel ktype(gaussian) support bw(0.05) cluster(id) robust // report
KERNEL PROPENSITY SCORE MATCHING DIFFERENCE-IN-DIFFERENCES
Estimation on common support
Matching iterations...
..............................................................................................................................................................................
> ............................................................................................................................................
DIFFERENCE-IN-DIFFERENCES ESTIMATION RESULTS
Number of observations in the DIFF-IN-DIFF: 778
Baseline Follow-up
Control: 75 75 150
Treated: 314 314 628
389 389
------------------------------------------------------
Outcome var. | fte | S. Err. | t | P>|t|
----------------+---------+---------+-------+---------
Baseline | | | |
Control | 20.471 | | |
Treated | 17.069 | | |
Diff (T-C) | -3.402 | 1.481 | -2.30 | 0.022**
Follow-up | | | |
Control | 17.792 | | |
Treated | 17.518 | | |
Diff (T-C) | -0.274 | 1.067 | -0.26 | 0.798
| | | |
Diff-in-Diff | 3.128 | 1.388 | 2.25 | 0.025**
------------------------------------------------------
R-square: 0.02
* Means and Standard Errors are estimated by linear regression
**Robust Std. Errors
**Clustered Std. Errors
**Inference: *** p<0.01; ** p<0.05; * p<0.1
. assert _support==1
.
. /* (2a) Regression Using Kernel Weights for ATE */
. reg fte i.treated##i.t [aw=_weights], cluster(id) robust
(sum of wgt is 1.2560e+03)
Linear regression Number of obs = 778
F(3, 388) = 2.04
Prob > F = 0.1075
R-squared = 0.0193
Root MSE = 9.5
(Std. Err. adjusted for 389 clusters in id)
------------------------------------------------------------------------------
| Robust
fte | Coef. Std. Err. t P>|t| [95% Conf. Interval]
-------------+----------------------------------------------------------------
treated |
NJ | -3.401828 1.481415 -2.30 0.022 -6.314433 -.4892233
1.t | -2.679224 1.307741 -2.05 0.041 -5.250369 -.1080794
|
treated#t |
NJ#1 | 3.128269 1.388281 2.25 0.025 .3987744 5.857763
|
_cons | 20.4711 1.39505 14.67 0.000 17.72829 23.2139
------------------------------------------------------------------------------
. xtset id t
panel variable: id (strongly balanced)
time variable: t, 0 to 1
delta: 1 unit
. xtreg fte i.treated##i.t [aw=_weights], fe cluster(id) robust
note: 1.treated omitted because of collinearity
Fixed-effects (within) regression Number of obs = 778
Group variable: id Number of groups = 389
R-sq: Obs per group:
within = 0.0376 min = 2
between = 0.0065 avg = 2.0
overall = 0.0000 max = 2
F(2,388) = 2.57
corr(u_i, Xb) = -0.1104 Prob > F = 0.0781
(Std. Err. adjusted for 389 clusters in id)
------------------------------------------------------------------------------
| Robust
fte | Coef. Std. Err. t P>|t| [95% Conf. Interval]
-------------+----------------------------------------------------------------
treated |
NJ | 0 (omitted)
1.t | -2.679224 1.306897 -2.05 0.041 -5.24871 -.1097388
|
treated#t |
NJ#1 | 3.128269 1.387385 2.25 0.025 .4005359 5.856002
|
_cons | 18.77018 .3468462 54.12 0.000 18.08825 19.45211
-------------+----------------------------------------------------------------
sigma_u | 8.0179596
sigma_e | 6.8873514
rho | .57541878 (fraction of variance due to u_i)
------------------------------------------------------------------------------
.
. /* Create ATT and ATE Weights for regressions*/
. /* You can install -xfill- by typing net from https://www.sealedenvelope.com/ and clicking on the name */
. xfill _ps, i(id)
. gen double w_att = cond(treated==1,1,_ps/(1-_ps))
. gen double w_ate = cond(treated==1,1/_ps,1/(1-_ps))
.
. /* (2b) Regression Using ATT and ATE IPW Weights */
. reg fte i.treated##i.t [pw=w_att] /*if _support==1*/, cluster(id) robust
(sum of wgt is 1.2560e+03)
Linear regression Number of obs = 778
F(3, 388) = 1.48
Prob > F = 0.2187
R-squared = 0.0079
Root MSE = 9.4359
(Std. Err. adjusted for 389 clusters in id)
------------------------------------------------------------------------------
| Robust
fte | Coef. Std. Err. t P>|t| [95% Conf. Interval]
-------------+----------------------------------------------------------------
treated |
NJ | -1.897508 1.458586 -1.30 0.194 -4.765229 .9702128
1.t | -2.182001 1.170245 -1.86 0.063 -4.482815 .118813
|
treated#t |
NJ#1 | 2.631046 1.259607 2.09 0.037 .1545361 5.107555
|
_cons | 18.96678 1.370783 13.84 0.000 16.27168 21.66187
------------------------------------------------------------------------------
. reg fte i.treated##i.t [pw=w_ate] /*if _support==1*/, cluster(id) robust
(sum of wgt is 1.5560e+03)
Linear regression Number of obs = 778
F(3, 388) = 1.53
Prob > F = 0.2068
R-squared = 0.0086
Root MSE = 9.4409
(Std. Err. adjusted for 389 clusters in id)
------------------------------------------------------------------------------
| Robust
fte | Coef. Std. Err. t P>|t| [95% Conf. Interval]
-------------+----------------------------------------------------------------
treated |
NJ | -2.014025 1.455524 -1.38 0.167 -4.875727 .8476772
1.t | -2.247811 1.184075 -1.90 0.058 -4.575816 .0801949
|
treated#t |
NJ#1 | 2.712311 1.275497 2.13 0.034 .2045602 5.220061
|
_cons | 19.1994 1.367706 14.04 0.000 16.51036 21.88845
------------------------------------------------------------------------------
. xtreg fte i.treated##i.t [pw=w_att], fe cluster(id) robust
note: 1.treated omitted because of collinearity
Fixed-effects (within) regression Number of obs = 778
Group variable: id Number of groups = 389
R-sq: Obs per group:
within = 0.0275 min = 2
between = 0.0065 avg = 2.0
overall = 0.0000 max = 2
F(2,388) = 2.21
corr(u_i, Xb) = -0.1023 Prob > F = 0.1116
(Std. Err. adjusted for 389 clusters in id)
------------------------------------------------------------------------------
| Robust
fte | Coef. Std. Err. t P>|t| [95% Conf. Interval]
-------------+----------------------------------------------------------------
treated |
NJ | 0 (omitted)
1.t | -2.182001 1.169489 -1.87 0.063 -4.481331 .1173282
|
treated#t |
NJ#1 | 2.631046 1.258794 2.09 0.037 .1561344 5.105957
|
_cons | 18.01802 .3146986 57.25 0.000 17.39929 18.63675
-------------+----------------------------------------------------------------
sigma_u | 8.0031528
sigma_e | 6.6447787
rho | .59194417 (fraction of variance due to u_i)
------------------------------------------------------------------------------
. xtreg fte i.treated##i.t [pw=w_ate], fe cluster(id) robust
note: 1.treated omitted because of collinearity
Fixed-effects (within) regression Number of obs = 778
Group variable: id Number of groups = 389
R-sq: Obs per group:
within = 0.0287 min = 2
between = 0.0065 avg = 2.0
overall = 0.0000 max = 2
F(2,388) = 2.28
corr(u_i, Xb) = -0.1037 Prob > F = 0.1032
(Std. Err. adjusted for 389 clusters in id)
------------------------------------------------------------------------------
| Robust
fte | Coef. Std. Err. t P>|t| [95% Conf. Interval]
-------------+----------------------------------------------------------------
treated |
NJ | 0 (omitted)
1.t | -2.247811 1.183311 -1.90 0.058 -4.574314 .0786925
|
treated#t |
NJ#1 | 2.712311 1.274674 2.13 0.034 .2061786 5.218443
|
_cons | 18.19239 .3186684 57.09 0.000 17.56586 18.81892
-------------+----------------------------------------------------------------
sigma_u | 8.0054923
sigma_e | 6.6977382
rho | .58824522 (fraction of variance due to u_i)
------------------------------------------------------------------------------
.
. /* (3) psmatch2 for ATE and ATT */
. keep id fte treated t bk kfc roys
. qui reshape wide fte, i(id treated bk kfc roys) j(t)
. gen did_fte = fte1 - fte0
. psmatch2 treated bk kfc roys, outcome(fte0 fte1 did_fte) kernel kerneltype(normal) bw(0.05) common ate
Probit regression Number of obs = 389
LR chi2(3) = 2.97
Prob > chi2 = 0.3956
Log likelihood = -189.22405 Pseudo R2 = 0.0078
------------------------------------------------------------------------------
treated | Coef. Std. Err. z P>|z| [95% Conf. Interval]
-------------+----------------------------------------------------------------
bk | .1368372 .2190829 0.62 0.532 -.2925573 .5662318
kfc | .4092482 .2579891 1.59 0.113 -.0964012 .9148976
roys | .2448943 .2415267 1.01 0.311 -.2284893 .7182778
_cons | .6744898 .1889631 3.57 0.000 .3041288 1.044851
------------------------------------------------------------------------------
----------------------------------------------------------------------------------------
Variable Sample | Treated Controls Difference S.E. T-stat
----------------------------+-----------------------------------------------------------
fte0 Unmatched | 17.0692675 20.1733333 -3.10406582 1.21654594 -2.55
ATT | 17.0692675 20.4710986 -3.40183108 1.4673544 -2.32
ATU | 20.1733333 17.6299058 -2.54342757 . .
ATE | -3.23632912 . .
----------------------------+-----------------------------------------------------------
fte1 Unmatched | 17.5183121 17.65 -.131687898 1.11242821 -0.12
ATT | 17.5183121 17.7918742 -.273562085 1.05193873 -0.26
ATU | 17.65 18.0632041 .413204067 . .
ATE | -.141152158 . .
----------------------------+-----------------------------------------------------------
did_fte Unmatched | .449044586 -2.52333333 2.97237792 1.1318176 2.63
ATT | .449044586 -2.67922441 3.128269 1.35190664 2.31
ATU |-2.52333333 .433298308 2.95663164 . .
ATE | 3.09517696 . .
----------------------------+-----------------------------------------------------------
Note: S.E. does not take into account that the propensity score is estimated.
| psmatch2:
psmatch2: | Common
Treatment | support
assignment | On suppor | Total
-----------+-----------+----------
Untreated | 75 | 75
Treated | 314 | 314
-----------+-----------+----------
Total | 389 | 389
.
. /* (4) IPW Matching ATT=ATET and ATE */
. teffects ipw (did_fte) (treated bk kfc roys, probit), atet vce(robust) // osample(ipw1)
Iteration 0: EE criterion = 1.142e-28
Iteration 1: EE criterion = 3.433e-31
Treatment-effects estimation Number of obs = 389
Estimator : inverse-probability weights
Outcome model : weighted mean
Treatment model: probit
------------------------------------------------------------------------------
| Robust
did_fte | Coef. Std. Err. z P>|z| [95% Conf. Interval]
-------------+----------------------------------------------------------------
ATET |
treated |
(NJ vs PA) | 2.631046 1.220372 2.16 0.031 .2391604 5.022931
-------------+----------------------------------------------------------------
POmean |
treated |
PA | -2.182001 1.130429 -1.93 0.054 -4.3976 .0335979
------------------------------------------------------------------------------
. teffects ipw (did_fte) (treated bk kfc roys, probit), ate vce(robust) // osample(ipw2)
Iteration 0: EE criterion = 1.147e-28
Iteration 1: EE criterion = 1.002e-31
Treatment-effects estimation Number of obs = 389
Estimator : inverse-probability weights
Outcome model : weighted mean
Treatment model: probit
------------------------------------------------------------------------------
| Robust
did_fte | Coef. Std. Err. z P>|z| [95% Conf. Interval]
-------------+----------------------------------------------------------------
ATE |
treated |
(NJ vs PA) | 2.712311 1.238966 2.19 0.029 .283982 5.14064
-------------+----------------------------------------------------------------
POmean |
treated |
PA | -2.247811 1.146997 -1.96 0.050 -4.495884 .0002625
------------------------------------------------------------------------------
To summarize, Kernel PSM DID and its regression analogue as well as PSM with psmatch2 on differences yield an ATT of 3.13 FTEs. The weights from this program are more involved than the IPW weights.
The IPW on differences and the regression analogue you propose are similar to each other with an ATT of 2.63 FTEs, but smaller than the PSM based-estimates, though the differences are not significant.
Code:
cls
set more off
estimates clear
use http://fmwww.bc.edu/repec/bocode/c/CardKrueger1994.dta, clear
/* Need to balance the panels so that all the methods use the same sample */
drop if missing(fte)
bys id (t): keep if _N==2
/* (1) -diff- to get ATT */
diff fte, treated(treated) period(t) id(id) cov(bk kfc roys) kernel ktype(gaussian) support bw(0.05) cluster(id) robust // report
assert _support==1
/* (2a) Regression Using Kernel Weights for ATE */
reg fte i.treated##i.t [aw=_weights], cluster(id) robust
xtset id t
xtreg fte i.treated##i.t [aw=_weights], fe cluster(id) robust
/* Create ATT and ATE Weights for regressions*/
/* You can install -xfill- by typing net from http://www.sealedenvelope.com/ and clicking on the name */
xfill _ps, i(id)
gen double w_att = cond(treated==1,1,_ps/(1-_ps))
gen double w_ate = cond(treated==1,1/_ps,1/(1-_ps))
/* (2b) Regression Using ATT and ATE IPW Weights */
reg fte i.treated##i.t [pw=w_att] /*if _support==1*/, cluster(id) robust
reg fte i.treated##i.t [pw=w_ate] /*if _support==1*/, cluster(id) robust
xtreg fte i.treated##i.t [pw=w_att], fe cluster(id) robust
xtreg fte i.treated##i.t [pw=w_ate], fe cluster(id) robust
/* (3) psmatch2 for ATE and ATT */
keep id fte treated t bk kfc roys
qui reshape wide fte, i(id treated bk kfc roys) j(t)
gen did_fte = fte1 - fte0
psmatch2 treated bk kfc roys, outcome(fte0 fte1 did_fte) kernel kerneltype(normal) bw(0.05) common ate
/* (4) IPW Matching ATT=ATET and ATE */
teffects ipw (did_fte) (treated bk kfc roys, probit), atet vce(robust) // osample(ipw1)
teffects ipw (did_fte) (treated bk kfc roys, probit), ate vce(robust) // osample(ipw2)
