2
$\begingroup$

I want to perform propensity score matching of observational data of an Intensive Care Unit in order to find out wheather hydroxyethyl starch is better or worse than colloids in terms of renal replacement therapy (RRT), Akute Kidney Injury (AKI) and mortality.

I use the MatchIt package in R (King et al. 2007 - http://gking.harvard.edu/matchit). This package is quite well documented. But there are some things that I dont understand. First I matched on sociodemographic covariates (as this seems standard protocol with matching): Gender, weight, height and age. Nearest neighbor matching seems to have worked:

NN matching

m.out.nn

Call: 
matchit(formula = treat1 ~ Geschlecht + Gewicht.kg + Groesse.cm + 
    Alter.bei.ITS.Aufn + BMI, data = hes.vs.kristall.clean, method = "nearest")

Sample sizes:
          Control Treated
All           359    3944
Matched       359     359
Unmatched       0    3585
Discarded       0       0

Treatment status treat1 is HES = yes , Colloids otherwise btw. I did a numerical balance check and balance actually WORSENED after matching. Overall as well as some of the covariates drastically:
Call:
matchit(formula = treat1 ~ Geschlecht + Gewicht.kg + Groesse.cm + 
    Alter.bei.ITS.Aufn + BMI, data = hes.vs.kristall.clean, method = "nearest")

Summary of balance for all data:
                   Means Treated Means Control SD Control Mean Diff eQQ Med eQQ Mean  eQQ Max
distance                  0.9168        0.9145     0.0131    0.0022  0.0018   0.0023   0.0388
Geschlecht                0.0041        0.0000     0.0000    0.0041  0.0000   0.0056   1.0000
Geschlechtm               0.6463        0.6100     0.4884    0.0363  0.0000   0.0362   1.0000
Geschlechtw               0.3496        0.3900     0.4884   -0.0403  0.0000   0.0390   1.0000
Gewicht.kg               79.1349       77.8930    18.2092    1.2419  2.0000   1.8462  30.0000
Groesse.cm              169.9184      169.9861    11.9693   -0.0677  0.0000   0.7604  30.0000
Alter.bei.ITS.Aufn       64.5950       63.4808    14.4918    1.1142  0.8000   1.2916   6.2000
BMI                      28.4858       27.8005    15.1559    0.6853  0.7080   2.1550 347.8520


Summary of balance for matched data:
                   Means Treated Means Control SD Control Mean Diff eQQ Med eQQ Mean  eQQ Max
distance                  0.9357        0.9145     0.0131    0.0212  0.0172   0.0212   0.0687
Geschlecht                0.0446        0.0000     0.0000    0.0446  0.0000   0.0446   1.0000
Geschlechtm               0.9053        0.6100     0.4884    0.2953  0.0000   0.2953   1.0000
Geschlechtw               0.0501        0.3900     0.4884   -0.3398  0.0000   0.3398   1.0000
Gewicht.kg               98.1744       77.8930    18.2092   20.2813 17.0000  20.2813  62.0000
Groesse.cm              164.7103      169.9861    11.9693   -5.2758  2.0000   5.4540  77.0000
Alter.bei.ITS.Aufn       72.5702       63.4808    14.4918    9.0894  7.0000   9.0894  26.5000
BMI                      44.1753       27.8005    15.1559   16.3748  6.7500  16.3748 258.9020

Percent Balance Improvement:
                   Mean Diff.   eQQ Med  eQQ Mean   eQQ Max
distance            -852.5593 -839.8420 -808.2029  -77.0723
Geschlecht          -998.6072    0.0000 -700.0000    0.0000
Geschlechtm         -714.0668    0.0000 -715.3846    0.0000
Geschlechtw         -742.6908    0.0000 -771.4286    0.0000
Gewicht.kg         -1533.1522 -750.0000 -998.5214 -106.6667
Groesse.cm         -7691.0845      -Inf -617.2161 -156.6667
Alter.bei.ITS.Aufn  -715.7611 -775.0000 -603.7093 -327.4194
BMI                -2289.4307 -853.3898 -659.8482   25.5712

Sample sizes:
          Control Treated
All           359    3944
Matched       359     359
Unmatched       0    3585
Discarded       0       0

How can this be possible?

I also did genetic matching (Sekhon 2011 - http://sekhon.berkeley.edu/matching/). This is a fancy algorithm that automatically optimizes covariate balance. There covariate balance has indeed improved (as it should have):

Genetic matching
load(file="m.out.genetic.RData")
Numerical Balance Check 
summary(m.out.genetic)

Call:
matchit(formula = treat1 ~ Geschlecht + Gewicht.kg + Groesse.cm + 
    Alter.bei.ITS.Aufn, data = hes.vs.kristall.clean, method = "genetic")

Summary of balance for all data:
                   Means Treated Means Control SD Control Mean Diff eQQ Med eQQ Mean eQQ Max
distance                  0.9167        0.9147     0.0126    0.0021  0.0019   0.0022  0.0374
Geschlecht                0.0041        0.0000     0.0000    0.0041  0.0000   0.0056  1.0000
Geschlechtm               0.6463        0.6100     0.4884    0.0363  0.0000   0.0362  1.0000
Geschlechtw               0.3496        0.3900     0.4884   -0.0403  0.0000   0.0390  1.0000
Gewicht.kg               79.1349       77.8930    18.2092    1.2419  2.0000   1.8462 30.0000
Groesse.cm              169.9184      169.9861    11.9693   -0.0677  0.0000   0.7604 30.0000
Alter.bei.ITS.Aufn       64.5950       63.4808    14.4918    1.1142  0.8000   1.2916  6.2000


Summary of balance for matched data:
                   Means Treated Means Control SD Control Mean Diff eQQ Med eQQ Mean eQQ Max
distance                  0.9167        0.9164     0.0105    0.0003  0.0018   0.0021  0.0374
Geschlecht                0.0041        0.0000     0.0000    0.0041  0.0000   0.0056  1.0000
Geschlechtm               0.6463        0.6481     0.4782   -0.0018  0.0000   0.0364  1.0000
Geschlechtw               0.3496        0.3519     0.4782   -0.0023  0.0000   0.0392  1.0000
Gewicht.kg               79.1349       79.0556    15.9832    0.0793  2.0000   1.7801 30.0000
Groesse.cm              169.9184      170.0479    10.6992   -0.1296  0.0000   0.7703 30.0000
Alter.bei.ITS.Aufn       64.5950       64.7378    13.3160   -0.1428  0.8000   1.2440  6.2000

Percent Balance Improvement:
                   Mean Diff. eQQ Med eQQ Mean eQQ Max
distance              83.7418  3.8423   2.7923       0
Geschlecht             0.0000  0.0000  -0.5602       0
Geschlechtm           95.1066  0.0000  -0.5602       0
Geschlechtw           94.3414  0.0000  -0.5602       0
Gewicht.kg            93.6115  0.0000   3.5817       0
Groesse.cm           -91.3359  0.0000  -1.2969       0
Alter.bei.ITS.Aufn    87.1817  0.0000   3.6903       0

Sample sizes:
          Control Treated
All           359    3944
Matched       357    3944
Unmatched       2       0
Discarded       0       0

I also checked balance graphically and it did improve (despite being good pre-matching).

Now my questions are:

  1. Can I use the nearest neighbor matched data? How could I change this so that balance does improve? What kind of distance metric does Nearest neighbor matching use (by default) (Euclidean ?). Because with Euclidean the non-Boolean covariates (Gender) could be made more important than they are.

  2. How can I perform analysis after matching? - How can I get the Average Treatment Effect (ATE) and the Average Treatment Effect on the Treated (ATET) in terms of HES for AKI, RRT and mortaility and does that make sense for these response variables (AKI, RRT and mortaility)? Or should I get the odds ratio for Akute Kidney Injury, renal replacement therapy and mortaility from the matched observational data? How do I get these values? I know that MatchIt recommends using Zelig to get these values but that didn't seem to work with my data. Can I use logistic regression with the matched data to get the odds ratio of HES vs. Cristalloids of AKI, RRT and mortality ?

Improve this question
$\endgroup$
3
  • 1
    $\begingroup$ It appearst that the genetic matching has created ATT weights. Is there an option to change to ATE? (I don't know the package). I like the twang package, and use that. It's pretty clear how to get ATE and ATT. $\endgroup$
    – Jeremy Miles
    Commented Feb 18, 2015 at 18:24
  • $\begingroup$ Hi Jeremy, wow, that was fast. Thanks. yeah, i will look into the twang package. Do you also happen to know how i could get the odds ratio for AKI and RRT in HES vs. Cristalloids ? thx. Greetings, Philipp $\endgroup$
    – Philipp-Packmohr
    Commented Feb 18, 2015 at 18:36
  • $\begingroup$ I don't, I'm afraid. With twang, you use the survey package with svyglm to get odds ratios. $\endgroup$
    – Jeremy Miles
    Commented Feb 18, 2015 at 21:30

1 Answer 1

Reset to default
3
$\begingroup$

1) Using the defaults for MatchIt, nearest neighbor matching matches on the propensity score as defined by a logistic regression of treatment on the covariates included in your formula. For each treated unit, it finds the one unmatched control with the closest propensity score, and then throws out the unmatched control units. There is no issue with continuous vs. categorical covariates here. Note King & Nielsen (2016), who describe why propensity score matching can actually make balance worse, as in your example.

2) MatchIt creates matches for the ATET, but the Matching package, which also implements genetic matching, allows you to specify that you want the ATE. After matching, you can simply perform the regression analysis you would have had you randomly assigned your units (assuming balance has been achieved).

Improve this answer
$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.