0
$\begingroup$

I am using MatchIt to carry out some propensity score matching, and then using the cobalt package to generate the balance diagnostic. The summary() command on MatchIt has a known bug where it does not generate the standard mean differences for the first level of multi-categorical variables. I know that, based on this post:

MatchIt: Explanation of Missing Factors from Summary

the bal.tab in the cobalt package is supposed to print out the standard mean differences for all levels of multi-categorical variables. However, for me, the cobalt package is still missing the first levels. For instance, I have the following multi-categorical variables in the MatchIt formula (number of levels in parenthesis) - RACE (4), PAY1 (4), ZIPINC_QRTL (4), PL_UR_CAT4 (4), CM_ELIX_BIN (5): 

set.seed(103)
matched_avgdata <- matchit(SURGERY_SETTING ~ 
                             AGE + FEMALE + RACE + PAY1 +  ZIPINC_QRTL + PL_UR_CAT4 + STATE + YEAR + CM_ELIX_BIN +
                             CM_CHF +  CM_VALVE +  CM_PULMCIRC +  CM_PERIVASC +  CM_HTN_C +  CM_PARA +  CM_NEURO +  CM_CHRNLUNG +  
                             CM_DM +  CM_DMCX +  CM_HYPOTHY +  CM_RENLFAIL +  CM_LIVER +  CM_ULCER +  CM_AIDS +  CM_LYMPH +  CM_METS +  CM_TUMOR +  
                             CM_ARTH +  CM_COAG +  CM_OBESE +  
                             CM_WGHTLOSS +  CM_LYTES +  CM_BLDLOSS +  CM_ANEMDEF +  CM_ALCOHOL +  CM_DRUG +  CM_PSYCH +  
                             CM_DEPRESS,
                        data = avgdata_selected, distance=PS_average$myList, ratio = 1, method = "nearest", caliper = 0.1)

Output with MatchIt summary() formula:

summary(matched_avgdata, standardize=TRUE)

Summary of balance for matched data:
             Means Treated Means Control SD Control Std. Mean Diff. eCDF Med eCDF Mean eCDF Max
distance            0.0736        0.0734     0.0617          0.0032   0.0040    0.0043   0.0182
AGE                64.9960       64.5648     9.5577          0.0397   0.0101    0.0166   0.0668
FEMALE0             0.5587        0.5547     0.4975          0.0082   0.0020    0.0020   0.0040
FEMALE1             0.4413        0.4453     0.4975         -0.0082   0.0020    0.0020   0.0040
RACE2               0.0607        0.0688     0.2534         -0.0321   0.0040    0.0040   0.0081
RACE3               0.0931        0.1073     0.3098         -0.0489   0.0071    0.0071   0.0142
RACE4               0.0344        0.0263     0.1602          0.0446   0.0040    0.0040   0.0081
PAY12               0.0202        0.0202     0.1410          0.0000   0.0000    0.0000   0.0000
PAY13               0.4271        0.4372     0.4965         -0.0204   0.0051    0.0051   0.0101
PAY14               0.1032        0.1032     0.3046          0.0000   0.0000    0.0000   0.0000
ZIPINC_QRTL2        0.3077        0.2976     0.4577          0.0219   0.0051    0.0051   0.0101
ZIPINC_QRTL3        0.2814        0.2672     0.4429          0.0314   0.0071    0.0071   0.0142
ZIPINC_QRTL4        0.1599        0.1761     0.3813         -0.0443   0.0081    0.0081   0.0162
PL_UR_CAT42         0.2632        0.2632     0.4408          0.0000   0.0000    0.0000   0.0000
PL_UR_CAT43         0.0486        0.0425     0.2020          0.0278   0.0030    0.0030   0.0061
PL_UR_CAT44         0.0263        0.0263     0.1602          0.0000   0.0000    0.0000   0.0000
STATE1              0.2834        0.3077     0.4620         -0.0540   0.0121    0.0121   0.0243
YEAR2015            0.3158        0.3482     0.4769         -0.0696   0.0162    0.0162   0.0324
YEAR2016            0.4271        0.4130     0.4929          0.0286   0.0071    0.0071   0.0142
CM_ELIX_BIN1        0.2814        0.3158     0.4653         -0.0767   0.0172    0.0172   0.0344
CM_ELIX_BIN2        0.1842        0.1619     0.3688          0.0576   0.0111    0.0111   0.0223
CM_ELIX_BIN3        0.0506        0.0405     0.1973          0.0463   0.0051    0.0051   0.0101
CM_ELIX_BIN4        0.0364        0.0385     0.1925         -0.0106   0.0010    0.0010   0.0020
CM_CHF1             0.0101        0.0101     0.1002          0.0000   0.0000    0.0000   0.0000
CM_VALVE1           0.0081        0.0121     0.1096         -0.0454   0.0020    0.0020   0.0040
CM_PULMCIRC1        0.0040        0.0020     0.0450          0.0320   0.0010    0.0010   0.0020
CM_PERIVASC1        0.0162        0.0162     0.1264          0.0000   0.0000    0.0000   0.0000
CM_HTN_C1           0.4555        0.4514     0.4981          0.0081   0.0020    0.0020   0.0040
CM_PARA1            0.0000        0.0000     0.0000             NaN   0.0000    0.0000   0.0000
CM_NEURO1           0.0061        0.0061     0.0778          0.0000   0.0000    0.0000   0.0000
CM_CHRNLUNG1        0.0891        0.0870     0.2822          0.0071   0.0010    0.0010   0.0020
CM_DM1              0.1154        0.0992     0.2992          0.0509   0.0081    0.0081   0.0162
CM_DMCX1            0.0101        0.0081     0.0897          0.0203   0.0010    0.0010   0.0020
CM_HYPOTHY1         0.0547        0.0466     0.2109          0.0358   0.0040    0.0040   0.0081
CM_RENLFAIL1        0.0040        0.0040     0.0636          0.0000   0.0000    0.0000   0.0000
CM_LIVER1           0.0101        0.0061     0.0778          0.0406   0.0020    0.0020   0.0040
CM_ULCER1           0.0000        0.0000     0.0000             NaN   0.0000    0.0000   0.0000
CM_AIDS1            0.0020        0.0000     0.0000          0.0452   0.0010    0.0010   0.0020
CM_LYMPH1           0.0000        0.0000     0.0000             NaN   0.0000    0.0000   0.0000
CM_METS1            0.0000        0.0000     0.0000          0.0000   0.0000    0.0000   0.0000
CM_TUMOR1           0.0040        0.0020     0.0450          0.0262   0.0010    0.0010   0.0020
CM_ARTH1            0.0121        0.0061     0.0778          0.0557   0.0030    0.0030   0.0061
CM_COAG1            0.0040        0.0061     0.0778         -0.0320   0.0010    0.0010   0.0020
CM_OBESE1           0.0385        0.0425     0.2020         -0.0211   0.0020    0.0020   0.0040
CM_WGHTLOSS1        0.0000        0.0000     0.0000             NaN   0.0000    0.0000   0.0000
CM_LYTES1           0.0081        0.0121     0.1096         -0.0454   0.0020    0.0020   0.0040
CM_BLDLOSS1         0.0000        0.0000     0.0000             NaN   0.0000    0.0000   0.0000
CM_ANEMDEF1         0.0061        0.0061     0.0778          0.0000   0.0000    0.0000   0.0000
CM_ALCOHOL1         0.0000        0.0000     0.0000             NaN   0.0000    0.0000   0.0000
CM_DRUG1            0.0040        0.0040     0.0636          0.0000   0.0000    0.0000   0.0000
CM_PSYCH1           0.0000        0.0000     0.0000             NaN   0.0000    0.0000   0.0000
CM_DEPRESS1         0.0526        0.0567     0.2315         -0.0182   0.0020    0.0020   0.0040

Notice how RACE (4), PAY1 (4), ZIPINC_QRTL (4), PL_UR_CAT4 (4), CM_ELIX_BIN (5) are all missing a level. Standard mean differences are only calculated for 3 levels in RACE, 3 levels in PAY1, 3 levels in ZIPINC_QRTL, 3 levels in PL_UR_CAT4, and 4 levels in CM_ELIX_BIN.

Here is the implementation with the bal.tab command in the cobalt package:

bal.tab(matched_avgdata, un = TRUE, binary = "std")

Balance Measures
                 Type Diff.Un Diff.Adj
distance     Distance  0.7974   0.0032
AGE           Contin. -0.3945   0.0397
FEMALE0        Binary  0.1482   0.0082
RACE2          Binary  0.1120  -0.0321
RACE3          Binary  0.1484  -0.0489
RACE4          Binary -0.0051   0.0446
PAY12          Binary -0.0177   0.0000
PAY13          Binary  0.4511  -0.0204
PAY14          Binary  0.1471   0.0000
ZIPINC_QRTL2   Binary  0.0261   0.0219
ZIPINC_QRTL3   Binary  0.0949   0.0314
ZIPINC_QRTL4   Binary -0.1916  -0.0443
PL_UR_CAT42    Binary -0.3528   0.0000
PL_UR_CAT43    Binary  0.0209   0.0278
PL_UR_CAT44    Binary -0.0213   0.0000
STATE1         Binary -0.2455  -0.0540
YEAR2015       Binary -0.0462  -0.0696
YEAR2016       Binary  0.1211   0.0286
CM_ELIX_BIN1   Binary  0.0157  -0.0767
CM_ELIX_BIN2   Binary -0.1959   0.0576
CM_ELIX_BIN3   Binary -0.5315   0.0463
CM_ELIX_BIN4   Binary -0.5487  -0.0106
CM_CHF1        Binary -0.1244   0.0000
CM_VALVE1      Binary -0.3693  -0.0454
CM_PULMCIRC1   Binary -0.0274   0.0320
CM_PERIVASC1   Binary -0.0709   0.0000
CM_HTN_C1      Binary -0.4036   0.0081
CM_PARA1       Binary           0.0000
CM_NEURO1      Binary -0.3707   0.0000
CM_CHRNLUNG1   Binary -0.2873   0.0071
CM_DM1         Binary -0.1701   0.0509
CM_DMCX1       Binary -0.1209   0.0203
CM_HYPOTHY1    Binary -0.4758   0.0358
CM_RENLFAIL1   Binary -0.7866   0.0000
CM_LIVER1      Binary -0.0043   0.0406
CM_ULCER1      Binary           0.0000
CM_AIDS1       Binary  0.0381   0.0452
CM_LYMPH1      Binary           0.0000
CM_METS1       Binary  0.0304   0.0000
CM_TUMOR1      Binary  0.0156   0.0262
CM_ARTH1       Binary -0.3621   0.0557
CM_COAG1       Binary -0.2059  -0.0320
CM_OBESE1      Binary -0.7988  -0.0211
CM_WGHTLOSS1   Binary           0.0000
CM_LYTES1      Binary -0.5490  -0.0454
CM_BLDLOSS1    Binary           0.0000
CM_ANEMDEF1    Binary -0.5805   0.0000
CM_ALCOHOL1    Binary           0.0000
CM_DRUG1       Binary -0.0877   0.0000
CM_PSYCH1      Binary           0.0000
CM_DEPRESS1    Binary -0.4093  -0.0182

I get the same standard mean differences as MatchIt (called "Diff.Un" and "Diff.Adj" in the cobalt package), which is good, but still all the multi-categorical variables are missing the first level. Per the author of the package, this was supposed to have been fixed. Am I missing a parameter in the call to the bal.tab function?

Improve this question
$\endgroup$

1 Answer 1

Reset to default
1
$\begingroup$

I'm the author of cobalt. In short, it's because you're using your own distance variable. When you do this, matchit() doesn't save the original data set in the matchit output object; it only saves a version of the data set that has had model.matrix() used on it, which automatically separates the factors into dummies and, in MatchIt's implementation, will remove the first level of all but the first factor variable it sees. Unfortunately, there is no reliable way to recover the original factor variables in the original dataset from the output of the matchit object in this case, so cobalt just treats the variables in the matchit output as if they were binary variables unrelated to each other rather than as components of a factor.

One way around this is to use the data frame or formula methods for bal.tab, i.e., (replacing formula with your actual model formula)

bal.tab(formula, avgdata_selected, weights = get.w(matched_avgdata), 
        binary = "std", method = "matching")

I did think of another way to solve this problem, which is by allowing you to enter a data frame of the original covariates. I just updated the development version of cobalt so that you can run

devtools::install_github("ngreifer/cobalt") #Get development version
library(cobalt)
bal.tab(matched_avgdata, data = avgdata_selected, binary = "std")

That should give you the expected results. I'll make this clearer in the documentation.

Improve this answer
$\endgroup$
2
  • $\begingroup$ Hello Noah. Thanks so much for your incredible package and taking the time out to answer my questions. I just tried both methods - the first method (using the formula) worked, producing the SMDs for all levels. The second method (using the devtools) did not work, and the output was the same as before. $\endgroup$
    – somesurgeon
    Commented May 18, 2019 at 20:08
  • $\begingroup$ It works on my end, so maybe you need to restart your R session if you didn't do that. Either way, I'm glad the first method works as intended. Thank you for the kinds words about cobalt. $\endgroup$
    – Noah
    Commented May 18, 2019 at 22:44

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.