9
$\begingroup$

I'm using a package Matchit in R to do matching. Below is a sample output from a method: Coarsened Exact Matching (cem).

Can someone tell me what Matched (ESS) means in the summary() when I use method = "cem"?

Sample sizes
                      Control Treated

All                  37032.      4010

Matched (ESS)         6012.67    3127

Matched (Unweighted) 21506.      3127

Unmatched            15526.       883

I understand weighting is used in cem. But how come it is not the same as Treated (3127 in my example)?

I also see that when I use match.data(), weights and subclass are two new columns. What do they mean? I notice weight=1 for the Treated case. I think that makes sense. But the sum of the weights for the Control case is 21506, not 3127. Why is that?

Improve this question
$\endgroup$
0

2 Answers 2

Reset to default
7
$\begingroup$

The effective sample size (ESS) is the size of an unweighted sample carrying approximately the same precision as the weighted sample in question. The formula for the ESS is $$ESS = \frac{(\sum w)^2}{\sum w^2}$$ When the weights are scaled to have an average of 1, the formula can be rewritten as $$ESS = \frac{n}{1+\text{Var}(w)}$$ where $n$ is the sample size and $\text{Var}(w)$ is the variance of the weights (computed using the population formula). The second formula makes it clear that when there is variability in the weights (i.e., the weights are not all the same), the ESS will be smaller than $n$. This is described in the documentation for summary.matchit(), which can be viewed here (7th paragraph in Details).

In coarsened exact matching (CEM), a formula is used to compute the matching weights after the matching. The weights that arise from this formula are different for each unit and depend on how many units of each treatment group are in each matching stratum. This is explained in the matchit() documentation here (section "How Matching Weights Are Computed" in Details). The reason the ESS is not equal to the size of the treated sample is that CEM does not involve a 1:1 match of treated to control units. The ratio of treated to control units differs across strata. If you want 1:1 matching after CEM, set k2k = TRUE. Doing this drops units in each stratum so that the number of treated and control units is equal. With this option, all weights will be 1, the ESS will be equal to the sample size, and the ESS of the two groups will be the same. In your dataset, there would be no advantage to doing this.

The match.data() output contains the data from the matched sample as well as columns related to the output of the matching. weights contains the matching weights. These must always be used in estimating the treatment effect; the point of matching is to create matching weights, which create covariate balance and reduce the bias in an effect estimate. subclass contains stratum membership, i.e., the ID of the stratum to which each unit is assigned. In 1:1 matching, each stratum will contain two units: one control and one treated. In CEM, varying numbers of treated and control units may appear within strata. The strata are used in the estimation of the standard error of the effect estimate. See the vignette on how to estimate effects after matching for how to use these columns. You'll want to refer to the section "After Stratum Matching" for estimating effects after CEM.

Thank you for letting me know that it is unclear what those columns mean. I will update the documentation to be clearer. I encourage you to browse the MatchIt website as I have spent a considerable amount of time documenting every facet of MatchIt and providing instructions for how to use it validly.

Improve this answer
$\endgroup$
1
  • $\begingroup$ Thank you, @Noah. I actually used your "cobalt" package for my analysis. $\endgroup$
    – Ka Lee
    Commented Dec 5, 2020 at 16:49
1
$\begingroup$

ESS is the Effective Sample Size:

#Effective sample size
ESS <- function(w) {
  weights <- weights * s.weights
  n <- matrix(0, ncol=2, nrow=6, 
              dimnames = list(c("All (ESS)", "All", 
              "Matched (ESS)","Matched", "Unmatched","Discarded"),...

The treatment weight of 1 is why both the "unweighted" and ESS are 3127. If you had another question please clarify it.

These resources may help:

https://www.jstatsoft.org/article/view/v042i08

https://gking.harvard.edu/cem

https://github.com/kosukeimai/MatchIt/

Improve this answer
$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.