Merits of different matching & weighting methods for multiple treatments

Question

I have three land use classes (natural farming, chemical farming, and forest) and would like to compare the densities of different bird species between them. I would like to get the ATE (I think). I don't want to remove any samples as I have a small dataset, but I do want to ensure that the covariates (such as altitude, soil type, temperature etc.) are well-balanced. Whilst reading some of the literature on matching and weighting, the following questions arose:

1. Having multiple treatments seems to be more of an issue for matching than for weighting. Why is that?

2. Greifer & Stuart 2021 suggest using inverse probability weights when ATE is the estimand. Why can't matching weights be used? i.e. what is the difference between inverse probability weights and matching weights? This is in light of some criticism of propensity scores for matching and the suggestion that Mahalanobis may be preferable. Does the same criticism not apply to weighting?

3. The same paper doesn't mention genetic matching for ATE, but this is the method tentatively recommended in the conservation science literature (that is trying to make sense of the stats literature).

Thank you!

Answer 1

1. When most people think of matching, they imagine pairing, e.g., 1:1 nearest neighbor matching without replacement, which is by far the most popular matching method despite its relatively poor performance. With multi-category treatments, it becomes a challenge to find, e.g., triplets of units that are close to each other. It is ambiguous how to define the distance among more than two units, and it is a challenge to handle situations where, e.g., a pair of units from two of the treatment groups are close to each other but there isn't a close unit from the third group. Some attempts are described in Lopez and Gutman (2017). Any method that relies on distance matrices instead has to rely on a distance array, which increases in size exponentially and makes problems that rely on them intractable. There has been little research on matching for multi-category treatments, but it is quite different from matching with binary treatments. In contrast, all the theory used for weighting with binary treatments applies to weighting with multi-category treatments, and the computation of weights is simple. All weighting methods developed for binary treatments can be used with multi-category treatments.

2. Matching weights are the same as inverse probability weights! The difference is in the population they refer to (the estimand). Pair matching methods cannot target the ATE, but full matching and stratification methods like propensity score subclassification can, and should definitely be used! Both of these methods can be seen as nonparametric ways to estimate the propensity score for use in IPW. You can use distances other than the propensity score difference for full matching, and there has been some development in using the prognostic score for subclassification, which avoids some of the problems with the propensity score. That said, every problem King and Nielsen (2019) mention can be assessed empirically in your own dataset; you don't need to take their word for it and it's certainly possible for propensity score matching to be the best method for you. Their point is not to rely on it blindly, which is what most researchers do. I discuss their paper in this post and the performance of Mahalanobis distance matching in this post.

3. Genetic matching is not really a matching method; it's a way of computing the distance between units and can theoretically be used with any matching method, including one targeted for the ATE like full matching. I discuss this in detail on my blog here. There are ways to do pair matching for the ATE, but this involves a method called matching imputation, which is implemented in the Matching package and Stata teffects nnmatch. This is different from most other uses of matching and is rarely used outside economics. It does allow you to estimate the ATE, but you have to use matching with replacement and can only estimate the difference in means for a continuous outcome for the inference to be valid. The other philosophy of matching used by, e.g., MatchIt, and discussed in Greifer and Stuart (2021) is matching as nonparametric preprocessing, which does not allow you to estimate the ATE using pair matching. Again, other forms of matching can be used to estimate the ATE, and genetic matching is theoretically compatible with them.