3
$\begingroup$

This is probably a very basic question. But I spent a good part of 2 weeks trying to understand this (by reading textbooks, searching on internet, listening to lectures) but haven't gotten anywhere close:

When I do a NNM to determine the causality of a treatment, would I not just get every treatment element matched with some control element (unless there are ZERO control elements)? In which case the matched data is basically the original data and any regression I perform after matching gives the same result as that of the regression I've done before matching. Or am I missing the whole point of matching?

Maybe related question: Is Nearest Neighbor Matching not a type of Propensity Score Matching?

Note: I am talking in reference to the matching results I get using 'teffects nnmatch' command in Stata. What I basically want is the SEs of the covariates, for which I thought regression post matching is appropriate.

$\endgroup$

1 Answer 1

4
$\begingroup$

If you are using teffects nnmatch or teffects psmatch, you are doing matching with replacement. This means for every treated unit, you find a control unit to pair with it, and control units can be used more than once. This means that only the control units that are paired with treated units are used to estimate the treatment effect, in contrast to your suggestion that all control units remain making matching pointless. You can think of matching with replacement as creating a new pseudo-dataset that only contains the treated and matched control units, where each control unit appears once per time it is matched to a treated unit (and therefore may appear more than once).

What I described above is for estimating the ATT; for the ATE, you do the same thing but find a treated unit to match with each control unit. Although you are retaining every treated unit and every control unit, you still will not get the same results as were you not to match at all because the pseudo-dataset contains not just the original treated and control units but also additional entries for each unit that is paired with each other unit.

One thing I have found that confuses some users is that the implementation of matching used in teffects is not the same application of matching described in papers by Austin or Stuart, which use the philosophy of matching as "nonparametetric precprocessing" to reduce model dependence and confounding, a philosophy described in detail by Ho, Imai, King, and Stuart (2007). Rather, teffects uses an application of matching sometimes called "matching imputation", which is a specific estimation method that involves imputing the missing potential outcomes using the paired units. Matching imputation is described by Abadie and Imbens (2006). If you read the documentation for teffects nnmatch, you will notice that no literature other than Abadie and Imbens's series of papers on matching imputation is included. I have written about this distinction here and here on the site.

This is critical to understand because matching imputation does not involve actually creating a matched dataset and then fitting a t-test or regression to it; rather, it involves estimating missing potential outcomes and then using a specific estimator to estimate the treatment effect and its standard error. The standard error is estimated in a highly unusual way by performing a second round of matching. It is not the same as the standard error resulting from regression, with or without robust standard errors. There is no outcome regression model involved at all unless you use the biasadj option, which uses an auxiliary outcome model to adjust the estimated potential outcomes. In the simplest of cases, the matching imputation estimator is equal to the difference in means in the pseudo-dataset I described above.

It sounds like your understanding of matching has come from resources that consider matching as nonparametric preprocessing (such as the original Rosenbaum and Rubin papers, papers by Austin, Stuart, and King, and, indeed, almost all treatments of matching outside econometrics, where the literature is instead dominated by Abadie and Imbens). This is not what teffects does, so all that you have read that implicitly uses this philosophy of matching is largely inapplicable (except insofar the matching algorithms are the same). Matching as nonparametric preprocessing is not implemented in Stata; you instead should use the R package MatchIt (note the R package Matching performs matching imputation as well).

To answer your second question of whether nearest neighbor matching is a type of propensity score matching: "nearest neighbor" describes the algorithm used to decide who gets matched to whom, and "propensity score" describes the way the distance between units is computed. There are algorithms other than nearest neighbor, such as optional matching, which creates pairs by minimizing the average distance between paired units across the whole matched dataset; this means that a unit may not get its closest match if it would minimize the global criterion for that unit to be matched with a different unit. (Note that when matching with replacement, nearest neighbor matching is already optimal.) There are distance measures other than the propensity score difference, such as the Mahalanobis distance (the default in teffects nnmatch) or the Euclidean distance. All matching done using teffects is nearest neighbor matching; the reason there is a separate command for propensity score matching is that propensity score matching involves other matters like the form of the propensity score model and requires adjustment of the standard error of the treatment effect for having estimated the propensity scores using a model.

Regarding regression post-matching, in matching imputation, as I mentioned before, this does not happen. When doing matching as nonparametric preprocessing, performing regression in the matched dataset is highly recommended as a way to improve the precision of the treatment effect estimate. However, it is critical that you do not interpret any coefficients in the outcome regression than that on the treatment. Doing so otherwise is to commit the table 2 fallacy (Westreich & Greenland, 2013). These other coefficients are meaningless; they do not represent the effects of the covariates on the outcome and cannot be used to make inferences or inform practice. Thus, you should not seek to find out what these coefficients are or what their standard errors are, and you should not report them in your paper.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.