# After performing Matching w/ replacement, should tests to evaluate treatment effect between the matched groups include repeated subjects?

I am using Genetic Matching to infer causality from observational data. Because I am matching with replacement, the matched Control group has multiple instances of some of the same subjects. In this case, when I am performing t-tests/non-parametric tests after matching to determine if the groups are significantly different, should I use the matched Control group with multiple instances of the same subjects? Or should I take out any repeated measures before performing these tests? Leaving them in would ensure that both the treated and control groups have the same number of subjects. Removing repeated subjects, would mean that my control group is around 60-70% of the size of my treated group.

To clarify further, these tests I am referring to are not for evaluating post-matching balance. I am matching between 2 groups where the main difference is hours of therapy (Low Dose vs High dose) and I am using matching to ensure that these groups are similar in all demographic covariates (age, gender, etc)

Then after matching, I am performing t-tests/ non-parametric tests to see if the Low Dose and High Dose group-subjects are different in terms of their recovery (functional test scores. etc). But these post-matching tests are not being performed on the covariates that were used for matching

Yes, test should include the items that are picked multiple times due to matching (but read on). That is because in contrast with a standard "$$n_A$$ people in group A", "$$n_B$$ people in groub B" approach where we have to account unequal size test due, here we specifically paired each observation during matching. These repeated samples actually reflect different samples. That being said, do not use simple paired/matched $$t$$-test but rather look into procedure that account for the fact that this is sample was created by matching. Austin (2011) Comparing paired vs non-paired statistical methods of analyses when making inferences about absolute risk reductions in propensity-score matched samples gives a detailed discussion the subject. Effectively we would want to use the procedures described in Agresti & Min (2004) Effects and non-effects of paired identical observations in comparing proportions with binary matched-pairs data as the allow us to formally account for the loss of effciency.
• That would be comparing the null hypothesis that $s_1$ - $s_2$ comes from a distribution with zero median. I think using a covariate adjustment procedure would be more powerful but yes, on face value, a Wilcoxon would be fine. Apr 29, 2019 at 11:19