1
$\begingroup$

I am using Genetic Matching to infer causality from observational data. Because I am matching with replacement, the matched Control group has multiple instances of some of the same subjects. In this case, when I am performing t-tests/non-parametric tests after matching to determine if the groups are significantly different, should I use the matched Control group with multiple instances of the same subjects? Or should I take out any repeated measures before performing these tests? Leaving them in would ensure that both the treated and control groups have the same number of subjects. Removing repeated subjects, would mean that my control group is around 60-70% of the size of my treated group.

To clarify further, these tests I am referring to are not for evaluating post-matching balance. I am matching between 2 groups where the main difference is hours of therapy (Low Dose vs High dose) and I am using matching to ensure that these groups are similar in all demographic covariates (age, gender, etc)

Then after matching, I am performing t-tests/ non-parametric tests to see if the Low Dose and High Dose group-subjects are different in terms of their recovery (functional test scores. etc). But these post-matching tests are not being performed on the covariates that were used for matching

|cite|improve this question
$\endgroup$
1
$\begingroup$

Yes, test should include the items that are picked multiple times due to matching (but read on). That is because in contrast with a standard "$n_A$ people in group A", "$n_B$ people in groub B" approach where we have to account unequal size test due, here we specifically paired each observation during matching. These repeated samples actually reflect different samples. That being said, do not use simple paired/matched $t$-test but rather look into procedure that account for the fact that this is sample was created by matching. Austin (2011) Comparing paired vs non-paired statistical methods of analyses when making inferences about absolute risk reductions in propensity-score matched samples gives a detailed discussion the subject. Effectively we would want to use the procedures described in Agresti & Min (2004) Effects and non-effects of paired identical observations in comparing proportions with binary matched-pairs data as the allow us to formally account for the loss of effciency.

|cite|improve this answer
$\endgroup$
  • $\begingroup$ The first paper you mentioned states "We recommend using statistical methods for paired samples when using propensity-score matched samples for making inferences on the effect of treatment on the reduction in the probability of an event occurring." I was reading through both papers and I didn't understand all of it, but would you say it would be correct to use a non-parametric paired test like the Wilcoxon signed-rank test to evaluate if the 2 samples (treatment vs control) are different in this case? $\endgroup$ – stats_nerd Apr 29 at 9:49
  • $\begingroup$ That would be comparing the null hypothesis that $s_1$ - $s_2$ comes from a distribution with zero median. I think using a covariate adjustment procedure would be more powerful but yes, on face value, a Wilcoxon would be fine. $\endgroup$ – usεr11852 Apr 29 at 11:19
  • $\begingroup$ Thank you again. Just to make sure I understood correctly, when you mention a covariate adjustment procedure, do you mean something like fitting a linear regression model with therapy dosage (low/high dose) as the independent variable and the outcome measure (patient recovery) as the dependent variable, and then evaluating if the slope parameter is significant? $\endgroup$ – stats_nerd Apr 29 at 20:30
  • 1
    $\begingroup$ Yes, that's the idea! $\endgroup$ – usεr11852 Apr 29 at 22:06
  • 1
    $\begingroup$ No problem at all; I would go with "b" because it would increase power. That being said, you should consider creating a separate question about this as a one-sentence response does not do the question justice. :) $\endgroup$ – usεr11852 Apr 30 at 22:53

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.