1
$\begingroup$

If t-tests show that there is no significant difference between the control and treatment groups, is there a need to do a propensity score matching?

Thank you, =sa

$\endgroup$
1
$\begingroup$

A t-test should not be used for assessing balance. If you're performing randomization-based inference on your data, then you can use permutation tests as described in Hansen & Bowers (2008). More likely, if you're performing standard population-based inference, you need to assess balance using methods that do not depend on the sample size of the data. Sample size does not relate to the bias of an estimated treatment effect, but it does affect the assessment of balance when you use a t-test, which makes using t-tests inappropriate for assessing balance.

Read Ho, Imai, King, & Stuart (2007) who explain how to assess balance and why you should avoid using hypothesis tests to do so. You should check balance not just on the means (which is all that a t-test does) but also on the means of the squared and cubic terms and the interactions among variables. All this can be done easily in the cobalt R package (which I wrote for this purpose). If you only have slight imbalances in the covariates, you should control for them using regression rather than propensity score matching to retain as big a sample as you can. Propensity score matching can needlessly throw away information that could be useful in your outcome model.


Hansen, B. B., & Bowers, J. (2008). Covariate Balance in Simple, Stratified and Clustered Comparative Studies. Statistical Science, 23(2), 219–236. https://doi.org/10.1214/08-STS254

Ho, D. E., Imai, K., King, G., & Stuart, E. A. (2007). Matching as Nonparametric Preprocessing for Reducing Model Dependence in Parametric Causal Inference. Political Analysis, 15(3), 199–236. https://doi.org/10.1093/pan/mpl013

$\endgroup$
  • $\begingroup$ thank you very much for your thorough response. I will give a careful review to the sources you mentioned as they seem to provide an excellent road map to tackling my question. $\endgroup$ – IKA Aug 1 at 18:55

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.