1
$\begingroup$

In short, my question is:

Can group-level comparisons (such as t-test, Mann-Whitney tests, etc...) can be used after creating a pseudo-population using inverse probability weighting? (using propensity scores)

Longer version: Let's assume I am interested in comparing (e.g. t-tests) a control group with patients group, with respect to some continuous outcome, that I know also depends on a continuous covariate (e.g. weight/age/something similar).

After becoming desperate from trying to adjust for the covariate using linear regression, I thought of following a different path and just apply the group-level approaches on the pseudo population I obtain after IPW. The problem is that I couldn't find any paper following this approach. It seems like the use of IPW is limited to estimating average treatment effects, and not performing group level comparisons. Is there a reason for not using group level comparisons on the pseudo population obtained after IPW?

Thank you

Improve this question
$\endgroup$
5
  • $\begingroup$ Is the continuous covariate independent of the group? This would hold if, for example, the covariate was measured at "baseline" in an RCT. Or, is just that the continuous covariate is a confounder? $\endgroup$
    – Ben
    Commented Jul 25, 2022 at 19:39
  • $\begingroup$ This is not an RCT, as one of the groups is some patient group (of a disease). Assuming age is the covariate, I don't think we can assume the covariate is independent of the group. it is just a confounder (or maybe I don't understand what does it mean that a covariate is independent of the group). Thank you! $\endgroup$
    – Dr. John
    Commented Jul 25, 2022 at 20:32
  • $\begingroup$ OK, thanks. Do you believe that age is the only common cause of both the group and the outcome? $\endgroup$
    – Ben
    Commented Jul 25, 2022 at 20:37
  • $\begingroup$ @Ben: Yes. (I am at least assuming that I have all possible confounders measured and can be estimated in the propensity scores. ) $\endgroup$
    – Dr. John
    Commented Jul 25, 2022 at 20:51
  • $\begingroup$ Ben, by the way, since I am not an expert in this topic, if you'll also be able to explain a bit in your answer what would have changed if my answers were different it would be great. Do you maybe know any papers that used the approach I am referring to? Think it is doable? Or there's some fundamental aspect I am missing that doesn't make sense? Thank you! $\endgroup$
    – Dr. John
    Commented Jul 25, 2022 at 21:00

1 Answer 1

Reset to default
2
$\begingroup$

Group comparisons are average treatment effects. They generalize to the population from which the group was sampled, or equivalently, a population that resembles your sample (i.e., in terms of distributions of background characteristics). IPW is an alternative to regression adjustment for controlling for confounding by measured confounders like you have here. When you are comparing the IPW-weighted groups, you are estimating a treatment effect that generalizes to the same population that the unadjusted estimate does, except the estimate is free from confounding (by measured confounders [if done right]).

So, you can perform a "weighted t-test", which compares the weighted groups and estimates an average treatment effect. This weighted t-test compares the weighted means of the outcome in each treatment group. The cool thing about IPW is that the weighted mean of the outcome in the treated group is meant to represent the mean of the outcome in the full sample had everyone received treatment, and the weighted outcome in the control group is meant to represent the mean of the outcome in the full sample had everyone received control. You're still just comparing weighted group means, but the interpretation of these weighted means has a new causal flavor, representing the counterfactual means under each treatment for the whole population.

Rather than doing a weighted t-test, we usually run a weighted least squares regression of the outcome on the treatment, using the IPW weights as the weights and using a robust standard error. The coefficient on treatment in this model is equal to the treatment effect, the difference in weighted means. See here for a guide on doing IPW in R.

Improve this answer
$\endgroup$
3
  • $\begingroup$ Thank you for the details @Noah! I do have a few questions. Please forgive me if they are trivial: 1. Do you know of any papers that used this approach ("weighted t-test", "weighted Mann-Whitney test" or something similar)? Will be happy to see their approach 2. Do you maybe have any good reference for the use of weighted least squares regression with IPW weights as weights? Papers that use it and/or some theory and guidelines to go over? 3. do you have any recommendation on a book/paper that cover the applicative point of view of such an approach? Thank you! $\endgroup$
    – Dr. John
    Commented Jul 26, 2022 at 18:56
  • $\begingroup$ 1) No one uses weighted t-tests. There are specific procedures for performing IPW which are described in the link above. 2) The canonical reference is Robins et al (2000), though it is a little technical. I don't know many other good references though. Most papers focus on estimating or evaluating the weights. 3) What If by Hernán and Robins is excellent and a must-read. I recommend you consider taking a class to really understand these methods, e.g., this one. $\endgroup$
    – Noah
    Commented Jul 27, 2022 at 2:17
  • $\begingroup$ Thank you Noah! If you will recall any other reference please let me know. $\endgroup$
    – Dr. John
    Commented Jul 27, 2022 at 6:35

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.