3
$\begingroup$

I've always heard that there isn't a lot of ways to diagnose the assumption that you've found the correct set of confounders in causal inference in confounder-control studies, and the best we can do is to incorporate methods that make the assumption more convincing but never rigorously prove. For example there is sensitivity analysis, which aims to show how strong unobserved confounding would have to be to meaningfully change your results.

Recently I came across a text that said that one way to diagnose this assumption in a matching framework is to run a regression on $P(T_i=1|Y_i,X_i)$ on your matched dataset ($T$ is the treatment, $Y$ is the outcome, and $X$ are your confounders), and if you know that your matching went well (with the recommended balance diagnostic methods), and the coefficient in the regression for $Y$ is close to 0, then it helps reason (but not prove, ofcourse) if the assumption of unconfoundedness is fair (compared to if you didn't try this at all).

I wanted to read more about this because there are questions I have (such as what other assumptions is the above method making? model assumptions? recommended thresholds?), but I couldn't find anything about it online, both on Google and CrossValidated. Maybe I'm just looking up the wrong things so I was hoping to ask here.

$\endgroup$
1
  • 2
    $\begingroup$ By definition, matching procedures cannot account for unobservable heterogeneity, which is the source of the claim that exogeneity (or unconfoundedness) cannot be tested or proved, but rather must be argued for, either using randomization or some observational variation that mimics randomization (through, for example, an instrumental variable). $\endgroup$ Commented Nov 10, 2023 at 19:26

1 Answer 1

3
$\begingroup$

This is a completely invalid method. $P(T|Y,X)$ is simply the association between $Y$ and $T$ given $X$, which is exactly the effect you are trying to estimate in the first place. The effect of $Y$ on $T$ will be 0 when the treatment effect estimate is 0. This says nothing about unmeasured confounding.

Where this misconception may arise is from an alternate statement of the assumption of no confounding, $$ P(T|X) = P(T|X, Y^1, Y^0) $$ which is simply a way of saying treatment and potential outcomes $Y^1$ and $Y^0$ are independent given the covariates. Usually this is written as $$ \{Y^1, Y^0\} \perp T | X $$ which says the same thing. However, both statements are true only of potential outcomes, not observed outcomes.

You might say to yourself, "the potential outcomes under treatment in the treated group are just the observed outcomes, so why can't I use this heuristic in the treated group and control groups separately to assess strong ignorability?" The problem is that in the treated group, everyone has $T=1$, so you can't predict $T=1$ from the observed outcomes $Y$ and $X$.

This assumption is fundamentally untestable from the data at hand without further assumptions. Do not use the method you describe.

$\endgroup$
3
  • $\begingroup$ Thanks for the clarification. I was iffy about it as well due to the association aspect as you described but was unsure because it came from what I thought was a reputable author in a fairly recent (2019) paper. $\endgroup$ Commented Nov 12, 2023 at 6:45
  • $\begingroup$ @user11513145 It would be great if you could post a link to that paper. $\endgroup$
    – Noah
    Commented Nov 12, 2023 at 6:51
  • 1
    $\begingroup$ here is the link: arxiv.org/pdf/1903.07755.pdf, it is on section 2.1.3 $\endgroup$ Commented Nov 12, 2023 at 7:20

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.