2
$\begingroup$

Observational causal models such as causal forest and orthogonal/double machine learning requires the unconfoundedness assumption, i.e. there's no hidden confounders affecting both the treatments and the outcomes. To me, this seems a strong assumption to have in real world application. I feel that the best one could claim is that "at the best of our knowledge, there's no other confounder" and that surprising hidden factors will almost always pop up over time.

  1. What would these models behave if there are unknown, unaccounted for confounders (that affecting both treatment and outcome)?
  2. And what if there are unknown factors that affect either the treatment or the outcome but not both?
  3. What if we know the possibility of some factors but don't have the actual observational data about them and so can't incorporate them into the model?

All discussions I read only state the requirement of this assumption without elaborate on the effect of violating it.

$\endgroup$
2
  • $\begingroup$ Is an unknown confounder a variable you have but don't know if it confounders, or is it a variable known to be a confounder that you just haven't measured? $\endgroup$
    – AdamO
    Jan 12, 2023 at 18:33
  • $\begingroup$ @AdamO all of these scenarios would apply. I don't think one can enlist all weird surprising causes even though intuitions/common sense likely tell us something else may impact the decision of the subject taking a treatment. The 3rd question addresses the known but unmeasured confounders which could also happen especially when dealing with existing observational data. $\endgroup$
    – teddy
    Jan 12, 2023 at 19:24

1 Answer 1

4
$\begingroup$

An "unknown" confounder is a variable you've measured but you're unsure of the directionality of causation - is a time lagged mediator actually a confounder? Does history of birth control usage confound or mediate the causal relationship between maternal age at conception and incidence of pre-eclampsia? Regrettably what I've noticed in the literature is that many authors fit the model "both ways" and, if the "confounder" seriously changes the estimate, they declare that model the correct one. There are many reasons why this doesn't work - you might qualitatively exclude that variable from being a collider or a mediator, but to accept it as a confounder because of the quantitative result is an inconsistent approach to evaluation. Also, logistic models are prone to change covariate estimates when adding variables even when they are unrelated to the main predictor - a feature called non-collapsibility.

You seem to be talking about unmeasured confounders, meaning you don't actually have the variable (and you'd really like to) in your analysis dataset.

My practical experience with this matter is that it's very subjective. From the perspective of writing a paper or a grant, you simply point out that it's a limitation, and provide an overview of the problem: did other studies adjust for this value? what's a reasonable projection on the impact to analyses? How might others build on your work in the future to account for the data you did not get?

We also care about BIG confounders versus small ones. A "big" confounder is (roughly) highly predictive of the response and highly correlated with the response conditional on other variables in the model. This type of discussion is meant to engage the content experts more than the statisticians. I believe a reasonable statistician's perspective is that one can't theoretically adjust for all confounding values, and so observational studies need to be taken with a grain of salt. The belief in the results is strengthened with independent confirmation of the results, and interventions based on the results that provide an expected reversal of the trend in the outcome.

$\endgroup$
2
  • $\begingroup$ thanks for the reply. By "unknown", I originally meant something we are not aware of. Though the "unknown direction" problem would also arise in a study. Can graph learning be able to help us identify the edge and the direction? $\endgroup$
    – teddy
    Jan 12, 2023 at 19:34
  • $\begingroup$ @teddy "not being aware of" and "being unknown" are both equally ambiguous. "Graph learning" can mean 100 different things, but as an exploratory analysis technique, it won't solve any practical problems with causality. $\endgroup$
    – AdamO
    Jan 12, 2023 at 21:43

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.