5
$\begingroup$

I am quite new to the topic and trying to figure out a workflow for causal analysis. My aim is to establish a baseline of ATE (I think) and then experiment with disentangled representations and machine learning (e.g CEVAE). I am mainly going to use the Lalonde [A] dataset and the IHDP [B] simulated dataset by Hill and try to re-implement CEVAE. My questions towards folks who have an existing workflow and use causality for their research are as below:

  1. In studies where one only has observational data and the causal graph is intractable (is it?) are ATE, ATT and ATC the goals of the analysis to estimate causal effects?
  2. In order to compute the above metrics I need to find a way to compute counterfactuals. Does the following approach make sense: a) identify a method of counterfactual estimation (e.g linear regression) b) compute counterfactuals c) calculate metrics (e.g ATE) ?
  3. If so can you suggest examples with code (python preferably) and other methods more suitable than linear regression?
  4. Finally, the DoWhy library suggest an extra step of refutation? How do I verify that my counterfactual estimation method is robust? What is the theory? Many repetitions?

I have already looked at several tutorials and packages like DoWhy (Example with Lalonde dataset-DoWhy) and what baffles me is the fact that no one seems to explicitly compute counterfactuals and use them as an estimated quantity for following experiments. Also, this question seems relevant but does not really answer mine: Using counterfactual modeling techniques to assess racial bias in predictive models . I would be grateful if anyone could suggest worked examples rather than plain theory or Pearl's book. Thank you.

[A] R. J. LaLonde. Evaluating the econometric evaluations of training programs with experimental data. The American economic review, pages 604–620, 1986.

[B] J. L. Hill. Bayesian nonparametric modeling for causal inference. Journal of Computational and Graphical Statistics, 20(1):217–240, 2011.

Improve this question
$\endgroup$
7
  • 2
    $\begingroup$ Hi Nikos H., welcome to CV! The "and the causal graph" in step 1 seems a smidge glib: the point of counterfactual formal causal inference is that (a) causal graphs articulate beliefs about causal relationships, and (b) these beliefs have critically important consequences for causal inference. So I would urge you consider how you form causal graphs, what you do with competing causal graphs, and how strong and poor correspondence in ATE estimates with observed data informs your future beliefs about causal graphs. This kind of inference is about much more than statistical estimation problems. $\endgroup$
    – Alexis
    Jun 29, 2021 at 17:38
  • 1
    $\begingroup$ Hi @Alexis thank you for the comment. I am not sure if I get this right. Do you mean that I should first, try and estimate the causal graph? Since I am working with observational data and I don't know the true causal path, how do I approach this? The first paper I link just makes an assumption of the overall causal graph and continues with the analysis, the authors do not try to do causal discovery. $\endgroup$
    – Nikos H.
    Jun 29, 2021 at 17:47
  • 1
    $\begingroup$ There are basically two ways to get your causal graph, that I know of. The first is expert opinion. The second is the algorithm Pearl outlines in Causality: Models, Reasoning, and Inference, p. 50. I don't know what software there is, though, that implements that algorithm. $\endgroup$
    – Adrian Keister
    Jun 29, 2021 at 18:01
  • 1
    $\begingroup$ Thank you @AdrianKeister . I see that this algorithm requires a partially discovered DAG and absence of unmeasured confounders. Therefore, would it make sense to say that the causal graph is going to depend on which assumptions I impose to the initial system? $\endgroup$
    – Nikos H.
    Jun 29, 2021 at 18:12
  • 2
    $\begingroup$ Definitely agree with Alexis. I would also add that it's possible to test a candidate DAG against your data, modulo the $v$-structures (colliders) and skeleton. That is, if the $v$ structures and the skeleton are the same, then the data cannot distinguish between two DAGs as to whether one fits the data better than the other. That's also a theorem in Pearl's book. $\endgroup$
    – Adrian Keister
    Jun 29, 2021 at 19:22

0

Reset to default

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.