How to evaluate validity about Causal Discovery? When I was trying to do structural learning for causal inference, I perplex for too many causal discovery algorithms(PC, FCI, GES, NOTEARS and more...)
There are many structural learning algorithm for causal discovery.
For example, if you get two different structures from many algorithms, which one is best? which one is similar with real causal relation?
And, if you get only one structures from many algorithms, how to know it is similar with real causal relation?
 A: Often, you don't even get a complete structure, with constraint-based methods like PC or FCI, it is usually only a Markov Equivalence Class (MEC), which is a set of several possible structures (causal graphs).
And since those algorithms compute the MEC from noisy data, they can come up with different results. E.g., if they are based on conditional independence tests, those are quite difficult to get right, they prefer larger amounts of data, and the inferred independence relations often contradict each other.
I don't think that one can say that one of those algorithms is generally better than the others, the difference is more in what they are capable of. E.g. FCI can deal with latent confounders while PC cannot.
If it makes sense to use score-based methods, you have at least the score to guide you with your decision.
Note that there are a lot more possible approaches, like constraint-based, score-based, inequality-based, additive noise model based, ... And if you are willing to apply several of those, you might want to go for a majority vote, or some other, more sophisticated, ensemble method.
Once you have lowered the number of candidates, there is also the possibility of applying some model selection technique, e.g. the one described in this paper.
In general, it is often impossible to obtain the correct causal structure just from observed data. And one would require intervention data to properly check the results.
