Let's start by defining some terms. Bias is the average distance from the true parameter of effects estimated from an estimator across many repeated samples. A biased estimate is an estimate coming from an estimator that yields nonzero bias. Variance is the typical distance of each estimate from an estimator to the average of those estimates across many repeated samples. Error is the distance between a single estimate and the true parameter. The average error of an estimator (across many repeated samples) is a combination between its bias and its variance.
You want an effect estimate that is unbiased and has low variance (i.e., high precision). This ensures that the estimate in your sample has as little error as possible and that your confidence interval for the effect is as narrow as possible while remaining at the nominal level of coverage. If your DAG is true, finding the minimally sufficient adjustment set can potentially help in arriving at an unbiased estimate of the effect, but it doesn't necessarily help you arrive at an estimate with the lowest error (i.e., distance from the truth). In addition, the DAG tells you which variables to control for, but not how to control for them; you can still have a biased effect estimate if you incorrectly model the relationships among the adjustment variables, treatment, and outcome.
In essence, DAGs tell you about identification, not about estimation performance. There may be estimators that rely on adjustment sets that are not sufficient to remove confounding but have lower error than estimators that rely on sufficient adjustment sets. A few examples come to mind: conditioning on many near-instruments that are actually confounders will increase the variance of the effect estimate, while not conditioning on them, i.e., only conditioning on strong predictors of the outcome, might yield an estimate with lower error because it has lower variance, even if it's a little biased. In the case of butterfly bias, there may be no sufficient adjustment set, and yet one can arrive at a reasonably low-error estimate by conditioning on the collider (even though doing so induces bias). There are so many possibilities that it's impossible to enumerate them, and no DAG can distinguish among them.
There are a few considerations that have been agreed upon:
- Conditioning on confounders reduces bias
- Conditioning on instruments and near instruments may not affect bias, but will increase variance (and in the case of unobserved confounding, will increase bias)
- Conditioning on strong predictors of the outcome, even if they are not confounders, decreases variance without affecting bias.
A minimally sufficient adjustment set might have many near-instruments and will definitely not include predictors of the outcome that are not confounders, making it potentially a sub-optimal set to condition on. On the other hand, conditioning on several strong predictors of the outcome, even if they are not confounders, but missing some true confounders that have weak effects on the outcome, may yield an estimate that has low error due to its low variance, even if some bias is allowed in.
The are variety of variable selection and modeling techniques available to estimate treatment effects, taking into account the relative performance of different types of variables. See my answer here for some examples of contemporary methods.
If you get two estimates resulting from conditioning on different minimally sufficient adjustment sets, there are many possibilities that cannot be distinguished from each other. The first is that both estimates are unbiased, but one or both of them has high variance, so one or both of them have high error. You can check to see which one you'd expect to have lower error based on the considerations mentioned above, and favor that one. It will probably have a narrower confidence interval. The second is that bias is induced by failing to correctly control for the variables in the adjustment set. Here, you may have successfully chosen the correct variables to control for, but you incorrectly modeled the relationships among the adjustment set, the treatment, and the outcome. You can avoid this by using a technique that tries to get around this problem by using a flexible model. The third is that the DAG is wrong in some way, and one or both of the adjustment sets is incorrect. Here, it may still be possible that the incorrect adjustment set yields an estimate with lower error; it's impossible to know. The variety of adjustment sets act as sensitivity analyses for each other, and if they all yield different conclusions, then there is not enough information available to you to make a substantive conclusion (i.e., analogous to not having a large enough sample to distinguish between a true effect and sampling error).
To summarize: Minimally sufficient adjustment sets (and sufficient adjustment sets in general) don't tell you which estimates will have the lowest error; they only tell you which variables are required for nonparametric identification of a causal effect. Estimating an effect with low error requires many different considerations, some of which are apparent from a DAG and others which are not. Knowing a sufficient adjustment set doesn't mean you know how to control for the variables to arrive at an unbiased effect estimate. Even unbiased effect estimates may have high error due to having high variance. There are variety of reasons why estimates might differ from each other, and no good ways to adjudicate among them.
