Why is ANCOVA not appropriate for modelling post-intervention outcome, controlling for baseline I was trying to understand the different use-cases for differences-in-differences models vs. ANCOVA (post-period =  pre-period + experiment_group), for observational data.  I came across the below article, which starts with:
"When comparing pretest to posttest changes in non-randomized groups, most researchers are correctly avoiding ANCOVA with posttest as the dependent variable and pretest as the covariate. "
https://www.sciencedirect.com/science/article/abs/pii/S0167876003002757
Why should that approach be avoided?  I have used that approach in the past to control for  regression to the mean effect, in models where we are trying to understand which behaviors and covariates are associated with different changes (improvements/worsening) in clinical outcomes.
Also, if anyone has thoughts on the DiD vs. ANCOVA for large samples, I would love to hear!
 A: This is explained very clearly in Lüdtke and Robitzsch (2020). I'll briefly summarize their arguments below but the paper is clear and easy to read, and I recommend you read it.
Essentially, the change score approach requires two assumptions to be unbiased: 1) the effect of the unmeasured confounders on the outcome is consistent across time points and 2) the pre-treatment outcome does not cause selection into treatment after conditioning on the unobserved confounders; that is, the pre-treatment outcome is an indicator of the unmeasured confounders but itself is not a confounder.
The ANCOVA approach is unbiased when the pre-treatment outcome completely mediates the relationship between the unmeasured confounder and the outcome. This would occur when selection into treatment depends only on the pre-treatment outcome (and possibly other factors unrelated to the post-treatment outcome).
So, the question is, is the pre-treatment outcome simply a proxy for the unmeasured confounders or itself part of the process of treatment selection? If the answer is some of both (which is likely), neither method will be unbiased alone. Other methods of adjusting for confounding, such as including covariates in the outcome model or preprocessing the data with matching or weighting can increase the possibility of the required assumptions being true. Fundamentally, the assumptions are untestable, but substantive knowledge of the treatment selection process can go a long way.
