I am carrying out an evaluation for an intervention using secondary data. Unfortunately the design of the study is weak as participants cannot be randomised. I am looking to see whether the intervention has had any impact on patients quality of life? A measure of QoL has been taken before and after the intervention, i am also interested in a number of variables and their potential effect on reported QoL including: age, gender, pharmacological treatment, disease duration, mariatal status, and employment status(re literature). These IV's are both continuous and categorical, but can be all be converted to categorical if needs be. I would appreciate your advice concerning which design is best suited to this piece of research. I am considering using a ANCOVA but my experience with violations of assumptions may further weaken the merit of an already compromised design. Warm Regards Dara
QOL should be treated as ordinal in most cases, so the proportional odds model is a good candidate for ANCOVA. It is seldom a good idea to analyze change (you didn't suggest that it was but many do), so baseline QOL can be a covariate in the prop. odds ordinal logistic model. If QOL has only a handful of levels you can represent the baseline variable as categorical with multiple dummy variables.
If treatment was not randomized you may need to aggressively adjust for confounding. If you don't have the sample size to support simultaneous modeling of all potential confounders, the propensity score approach may be used in conjunction with the prop. odds model.
For anyone who stumbles upon this question and is interested in determining whether for their type of non-randomized study design, ANCOVA or change scores (or a comparison of both) is more suitable, I would recommend reading this paper (van Breukelen, 2013):
As Frank Harrell says, one has to be careful with change scores in general. As detailed in the paper, however, for the specific case of "preexisting groups"(i.e. non-random/true/explainable differences between groups at baseline), they will usually be less biased than ANCOVA.
A good read on Lord's Paradox (which is underlying this) in general is this paper (Pearl, 2016):
Pearl's first example is equal to van Breukelen's case of preexisting groups. Pearl's second example is equal to van Breukelen's case of treatment selection based on baseline.