I am carrying out an evaluation for an intervention using secondary data. Unfortunately the design of the study is weak as participants cannot be randomised. I am looking to see whether the intervention has had any impact on patients quality of life? A measure of QoL has been taken before and after the intervention, i am also interested in a number of variables and their potential effect on reported QoL including: age, gender, pharmacological treatment, disease duration, mariatal status, and employment status(re literature). These IV's are both continuous and categorical, but can be all be converted to categorical if needs be. I would appreciate your advice concerning which design is best suited to this piece of research. I am considering using a ANCOVA but my experience with violations of assumptions may further weaken the merit of an already compromised design. Warm Regards Dara
2 Answers
QOL should be treated as ordinal in most cases, so the proportional odds model is a good candidate for ANCOVA. It is seldom a good idea to analyze change (you didn't suggest that it was but many do), so baseline QOL can be a covariate in the prop. odds ordinal logistic model. If QOL has only a handful of levels you can represent the baseline variable as categorical with multiple dummy variables.
If treatment was not randomized you may need to aggressively adjust for confounding. If you don't have the sample size to support simultaneous modeling of all potential confounders, the propensity score approach may be used in conjunction with the prop. odds model.
-
2$\begingroup$ +1, Why do you disagree w/ change scores? Wainer, eg, has advocated them as more likely answering the question people have in mind than using a baseline covariate. $\endgroup$ Commented Apr 16, 2012 at 13:18
-
$\begingroup$ I'd be interested in your comments on stats.stackexchange.com/questions/17724/… . I think this change-score-vs.-covariate issue has been debated for a long time without a clear "victory." $\endgroup$– rolando2Commented Apr 16, 2012 at 13:33
-
$\begingroup$ Change scores are simply a dreadful way to think about this. The easiest way to see the problem is to note that the difference between two ordinal scales is no longer ordinal. For more information see biostat.mc.vanderbilt.edu/MeasureChange $\endgroup$ Commented Apr 16, 2012 at 15:29
-
$\begingroup$ Another thread dealing with this in greater depth is stats.stackexchange.com/questions/3466/… $\endgroup$– rolando2Commented Apr 21, 2012 at 11:07
For anyone who stumbles upon this question and is interested in determining whether for their type of non-randomized study design, ANCOVA or change scores (or a comparison of both) is more suitable, I would recommend reading this paper (van Breukelen, 2013):
https://doi.org/10.1080/00273171.2013.831743
As Frank Harrell says, one has to be careful with change scores in general. As detailed in the paper, however, for the specific case of "preexisting groups"(i.e. non-random/true/explainable differences between groups at baseline), they will usually be less biased than ANCOVA.
A good read on Lord's Paradox (which is underlying this) in general is this paper (Pearl, 2016):
https://econpapers.repec.org/article/bpjcausin/v_3a4_3ay_3a2016_3ai_3a2_3ap_3a0_3an_3a5.htm
Pearl's first example is equal to van Breukelen's case of preexisting groups. Pearl's second example is equal to van Breukelen's case of treatment selection based on baseline.
