2
$\begingroup$

We have conducted a trial with 30 participants, each participant given different intervention (Intervention A, Intervention B and Intervention C). The aim of the study is to compare the effectiveness of three interventions.

At baseline (T0), 6 weeks (T1) and 12 weeks (T2) participants assessed their conditions using a 5 point likert scale (1. No pain; 2.Slight discomfort; 3.Slight pain, 4.moderate pain, 5.severe pain)

I have attempted to analysis the outcomes using Kruskal-Wallis and ANCOVA on different occasions however: (1) Kruskal-Wallis is not appropriate as it looks at the degree of change from baseline to follow-up within each group, rather than comparing the findings across the groups at follow-up, adjusting for any baseline differences; (2) ANCOVA considers for baseline differences however it also treats the data as continuous variable.

Any advice would be highly appreciated.

Improve this question
$\endgroup$
1
  • $\begingroup$ Use Repeated Measures ANOVA, with timepoint as within-subject variable. $\endgroup$ – Joseph Apr 23 '18 at 11:23
2
$\begingroup$

Your situation is ideal for the generalization of the Kruskal-Wallis test: the proportional odds ordinal logistic model. With that model you can adjust for covariates, avoid problematic change scores, and analyze the ordinal response directly while handling even extreme clumping in the data. The prop. odds model has as many intercepts as you have distinct values of $Y$, less one. For a start see my RMS course notes. But this deals with the non-repeated-measures part of your issue. To handle repeated measures you have three choices: Bayesian hierarchical prop. odds model (e.g.. R brms package), mixed effects classical proportional odds model (e.g., R ordinal package), or fit the regular prop. odds univariate model on a tall and thin dataset and correct after-the-fact for within-subject correlation (e.g., R rms package lrm, orm, and robcov functions).

Improve this answer
$\endgroup$
0
$\begingroup$

For the classical ANOVA you have to assume normally distributed errors. However, with an ordinal outcome variable you will not get such an error distribution by definition. This may not be a big problem, depending on the sample size the ANOVA may be quite robust to such violations of that assumption. As Joseph pointed out: it is much more important to account for the fact that the data-points at T0-T3 are not independent by using a RM-ANOVA. However, there is a way to do better and get around this limitation: Use a permutation test. You do basically create a loop that performs the ANOVA say 10000 times, and each time you randomly shuffle treatment labels for each subject (BUT NOT TIMEPOINT!). This way you will get a distribution of values (e.g. F-values) under the null hypothesis. You then can compare your actual ANOVA results to this distribution and determine how extreme your original values are on this distribution. Check out a permutation test tutorial (e.g. https://www.tandfonline.com/doi/abs/10.1080/15374410902740411?journalCode=hcap20), its easier than you may think and it will resolve potential problems with supervisors/reviewers.

Improve this answer
$\endgroup$
2
  • 1
    $\begingroup$ Permutation tests based on classical statistics are in fact not very robust. For example, they don't deal correctly with excessive ties or skewness in the response variable. This non-robustness affects type II error even more than type I error. $\endgroup$ – Frank Harrell Apr 23 '18 at 12:00
  • $\begingroup$ I wasn't aware of that, but after some reading I agree. There is a dissertation with simulations that supports your statement at digitalcommons.wayne.edu/cgi/… $\endgroup$ – mzunhammer May 4 '18 at 20:21

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.