I've been asked to analyse some data and I'm struggling to get my head round it. 14 people in a therapy group were rated on a questionnaire containing 16 6-point Likert scale questions when they started. The same questionnaires were administered 4 months later. So I'm looking a pre- (or quasi) experimental design. The independent variable is the therapy and the outcome in the score on the measure.

How do I analyse differences in the scores across the two time points? Can I implement a Wilcoxon rank signed test?

  • $\begingroup$ Try A Diff-in-Diff-Approach $\endgroup$
    – Ferdi
    Commented Dec 31, 2016 at 20:35

1 Answer 1


The pre-post design is perhaps the weakest of all study designs and should only be used with severe trepidation. You have to ensure the lack of time trends that could explain the pre-post difference. And the design is so brittle that in many cases you cannot afford a single missing observation, e.g., a pre-observation that has no post-observation.

Unlike the Wilcoxon two-sample test, the Wilcoxon signed-rank test for paired data gives different results for different transformations of Y. A better test is the Wilcoxon rank difference test discussed here.

  • $\begingroup$ Do you recommend instead something like a BACI design i.e. with a control group that does not get the treatment? Or are there better options than that? $\endgroup$
    – mkt
    Commented Jul 25, 2023 at 12:01
  • 2
    $\begingroup$ A randomized trial with a control group that does not get the intervention is the usual approach to figuring out the causal effect of an intervention compared to not applying the intervention. One important reason is that one usually applies interventions to people that need an intervention, which usually means they are required to be high on the score one tries to improve (or something that is heavily correlated with it), which then in turn induces regression to the mean (=on average there will usually be improvements even without intervention aka "regression to the mean"). $\endgroup$
    – Björn
    Commented Jul 25, 2023 at 13:46

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.