I am currently running a randomized, placebo-controlled, cross-over design for a new type of intervention. This intervention is compared to a placebo and another more conventional type of treatment. Each participant is allocated in a randomized order to each of the three interventions (i.e., they receive all three interventions with wash-out periods of 1-week+), a repeated measures design.

The dependent variable was not normal distributed (Shapiro-Wilk), and I therefore decided to run a Friedman's test. Following testing of the specific variable, a significant difference in the distribution of ranks was found for the three treatments. Using SPSS' new legacy Friedman test, multiple comparisons with Bonferroni correction, however, showed no significant differences between the three pairs. After reading up on the procedure, it seems that the new legacy Friedman's test runs the test on all data as a whole, compared to multiple Wilcoxon signed-rank tests, which naturally would indicate different methods and therefore also results. I am, however, not entirely sure of the impact this would have. I am not willing to compromise the sound approach to statistical analysis, and start digging around with multiple different tests to see what give the best result. Therefore, I thought I would kindly ask for advice, and the reason why I get a significant distribution of ranks difference but nothing in the multiple comparisons, and how (if this is correct) to report it.

  • $\begingroup$ The DV need not be normal, only the residuals. Did you test the DV itself, or the residuals of a model? How much data do you have? How far from normality were they? $\endgroup$ Commented Mar 5, 2015 at 17:27
  • $\begingroup$ Thank you for your comment, gung. I am sorry for not stating this. I used the SAVE option for predicted and unstandardized/standardized residuals in the GLM (repeated measures), and ran Shapiro-Wilk test on the residuals. Only one of the residual variables showed normal distribution, whereas the remaining were severely non-normal distributed. I should mention that my variable is ordinal (0-10 scale). The analysis is performed on 13 different descriptors (DVs) with treatment (A, B, or C) as independent variable. I have included 30 participants for the study. $\endgroup$
    – D.B
    Commented Mar 5, 2015 at 18:17
  • $\begingroup$ continued.. Each participant rate the 13 descriptors on a scale from 0-10 - yielding 13 descriptors per participant per session = 13*30*3 = 1170 datapoints. $\endgroup$
    – D.B
    Commented Mar 5, 2015 at 18:22
  • $\begingroup$ I haven't studied the algorithm doc on the multiple comparisons done by SPSS after Friedman. However you should take into account that Friedman is not comparable with Wilcoxon. $\endgroup$
    – ttnphns
    Commented Mar 5, 2015 at 18:23
  • $\begingroup$ Thank you for the input, ttnphns. I am aware of the differences broadly speaking, however, I am still unaware of the impact that a significant overall Friedman's test on a dependent variable means, when the subsequent multiple comparisons give no indication of any significant differences among the distributions of ranks. I do believe, since I am testing differences between rating of each of the 13 DVs within each treatment paradigm (well-knowing that these are ranked in Friedman's test), that Friedman's test is justified, but I may be wrong. $\endgroup$
    – D.B
    Commented Mar 5, 2015 at 18:26


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.