I'm currently trying to reconstruct a significance analysis that I did years ago (when I knew even less about statistics that I do now). I have the strong feeling that what I did was invalid, and that SPSS kind of mislead me there…
I have an experiment in which a set of participants are presented with a set of tasks. We measure how well each participant performs in each task. The tasks have a treatment variable, lets call it "type". Each task can have one of the types "PO", "SL", "DO" or "OPO". Also, the tasks are coupled, i.e., for any task of e.g. type "PO", there is one equivalent task of "SL", "DO" and "OPO" each.
I want to compare the types pairwise, i.e., test the hypothesis that participants perform differently well in e.g. "SL" and "OPO" tasks. By today, I know (or think I know?) that I should probably do Wilcoxon's Signed Rank test on every pair and apply a Dunn-Bonferroni post correction.
Back then, it seems like a performed Friendman's Two-Way Analysis of Variance by Ranks on the data set. However, if I understand the test correctly, it only tests whether all four groups stem from the same population. In other words: A low p-Value doesn't tell me that e.g. SL is significantly different from OPO, but just that not all SL, DO, PO and OPO are the same.
This is what SPSS gave me back then when I ran the test, on the right you can see pairwise significance values:
The question that I now have:
- Is SPSS correct? Can you derive p-Values for the hypothesis "group PO is from the same population as group SL" from Friedman's Two-Way ANOVA by Ranks?
- If not: What are those reported values? Did it just re-run Friedman's test with just the respective two groups? In that case, you'd need another Dunn-Bonferroni post correction, right? (That would explain the "Adj. Sig." column.) Would that be valid in my case?
- Is my assumption (pairwise Wilcoxon Signed-Rank test) valid in my case?
Thanks a lot,