I have a, perhaps faulty, intuition that if a paired difference test (e.g., Wilcoxon rank sum, t-test) for a pre/post item looks significant, one should use caution in interpreting this fact if an independence test (e.g., Kendall's tau-b, Spearman's rho, Pearson rank) looks insignificant. It's not that independence (association or correlation) has anything to do with differences, but that alternative explanations for the paired difference test (apart from treatment effects) seem to increase in likelihood relative to the conclusion, "Use our medicine, data suggests that the typical patient will do better in random (or unpredictable) ways."
While "caution" may be wrong, the question is: In what ways, if any, should independence tests and paired difference tests inform each other under the 4 possible significant/insignificant combinations on pre/post studies of treatment effects?
As concrete problem details may be useful (or bring up other problems), my practical problem includes the following:
- 10 - 40 observation pairs
- Items are "ordered 1 - 5," Strongly disagree to Strongly agree
- Limited knowledge of false negatives profiles for independence tests referenced