2
$\begingroup$

I've created an error plot with CI 95% to visualize the difference (post test - pre test) between achieved scores.

error bars

n = 64

Judging from the confidence intervals one could conclude that there's is no significant improvement between the pre and post test scores (the CI does include 0).

When I use a paired t-test for the pre and post scores in combination, none of them return significant as expected.

But since the differences of the scores are not normally distributed for Total Score Change (p < .001) and Sub Score Change I (p < .05) according to Shapiro-Wilk, i've use the paired Wilcoxon test, which turns out significant for Sub Score Change I (p < .05).

So which test should I believe and why?

$\endgroup$
1
  • $\begingroup$ Does the rule for determining significance by whether the confidence intervals include 0 or not apply only for normally distributed data? $\endgroup$
    – TheLostOne
    Mar 23, 2014 at 14:04

1 Answer 1

3
$\begingroup$

You need to use a point estimate (pseudo median) and confidence interval that is consistent with the Wilcoxon signed-rank test. Here is an example in R, where there are no ties in absolute values of diff so that exact calculations are available (otherwise use the R coin package).

diff <- c(-3.1, -2.2, 1.3, 2.4, 3.5, 4.6, 5.7)
wilcox.test(diff, exact=TRUE, conf.int=TRUE, conf.level=0.95)

Wilcoxon signed rank test

data:  diff
V = 22, p-value = 0.2188
alternative hypothesis: true location is not equal to 0
95 percent confidence interval:
 -2.2  4.6
sample estimates:
(pseudo)median 
           1.8 

Note that the pre-post design does not allow cause and effect type of inferences.

$\endgroup$
4
  • $\begingroup$ Thank you! But to be honest I don't know how to apply answer to my example. I still don't understand why I can't use the Wilcoxon-signed-rank test? It yields a more desirable result (significant) although it is a non-parametric test, which normally are supposed to be less powerful. $\endgroup$
    – TheLostOne
    Mar 23, 2014 at 18:38
  • $\begingroup$ If you have paired data, the Wilcoxon test may a good one. But you were asking why the inconsistency between parametric confidence intervals and the nonparametric test. I think I answered that. If you want a confidence interval that is consistent with the test (which is desirable) then go with the example above. This confidence interval can be thought of as the set of pseudomedians (median of all possible pairwise means) that if hypothesized to be the population pseudomedian would not be rejected at the $1 - \alpha$ level. $\endgroup$ Mar 23, 2014 at 18:42
  • $\begingroup$ Okay, thanks. So you suggested to use the CIs from your example, when I use the Wilcoxon test. Same goes for plotting. Instead of mean the pseudo median with the adjusted CIs. So the rule for determining significance by whether the confidence intervals include 0 or not, does probably only apply when all assumption of a parametric test are met. $\endgroup$
    – TheLostOne
    Mar 23, 2014 at 19:35
  • 1
    $\begingroup$ Right about the first part, but you still don't understand the big picture. If you are going to be using statistics you should study some standard texts. My point was that the parametric confidence interval is not supposed to be concordant with the nonparametric test. One deals with the mean and the other with the pseudomedian. Whether the assumptions of a parametric test are satisfied or not is another matter. $\endgroup$ Mar 23, 2014 at 21:10

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.