# Using Student or Wilcoxon test on small sample without information on the normality of distribution

I have three subjects in a qualitative study. I am comparing the features of their writing before and after treatment.

I have no idea about the normality of the distribution since they were selected purposefully. I have counted the number of everything (like the complex sentences, correct sentences, etc.) and now I want to compare each case in the subjects before and after treatment. When you look at the numbers, the difference is too high (like for example around 800 before the treatment but more than 2000 after). T-test results showed a p-value of 0.017 while wilcoxon showed 0.109. Hence t-test showed a significant difference while wilcoxon showed a nonsignificant difference.

Now my question is, due to the small sample size (only three) and not being sure of the normality of distribution, is t-test a correct procedure? I have also run chi square and it has shown a significant difference.

• Is it the case that you have multiple dependent variables & you're checking all of them? You are likely to find something just by virtue of checking many things. OTOH, you will have pretty severe issues w/ statistical power (ie, the ability to achieve significance) in many cases. My hunch is that normality is the least of your issues. Drop any pretense of statistically testing your data; simply present them & assess them qualitatively. Commented Oct 29, 2012 at 2:31
• Hi. Actually, I agree with you to some extent. But suppose that I had only one variable, i.e. the complexity of writing before and after the treatment operationalized by counting the t-units in the three students' essays before and after treatment, and, hence, coming to sth like 600 for students one before the treatment and 2100 for the same student after the treatment and more or less the same thing for the other two. What statistical procedure would you use? t-test, wilcoxon, or simply chi-square? Thanks Commented Oct 29, 2012 at 23:53
• If you were going to do only one test, the question is whether you are willing to make the theoretical assumption of normality. If so, you can use the t-test. If not, then either the Wilcoxon or even the sign test. The Wilcoxon can actually be more powerful than the t-test when the data aren't normal, I believe, but you're typically not going to have much power. Commented Oct 30, 2012 at 2:52
• Thanks. But you see, the problem is that although the difference between the two figers is too large (600 and 2100) and the quality of the students' writing show much improvement, the wilcoxon does not yield significant results, while those of t-test are significant. Commented Oct 30, 2012 at 5:12
• I recognize that, but we don't pick which test to use by checking to see which one will give us the results we want. I would just drop any pretense to statistically testing your data and simply present them & analyze them qualitatively. Another possibility is to run Bayesian analyses starting from a couple of prior belief states that people might find reasonable, & show what you ought to believe now given the prior & what you found. Commented Oct 30, 2012 at 13:27