2
$\begingroup$

I conducted a training session and for each of my users, I measured pre and post score (before and after training session). Each of my users (n=28) is measured twice, so my data is paired:

Data:

user pre-score post-score
1    10        30
....

Hypothesis: $$ H_0: \mu_{Post-Pre} \leq 0\\ H_1: \mu_{Post-Pre} > 0 $$

In order to check the hypothesis, whether the post-score is significantly higher than the pre-score (whether the training had a positive effect on users' score), I would conduct a paired t-test (t.test(Post, Pre, mu=0, conf.level=0.95, alt="greater") in R).

Problem: The post data is not normally distributed. That's why I would go with a non-parametric test, i.e. Wilcoxon signed-rank, instead of the paired t-test.

However, if a calculate the difference between the pairs (Post-Pre), I end up with one sample which is normally distributed. So I would have to test the following hypothesis in a one sample t-test (t.test(Post-Pre, mu=0, conf.level=0.95, alt="greater") in R): $$ H_0: \mu_{Diff.} \leq 0\\ H_1: \mu_{Diff.} > 0\\ $$

Question: Which method is appropriate? Should I use the non-parametric paired Wilcoxon text because my post data is not normally distributed? Or can I work with the difference of the pairs (one sample) which is normally distributed and use a one sample t-test?

Improve this question
$\endgroup$
3
  • 3
    $\begingroup$ The one sample t-test assumes that the differerences are normally distributed, not the scores. $\endgroup$
    – Jeremy Miles
    Jan 14, 2019 at 17:54
  • $\begingroup$ So, both samples (pre and post) each must follow a normal distribution? If one does not (like my post data), I'd use a non-parametric test? $\endgroup$
    – Ioannis K.
    Jan 14, 2019 at 18:07
  • 1
    $\begingroup$ Only the differences would need to be drawn from a normal population (or something sufficiently close to a normal population). Normality of the differences is used in deriving the t-distribution (you need it for the distribution of the numerator and the distribution of the denominator and for the two being independent). Outside that, you may be able to justify an asymptotic normal distribution for the statistic (via CLT and Slutsky's theorem). $\endgroup$
    – Glen_b
    Jan 15, 2019 at 0:04

1 Answer 1

Reset to default
3
$\begingroup$

First, it's not necessary that the differences be Normally distributed, just that the sample mean of the differences be (pretty close to) Normally distributed. The Central Limit Theorem lets us know that the distribution of the sample mean converges to Normal as the sample size gets larger under a broad set of conditions, which your data certainly satisfy. For example, calculating the sample mean of a Uniform distribution with 12 observations was, in the very old days, one of the ways used to generate approximately Normally distributed random variables.

Second, the Wilcoxon signed-rank test doesn't test the same thing as the paired t-test. It tests whether the differences follow a symmetric distribution around 0. Thus, substantial asymmetry can cause the test to reject the null hypothesis even if the true mean is zero. (In your case, given that your analysis of the differences indicates Normality is plausible, this isn't likely to be an issue.)

In your case, I suspect the two tests will give the same result, but I'd go with the t-test. With 28 observations of data that isn't very far from Normally distributed, you should be fine.

Improve this answer
$\endgroup$
4
  • $\begingroup$ First, thanks for this clear explanation! So, although my post data is not normally distributed, I can still use the t-test? Does it make a difference whether I use the paired t-test or the one sample t test with regard to the differences in score? I read many articles and some say I have to use non-parametric tests because at least one sample is not normally distributed.Others claim t-test is robust against non-normality; there must be at least 25 data points. Is there any (common) reference I can cite to justify my selection on the parametric t-test in my paper/dissertation? $\endgroup$
    – Ioannis K.
    Jan 14, 2019 at 21:22
  • 1
    $\begingroup$ @jbowman a t-statistic has a numerator and a denominator, both are random variables. Showing that the numerator is asymptotically normal won't make the ratio t-distributed. While in practice it often turns out that the t is a reasonable approximation, the behavior of the ratio of the numerator and denominator matter (and in some cases, the behavior of the denominator seems to make a difference). As far as I know we don't have a good theoretical justification for claiming anything more than asymptotic normality for the t-statistic (once we also invoke Slutsky). Of course... $\endgroup$
    – Glen_b
    Jan 15, 2019 at 0:12
  • 1
    $\begingroup$ ... the t will also be approximately normal, but on what basis would we think it would be a better approximation for the distribution of the test statistic than the asymptotic normal that we have a justification for? I see the argument for a t in terms of the behavior of the mean (in the numerator) a lot but I think there's a gap between what that gives us and what we'd need to show. $\endgroup$
    – Glen_b
    Jan 15, 2019 at 0:14
  • $\begingroup$ Appreciate your help! Since the difference in means is normally distributed, I'd go with a paired t-test. Or do I have to use one sample t-test for this case? $\endgroup$
    – Ioannis K.
    Jan 15, 2019 at 8:30

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.

Not the answer you're looking for? Browse other questions tagged or ask your own question.