I would like to conduct a paired sample t test and thus, I'm checking for the assumptions of normality. Upon conducting normality testing, each group scores were found to be normally distributed however the difference between pairs of scores are found to be not normally distributed. So my question here is, what went wrong and what can I do about this?
1
-
1$\begingroup$ What's your sample size? Normality testing is essentially useless, especially since with large N the test always rejects since nothing in life is exactly normal. But those little differences have no practical impact on the test. $\endgroup$– AdamOFeb 17, 2019 at 4:08
Add a comment
|
1 Answer
$\begingroup$
$\endgroup$
3
Nothing has gone wrong, and you don't need to do anything about it. The paired sample t-test relies on the average of the differences of the paired samples being approximately normally distributed, not the differences themselves. The two statements are not incompatible; even if the differences are significantly skewed, the distribution of the sample mean will become "more normal" as the sample size gets larger, subject to a few conditions that are almost always met in practical applications.
For example, consider the sample mean of twelve observations from a Uniform$(0,1)$ distribution. Clearly the U$(0,1)$ distribution is a long way from Normal, but the sample mean, even of only 12 observations, is not:
> x <- rep(0,1000)
> for (i in 1:1000) x[i] <- mean(runif(12))
> hist(x)
which produces:
Not too bad, and with 10,000 draws, it would be better still...
-
$\begingroup$ While the numerator of the t-test statistic will be close to normal the t-test statistic doesn't only consist of a numerator; it's not just the distribution of the average difference that matters, since the derivation of the t- relies on the behaviour of both numerator, denominator and on their independence. These only come with the normal. Nevertheless, between CLT and Slutsky you're going to get an approach to normality so the significance level will tend to be fine in large samples. So in large enough samples everything is generally okay if you don't care about power. ...ctd $\endgroup$– Glen_bAug 3, 2018 at 0:37
-
$\begingroup$ If we do care about power, then we have to worry about the distribution of pair differences, not averages. If the tails of the population distribution differences are heavy, the t can have very poor relative efficiency, even while the distribution of the test statistic looks as normal as you like. (Not that I'd advocate testing normality to choose which tests to apply). That said, I'd think your assessment that the OP probably doesn't need to do anything in this case is likely fine; the significance level should be okay and power is probably okay. More info on the response would be useful. $\endgroup$– Glen_bAug 3, 2018 at 0:38
-
$\begingroup$ Also note, that this effect does depend upon the underlying distribution of data. Compare, for example,
x <- rep(0,1000); for(i in 1:1000) x[i] <- mean(rlnorm(12, 0, 1)); hist(x)$\endgroup$ Feb 17, 2019 at 13:55