I have data from a paired experiment (each participant does both the control and experiment and we find the difference). I know that for the paired t-test, an assumption is that the difference is normally distributed. I also now that due to CLT, samples of n > 30 can be assumed normal, and that my sample size is 58; however, I ran a Shapiro-Wilk Test for normality and it rejected the null hypothesis that the data is normally distributed. Is it safe to still use the paired t-test? Or should I use another test that does not assume normality, such as the Wilcoxon?
10
-
$\begingroup$ I naively forgot about the other assumption, which is that the data must not have any outliers. I remove the outliers and ended up with a normal distribution (according to Shapiro-Wilk) with n = 50 (8 outlier removed). Does this seem to be the mistake in my original assessment of the problem? $\endgroup$– Ryohei NamikiCommented Jun 11, 2020 at 15:00
-
2$\begingroup$ Removing extreme observations may remove the most interesting and scientifically meaningful ones. $\endgroup$– mdeweyCommented Jun 11, 2020 at 15:10
-
2$\begingroup$ The CLT doesn't say anything about whether the difference is distributed normally, it says that sample mean of the differences will tend towards a normal distribution! The fact that your data doesn't pass a Shapiro-Wilk Test doesn't contradict this, since it's testing something else, namely whether the differences themselves are normally distributed. Of course appealing to the CLT is always a bit fuzzy, but as you mentioned the rule of thumb of n > 30 can be applied here. I think you're OK to use a paired t-test as is, without removing outliers. $\endgroup$– Max S.Commented Jun 11, 2020 at 15:23
-
2$\begingroup$ @whuber Good point, I retract the last sentence of my comment. Is it fair to say "The data not passing a Shaprio-Wilk Test does not necessarily imply that a paired t-test is inappropriate"? $\endgroup$– Max S.Commented Jun 11, 2020 at 15:37
-
2$\begingroup$ @Max Yes, that's a very good way to conclude your comment. Ryohei: those p-values might not be as different as they seem. In any case, unless you can adopt some strong specific distributional assumptions about the pair differences, the t-test will be among the most powerful options, so the relatively high p-values indicate you're unlikely to detect a significant difference using some other generic test such as the Wilcoxon. $\endgroup$– whuber ♦Commented Jun 11, 2020 at 16:19
|
Show 5 more comments
1 Answer
$\begingroup$
$\endgroup$
Suppose that there are $n$ paired differences $D_i.$ It seems worthwhile to emphasize that the paired t test assumes that $\bar D$ is nearly normal. The rule that $n \ge 30$ is sufficient for $\bar D$ to be normal is too simple. For some distributions of the $D_i,$ a dozen observations would suffice, and for others, thirty observations are not enough. A reasonable clue whether thirty are not enough would be that the sample is obviously heavily skewed or for the sample to contain far outliers.
For example, suppose $n=40.$ If $D_i \sim \mathsf{Norm}(\mu = 0.3, \sigma=1),$ then $E(D_i) = 0.3$ and $SD(X_i) = 1.$ However, if $D_i \sim \mathsf{Exp}(1) - 0.7,$ then we also have $E(D_i) = 0.3$ and $SD(X_i) = 1,$ but the distribution of $\bar D$ is noticeably non-normal, as illustrated below.
set.seed(2020)
a.exp = replicate(10^5, mean(rexp(40)-.7))
summary(a.exp)
Min. 1st Qu. Median Mean 3rd Qu. Max.
-0.2568 0.1895 0.2915 0.2998 0.4009 1.2210
hist(a.exp, prob=T, br=30, col="skyblue2", main="Skewed Dist'n of Means")
curve(dnorm(x, mean(a.exp), sd(a.exp)), add=T, col="red", lwd=2)
Below are boxplots for twenty samples of size $n=40$ of such "exponential" paired differences $D_i.$ Clearly, these samples typically show fair warning of skewness, often along with high outliers.
set.seed(1234); m = 20; n = 40
d = rexp(m*n) - .7; g = rep(1:m, n)
boxplot(d ~ g, col="skyblue2", pch=20)
abline(h=.3, col="red", lwd=2)
The departure of the distribution of sample averages from normal is enough to degrade the power of the t test to detect population paired difference of $0.3$--- from about 46% to about 44%, as illustrated in the simulations below:
set.seed(611)
pv.exp = replicate(10^5, t.test(rexp(40)-.7)$p.val)
mean(pv.exp <= .05)
[1] 0.43727
pv.nor = replicate(10^5, t.test(rnorm(40,.3,1))$p.val)
mean(pv.nor <= .05)
[1] 0.45735
However, in case the distribution of the $D_i$ is clearly not symmetrical, a one-sample Wilcoxon (signed-rank) test is not an attractive alternative to the paired t test: This Wilcoxon test would have only about 16% power to detect a difference of $0.3.$
wpv.exp = replicate(10^5, wilcox.test(rexp(40)-.7)$p.val)
mean(wpv.exp <= .05)
[1] 0.16366
Overall, the Wilcoxon test is not quite as powerful as the t test for normal data (which are symmetrical), but the loss in power from about 46% for a t test (above) to about 44% for the Wilcoxon SR test is not so great for normal data.
wpv.nor = replicate(10^5, wilcox.test(rnorm(40,.3,1))$p.val)
mean(wpv.nor <= .05)
[1] 0.44338
It is true that nonparametric tests work in some circumstances where data are not normal. However, nonparametric tests can have their own essential assumptions, and for the Wilcoxon SR test, symmetry of the data is an important assumption.
