Is the practice of doing normality test before doing t test wrong? [duplicate]

Question

When I read research papers, I saw that many people do a normality test first, then decide if a t-test or a non-parametric test should be applied.

I have a concern about such practice. Let’s say the sample follows a gamma distribution, which is kind of bell shaped, but different from normal. The larger the sample size, the more power the normality test has. So a big sample size can lead to the rejection of normality. On the other hand, the larger the sample size, the better fit with central limit theorem. As a result, even though the sample is not normal, its mean can be approximated by normal.

So we have a paradox: on one hand, large sample means its mean can be approximated by normal, which means t-test can be applied.

on the other hand, large sample give normality test more power, which in turn rejects the use of t test.

Is my concern valid? Why people do normality test? Is there an alternative way?

Answer 1

Normality tests are mainly a diagnosis tool. As you already said, you can reject this hypothesis with a large enough sample, but you can also reject equality of means with a large enough sample! You could also have a look at QQ plots in order to check departures from normality.

Although the CLT guarantees, under some mild conditions, that the average is approximately normal, the rate of convergence varies for different distributions. Then, you never know whether or not in your case, assuming normality is guaranteed by the CLT. For example, consider the following example where you compare $10,000$ times two samples of size $100$ from a t distribution with 2.01 degrees of freedom with mean 0 and 1 respectively, using a t-test.

count=vector()
for(i in 1:10000){
x = rt(100,df=2.01)
y = 1+rt(100,df=2.01)
count[i] = ifelse(t.test(x,y)$p.val<0.05,1,0)
}
mean(count)

So, at the 0.05 level, you only reject $\approx 77\%$ of the times, while the means are clearly different. The reason for this is that the distribution of the sample mean of samples of size 100 from a t distribution with 2.01 degrees of freedom does not look normal:

samp.mean = vector()
for(i in 1:10000) samp.mean[i] = mean(rt(100,df=2.01))
hist(samp.mean)

So, the CLT cannot always save you, while conducting a normality test is a safe way to convince people that you are doing sensible things.

If the samples look far from normal, you need to think harder whether or not equality of means will answer your scientific question. You could think of alternative nonparametric tests, such as the Kolmogorov-Smirnov, Mann-Whitnney, Permutation tests, and etcetera.

Moreover, if the samples are very large and you also have concerns about the normality assumption, you could even plot kernel density estimators and compare them visually, or more formally using some distance between probability distributions (total variation, for instance).