# Small sample skewed but population is assumed to be normal, is a two sample t-test still valid?

Let's say I have two samples with relatively small sample size and their distributions are skewed.

The population distribution is assumed to be normal. It just so happens that these two particular samples were skewed.

The size of the samples is relatively small (10). Is an unpaired t-test still valid?

I don't understand if the central limit theorem still applies here, or perhaps it's not related to this issue. Is the mean of the samples somehow invalid because the samples are skewed? What test to use in this case?

• It is practically certain that any random sample of a Normal distribution will be skewed, so the first thing that would be useful to know is how skewed, according to what measure of skewness?
– whuber
Apr 4, 2022 at 19:21
• I don't have enough stats experience yet to answer that technically, but I would say the skew is not minor, visually speaking. The two samples are also skewed in opposite directions. Is there some threshold value of skewness under which the t-test would be valid? As a side question, why is it practically certain that a sample would be skewed when sampled from a normal population? I tried to look into this but only found resources on central limit theorem and sample mean Apr 4, 2022 at 19:52
• Lack of skewness, depending on how skewness is measured, reflects a precise balance in the sample. Random fluctuations will prevent that balance from occurring in most samples. This is no mere technicality: whether or not skewness in your samples is a concern depends on the form it takes (such as a single huge outlier or a more consistent imbalance) as well as its size.
– whuber
Apr 4, 2022 at 20:03
• Interesting, that is new to me! I intuitively assumed samples from a normal population would typically also be normal. The skew I observed is more consistent rather than a single huge outlier. Actually in repeated trials, some of the samples are normally distributed. The skewed samples have a spread that is similar or up to double. Apr 4, 2022 at 20:07
• Perhaps surprisingly, samples of size 10 from a Normal distribution mostly look non-Normal: they tend to be skewed and to have unusual-looking gaps and clusters. One method to determine whether your particular samples exhibit problematic behavior is through simulation, as illustrated at stats.stackexchange.com/a/69967/919 (an example of a problematic distribution in a rather large sample).
– whuber
Apr 4, 2022 at 20:11

Consider two fictitious samples of size $$n = 10$$ from normal distributions.

set.seed(2021)
x1 = rnorm(10, 50, 7)
summary(x1);  sd(x1)
Min. 1st Qu.  Median    Mean 3rd Qu.    Max.
36.54   50.53   52.48   52.12   55.68   62.11
 6.621324  # sample SD
x2 = rnorm(10, 60, 8)
summary(x2);  sd(x2)
Min. 1st Qu.  Median    Mean 3rd Qu.    Max.
45.27   58.63   66.65   63.88   72.10   72.99
 10.08623


From stripcharts of individual values of the two samples it would be difficult to judge whether the data are normal--if we did not know they were sampled from normal populations.

stripchart(list(x1,x2), ylim=c(.5,2.5), pch="|") However, if you have reason to believe that the the data are are normal (perhaps from past experience with similar data) a two-sample t test seems a reasonable choice to test $$H_0: \mu_1 =\mu_2$$ against $$H_a: \mu_1 \ne \mu_2.$$

Because I have no information in advance to suggest that the two population variances are equal, I choose to do a Welch two-sample t test, which does not require equal variances.

For our particular datasets, the t test finds a significant difference in means at the 1% level. For real data we would never know absolutely for sure whether the population means differ. Here we happen to know that $$\mu_1 = 50, \mu_2 = 60.$$

t.test(x1, x2)

Welch Two Sample t-test

data:  x1 and x2
t = -3.0806, df = 15.542, p-value = 0.007362
alternative hypothesis:
true difference in means is not equal to 0
95 percent confidence interval:
-19.86148  -3.64602
sample estimates:
mean of x mean of y
52.12437  63.87811


To get a fuller picture whether the Welch two-sample t test is appropriate to use on two such small normal samples, we can look to see how many times we reject $$H_0$$ in 100,000 repetitions of such an experiment:

set.seed(404)
pv = replicate(10^5, t.test(rnorm(10,50,7),
rnorm(10,60,8))\$p.val)
mean(pv <= 0.05)
 0.79861


The answer is that the null hypothesis is rejected about 80% of the time. In technical language, we say that the power of the Welch t test in this situation, with sample sizes of ten, is about 80%. The answer would be different if $$\delta = |\mu_2 - \mu_1|$$ were smaller or larger than 10. Or if the population variances were larger or smaller.

Notes: R code: The vector pv contains 100,000 P-values. The logical vector pv <= 0.05 contains 100,000 TRUEa and FALSEs, and its mean is the proportion of its TRUEs.

Normality. If the stripchart of the samples looked like the one below, then I would be reluctant to do a t test because there seems to be some evidence--even with samples as small as size ten--that the data may not be normal. I have to admit this is a difficult and subjective decision.

set.seed(123)
x1 = rexp(10, 1/50); x2 = rexp(10, 1/60)
stripchart(list(x1,x2), ylim=c(.5,2.5), pch="|") It hardly helps that both of these samples fail a Shapio-Wilk normality test, which can be misleading for such small sample sizes.

shapiro.test(x1)$$p.val  0.003781628 shapiro.test(x2)$$p.val
 0.0003721849


In practice, I guess one is more likely to be criticized for having so little data than for showing results from t tests.