# Statistical test with small samples

I want to compare 3 groups of samples. Each sample contains 5 measurements. I want to know if the samples differ significantly. First I wanted to do a ANOVA, but my samples are not normal distributed. I tested this with a Shapiro-Wilks Test. For statisitical analyses I use the python library scipy.stats.

import numpy as np
import scipy.stats as stats

sample1 = np.array([0.0016,  0.0012,  0.0009,  0.0011,  0.0016])
sample2 = np.array([0.0018,  0.0016,  0.0015,  0.0015,  0.0015])
sample3 = np.array([0.0007,  0.0005,  0.0013,  0.001 ,  0.0015])

stats.shapiro(sample1)[1] #returns p-Val
>0.33173635601997375
stats.shapiro(sample2)[1]
>0.021380668506026268
stats.shapiro(sample3)[1]
>0.8324998021125793


For sample2 we can not assume a normal distribution because the p-Value is smaller 0.05, thats why I decided to test for differences with the Kolmogorov-Smirnov Test where $H_0$ is that 2 independent samples are drawn from the same continuous distribution.

stats.ks_2samp(sample1, sample2)[1] #returns p-Val
>0.20898483057516717
stats.ks_2samp(sample1, sample3)[1]
>0.69740487802059081
stats.ks_2samp(sample2, sample3)[1]
>0.03614619076928504


My conclusion is that there is only a significant difference between sample2 and sample3.

• Is this statistically correct or are there any assumptions I missed?
• Is there maybe a better way to compare this data?
• Is there a statistical test for more than 2 groups where $H_0$ says that all groups are different?
• If you were to correct your test results for the multiple independent comparisons (as you should), you would not conclude the data are not normally distributed and you would proceed with the ANOVA. – whuber Sep 18 '15 at 14:21
• Do you mean, that I should use the Bonferroni correction and than could go on with a ANOVA? – Oli4 Sep 18 '15 at 14:27