Statistical test with small samples - Cross Validated most recent 30 from stats.stackexchange.com 2019-08-20T01:04:20Z https://stats.stackexchange.com/feeds/question/173094 http://www.creativecommons.org/licenses/by-sa/3.0/rdf https://stats.stackexchange.com/q/173094 1 Statistical test with small samples Oli4 https://stats.stackexchange.com/users/68158 2015-09-18T14:08:22Z 2015-09-18T14:08:22Z <p>I want to compare 3 groups of samples. Each sample contains 5 measurements. I want to know if the samples differ significantly. First I wanted to do a ANOVA, but my samples are not normal distributed. I tested this with a Shapiro-Wilks Test. For statisitical analyses I use the python library scipy.stats. </p> <pre><code>import numpy as np import scipy.stats as stats sample1 = np.array([0.0016, 0.0012, 0.0009, 0.0011, 0.0016]) sample2 = np.array([0.0018, 0.0016, 0.0015, 0.0015, 0.0015]) sample3 = np.array([0.0007, 0.0005, 0.0013, 0.001 , 0.0015]) stats.shapiro(sample1) #returns p-Val &gt;0.33173635601997375 stats.shapiro(sample2) &gt;0.021380668506026268 stats.shapiro(sample3) &gt;0.8324998021125793 </code></pre> <p>For sample2 we can not assume a normal distribution because the p-Value is smaller 0.05, thats why I decided to test for differences with the Kolmogorov-Smirnov Test where \$H_0\$ is that 2 independent samples are drawn from the same continuous distribution. </p> <pre><code>stats.ks_2samp(sample1, sample2) #returns p-Val &gt;0.20898483057516717 stats.ks_2samp(sample1, sample3) &gt;0.69740487802059081 stats.ks_2samp(sample2, sample3) &gt;0.03614619076928504 </code></pre> <p>My conclusion is that there is only a significant difference between sample2 and sample3. </p> <ul> <li>Is this statistically correct or are there any assumptions I missed? </li> <li>Is there maybe a better way to compare this data?</li> <li>Is there a statistical test for more than 2 groups where \$H_0\$ says that all groups are different?</li> </ul>