4
$\begingroup$

I am measuring stem widths of a control plant and a mutant plant and want to determine if there is any significant difference between them. The sample sizes are not the same. When I use a test like Kolmogorov-Smirnov, I get huge significance even though it seems like the distributions aren't really that different. It seems too sensitive; what test would be more suitable?

$\endgroup$
3
  • $\begingroup$ Welcome to CV! Have you evaluated the distribution of your stem width measurements for both the control and mutant plants? What are the exact sample sizes for your control and mutant plant groups? Do you have an estimate of the variability (e.g., standard deviation) within your control and mutant groups? $\endgroup$
    – ADAM
    Feb 23 at 1:28
  • 2
    $\begingroup$ "... it seems like the distributions aren't really that different." It would be nice to support such a statement with an image such that other people can see what you mean. $\endgroup$ Feb 23 at 9:56
  • 2
    $\begingroup$ " to determine if there is any significant difference between them" are you thinking about specific types of differences? Which ones? $\endgroup$ Feb 23 at 9:57

2 Answers 2

4
$\begingroup$

It depends on the parameter of interest. The KS test compares the whole cumulative distribution functions. A significant result could be because of different means, medians, variances, skewness, kurtosis, and so on, or any combination of them, but in general any quantiles.

If the difference to test is of means, Welch t test is usable. However, in small samples, the best method for independent group comparisons is the permutation test. See R package {coin}. We can use bootstrap method to find confidence intervals of the difference, using {confintr} https://cran.r-project.org/web/packages/confintr/vignettes/confintr.html

$\endgroup$
2
  • $\begingroup$ All well, but bootstrap may work poorly in small samples, as its nice properties are based on asymptotic theory. $\endgroup$ Feb 23 at 10:20
  • $\begingroup$ Sounds reasonable conceptually, but papers I read use very small group sizes, like n = 16 in a group, to illustrate advantage of bootstrap CI. Repeats R >= 20000 is recommended for accuracy. The bias depends on sampling randomness. $\endgroup$
    – DrJerryTAO
    Feb 23 at 23:02
3
$\begingroup$

The KS test is a test of the null hypothesis that normal stem widths and mutant stem widths were drawn from the same continuous distribution.

I would probably think that you perhaps are interested in Welch's two sample t-test, which tests the null hypothesis that the population mean of normal stem widths and mutant stem widths are the same.

Welch's two sample t-test is also appropriate when variances and/or samples sizes are different.

$\endgroup$

Not the answer you're looking for? Browse other questions tagged or ask your own question.