3
$\begingroup$

Data

I need to test if there is a significant difference in leakage between prototypes. I've tried kruskal wallis and post hoc wilcoxon pairwise and Dunn-Bonferroni test, and the post hoc test always say that there is no significant difference between them for except A.4 and C.4, but that can't be... for example how can there not be a significant difference between A.4 with mean 27% leakage and C.2 with mean 1.13 % leakage?

$\endgroup$
2
  • 2
    $\begingroup$ If you use non-parametric tests, then the scale of the difference has no impact on the test, but the ranking does, and the more other items are between them the more significant the ranking difference appears to be $\endgroup$
    – Henry
    Commented Apr 8, 2023 at 23:49
  • 1
    $\begingroup$ @Henry Where you say "If you use non-parametric tests, the scale of the difference has no impact on the test" you seem to be equating "nonparametric" with "based on ranks". OP mentioned having tried rank based tests but the question doesn't seem to restrict itself to that situation and there are nonparametric tests for which the scale does impact the test. Adding "rank-based" before "nonparametric tests" would work, or perhaps "if you consider those particular nonparametric tests" then I believe it all works as you suggest. $\endgroup$
    – Glen_b
    Commented Apr 9, 2023 at 2:34

1 Answer 1

3
$\begingroup$

Leakage data is exponential in character, so your 7 experimental groups will have unequal and non-constant variance.

Transform your data first by applying the logarithm function. Then repeat your ANOVA. Then apply a multiple comparison procedure. You've chosen non-parametric procedures in the past (which are fine), but you might look into the Tukey, Scheffe, and Bonferroni procedures.

There is a little cheat here. Tukey is less likely to flag differences.

$\endgroup$
2
  • 1
    $\begingroup$ +1 But the logarithm is too strong. A square root would be more appropriate. $\endgroup$
    – whuber
    Commented Apr 9, 2023 at 0:04
  • $\begingroup$ Did you mean to provide a link to a cheatsheet? $\endgroup$
    – dipetkov
    Commented Apr 10, 2023 at 17:06

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.