2
$\begingroup$

Say you have three groups, and each group has 5 observations.

You can figure out if there is a significant difference between means with a simple one-way ANOVA.

I read in my nonparametric book, one can also permute the observations and conduct a permutation F-test. If $n_i = 5$, then the number of permutations is $\binom{15}{5,5,5} =756756$ permutations -- you need to calculate the F-value 756756 different times.

What is the point of doing a permutation F-test? Is it more accurate than a normal F-test???

Improve this question
$\endgroup$
1

1 Answer 1

Reset to default
5
$\begingroup$

The F-distribution of the test statistic in an ordinary ANOVA F test relies on the assumption that the data are drawn from normal distributions (one for each group) with constant variance (and equal means under the null hypothesis).

If you don't have a good basis to assume normality (or at least that the F-distribution should be a good approximation to the statistic when the null hypothesis is true), then what are you to do?

One thing you can do is a permutation test, and the F statistic could certainly be used for that (the remaining concern would be whether it would have good power in whatever circumstances you're in).

Improve this answer
$\endgroup$
2
  • $\begingroup$ +1. By the way, the number of partitions of a 15-element set into 5-element sets seems to be wrongly computed in the Q: math.stackexchange.com/questions/640558. It's 126126. $\endgroup$
    – amoeba
    Mar 4, 2019 at 21:36
  • 1
    $\begingroup$ Yes, since in this particular case the groups are exchangeable (hence the factor of 6 this symmetry accounts for) but consider the possibility of directional alternatives (like a Jonckheere-Terpstra) or a one-sided post-hoc. Often that "interchange the groups" bit is ignored (leaving more work if you're enumerating them all in this particular situation, but the answer still comes out - you count six times as many and divide by six times as many) $\endgroup$
    – Glen_b
    Mar 9, 2019 at 5:07

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.

Not the answer you're looking for? Browse other questions tagged or ask your own question.