1
$\begingroup$

In my paper, I run some ANOVAs and t-tests, and the test statistics are F and t respectively. I have reason to believe the null distributions are not normal, so I have run permutation tests and obtained p values with respect to the empirical null distributions. How do I report the results?

For example, I have observed F(1, 16) = 14.21, but it's not REALLY an F statistic with those degrees of freedom, because the real null distribution is a bit different to the theoretical distribution with those degrees of freedom. Do I still state the degrees of freedom even if the p value comes from permutation, not the theoretical F? I do still state the observed F, right, even though I'm using a permutation test to test its significance?

As for t, should I even be using t as the test statistic for the permutation analysis, or should I just be looking at something like the difference between means as the test statistic?

$\endgroup$

1 Answer 1

1
$\begingroup$

If you are computing (approximating/simulating) the p-value of your test statistic, be it an F or otherwise, the usual normal-theory null distribution is irrelevant. That means the "degrees of freedom" you cite are irrelevant, you are not using them for anything! Your p-value comes from the permutation distribution (which do not have any degrees of freedom associated with it.)

So what you need for your paper is to describe how you computed (approximated) that permutation distribution.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.