For independent reasons, I'm working on two bootstrapped datasets, A and B, of the same size, each consisting of a 1000 samples. To compare the distributions of A and B, I ran a Fischer F-test on pairs of boostrapped samples from the two datasets to test the null hypothesis that the variances are homogeneous. I found that, across 1000 pairs of A and B samples, the null hypothesis was rejected 87.1% at p<0.05.

As is customary when performing the F-test, I would like to report the F-statistic, the degrees of freedom, and the p-value. However, given that I have a 1000 sets of these values, it is not feasible to report them all. In this case, it is reasonable to report the average F-statistic and p-value across the 1000 comparisons? The degrees of freedom should stay the same.

If reporting average statistics is not appropriate, then what would be a good alternative?

  • $\begingroup$ Can your question be generalized to "how to do and to report hypothesis testing based on bootstrap"? $\endgroup$
    – Michael M
    Jul 25, 2017 at 15:13
  • $\begingroup$ The title of the question is formulated in this more general way that you mention and the main text has some details as a way of providing an example that is applicable to other inferential statistics. Without the example, someone could say "Could you provide an example?" So it doesn't seem to me like the question needs to be made more general. $\endgroup$
    – Des Grieux
    Jul 25, 2017 at 15:40


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.