1
$\begingroup$

I have several groups of measurements like shown below, and I want to know of the distance from each group to the overall mean is significantly different from the rest. Can I do a bootstrap where I just draw distances from the total group of distances measured, and set 95% confidence from that? Or should I instead use Tukey's HSD to determine the points where distance is different?

enter image description here

enter image description here

Improve this question
$\endgroup$
3
  • $\begingroup$ Are you interested in comparing each mean to each other mean as is done with Tukey or each mean to the mean of the other groups? $\endgroup$
    – David Lane
    Commented Jan 8, 2017 at 23:04
  • $\begingroup$ I think each mean to each other mean as in Tukey - I want to determine which points have significantly larger distance than expected by chance. $\endgroup$
    – user144685
    Commented Jan 8, 2017 at 23:54
  • $\begingroup$ Tukey HSD is a good choice. Contrary to popular opinion, you don't have to do an ANOVA first since the Tukey HSD fully controls the Type I error rate. Just don't call it post hoc since in your case, there is nothing post hoc about it. $\endgroup$
    – David Lane
    Commented Jan 9, 2017 at 0:07

1 Answer 1

Reset to default
1
$\begingroup$

Tukey's test, following an omnibus ANOVA test is the way to go, assuming you meet the assumptions of the ANOVA test.

Improve this answer
$\endgroup$
7
  • $\begingroup$ if the data are weakly dependent, would I be better off using a bootstrap? $\endgroup$
    – user144685
    Commented Jan 9, 2017 at 0:30
  • $\begingroup$ You could go simpler in that case and use a non-parametric multiple comparison test like the Kruskal – Wallis test. $\endgroup$
    – StatsStudent
    Commented Jan 9, 2017 at 3:18
  • 1
    $\begingroup$ Tukey's HSD and bootstrap p-value adjustment are general ways to test for multiplicity.along with many others. I don't think the reasons given are justification for picking one over the other. Sometimes considerations like simplicity or how conservative the test is. As I say many times when issues of multiplicity come up take a look at Westfall and Young's 1993 book published by Wiley. It emphasizes permutation and bootstrap methods but also considers Bonferroni and some other methods. If you are in a situation where you are interested in an FDR criteria check other literature on that $\endgroup$
    – Michael R. Chernick
    Commented Jan 9, 2017 at 4:15
  • $\begingroup$ FDR stands for False Discovery Rate. $\endgroup$
    – Michael R. Chernick
    Commented Jan 9, 2017 at 4:16
  • $\begingroup$ I think I am not quite looking to do multiple comparisons - for each point I just want to know if it is different from the overall mean distance across all the measured data, rather than whether it is different from each other point or which other points it is different from. $\endgroup$
    – user144685
    Commented Jan 9, 2017 at 4:23

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.