3
$\begingroup$

Is there a book that explains why there aren't better standard techniques than Tukey and ANOVA, for example?

For comparison consider for example I read about the null hypothesis one-sample $t$ test and didn't even bother considering that there could be better tests. But that is probably just a biased belief that there isn't anything more mathematically sophisticated that can be done with the Gaussian to improve on the one-sample $t$ test.

However, Tukey and ANOVA are mathematically more complicated and it didn't feel obvious to me why they should even be considered in the first place. For example in a previous question I asked about why Tukey's method is needed over all pairwise two-sample $t$-tests. The answer I got for that question is that all pairwise two-sample $t$-tests suffers from false positives. But its not intuitive to me how Tukey's method is the best way to evade that problem. How does one intuitively see that generically Tukey and ANOVA are reasonably very good techniques among all possibilities?

| cite | improve this question | |
$\endgroup$
  • 1
    $\begingroup$ There's a number of premises in this question that I think should be justified. $\endgroup$ – Glen_b Jun 28 '13 at 3:24
  • 1
    $\begingroup$ Your question could be interpreted as "more mathematically sophicticated = better". Surely you did not mean that. So the question boils down to how you want to define "better". Various ideas on that exist, but it very hard to come up with one rule that applies equally to all situations. $\endgroup$ – Maarten Buis Jun 28 '13 at 7:35
3
$\begingroup$

First, different approaches clearly exist (for example, for post-hoc comparisons there is also Dunett's test, Scheffe's test, LSD and many others; there are non-parametric approaches; more complex modelling; bootstrapping and randomization techniques; maybe you can even drop the frequentist approach altogether and move to Bayesian factors).

Whether they are better or worse depends on the context. Some will be better for some applications, some will be better for other, depending on your data and the assumptions you are willing to accept. Some will have the tendency to be underpowered, some others will do dubious assumptions.

Arguably ANOVA has been shown, through simulations mostly, to be robust and to have a large power (low type II error rates). So yes, there are objective criteria to compare one approach or statistical test with another.

Notably, ANOVA is a just an instance of general linear models (GLM) really, and often other related variants of GLM might be more suitable for a particular case than ANOVA.

| cite | improve this answer | |
$\endgroup$
  • 2
    $\begingroup$ In some of your answers, @January, you seem to abuse acronyms without explaining them. That is not a good practice because many statistical abbreviations are ambiguous (e.g., GLM = 'general linear model' or 'generalized linear model'?) and because not every one knows a given acronym which you knows. $\endgroup$ – ttnphns Jun 28 '13 at 8:09
  • $\begingroup$ You are right. I'm sorry. $\endgroup$ – January Jun 28 '13 at 8:12
1
$\begingroup$

You may be interested in a really nice book by Thomas Baguley: Serious Stats - A guide to advanced statistics for the behavioral sciences.

It covers a lot of the topics in your question going beyond the standard stuff.

| cite | improve this answer | |
$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.