From answers on this site I have learned that it is possible to get significant results from multiple comparison tests ("post hoc"), even if ANOVA did not yield a significant result. I believe that the opposite is also possible – significant ANOVA result, and non-significant post-hoc results.
I am not sure that it is possible to answer a general question, but I would still like to ask it – which is generally speaking, more powerful, an ANOVA test or post hoc tests on the same data?
To make anova and posthoc tests comparable, you need to be conducting posthoc tests for at least $G-1$ contrasts (where $G$ is the number of groups) such that the contrasts span the space of all possible contrasts. In that case, accepting all the contrast null hypotheses implies that the true group means are all equal, equivalent to the anova F-test null hypothesis.
Given this assumption, which is more powerful depends on the configuration of true group means and on the specific post-hoc tests you plan to conduct.
Let's say you are doing a oneway anova with $G$ groups. While there are many ways to conduct post hoc tests, the following strategies are amongst the most common choices:
Perform an overall F-test for differences between the group means.
Perform t-tests for all possible pairwise comparisons. Adjust the p-values for multiple testing using Holm's method.
Choose $G-1$ linearly independent contrasts that correspond to your scientific hypotheses. Conduct a t-test for each contrast and adjust the p-values using Holm's method.
Consider the overall null hypothesis that the true group means are all equal. In approach 1, the null hypothesis is rejected if the F-test p-value is less than $\alpha$, where $\alpha$ is the significance level. In approaches 2 or 3, the null hypothesis of no differences is rejected if any of the adjusted p-values are less than $\alpha$. All three approaches control the type I error rate for this test at $\alpha$.
Approaches 2 and 3 test a number of null hypotheses besides the overall null of no differences, but I am answering your question in terms of the overall null. For approaches 2 and 3, the minimum adjusted p-value is an effective test of the overall null because the intersection of the null hypotheses for the individual t-tests is equal to the overall null hypothesis and the union of t-test alternative hypotheses is equal to the F-test alternative.
The F-statistic can be written as a weighted average of the squared t-statistics from approaches 2 or 3. Hence it works best when all or most of the t-tests contribute meaningfully to the average. For the t-test approaches, the result is driven mainly or entirely by the largest t-statistic. In general, the F-statistic will give a smaller p-value if the individual t-statistics are all similar in size whereas approach 3 will give a smaller p-value if one of the t-statistics is much larger than the others in absolute size.
The F-test is more powerful than the t-tests if the true group means are equally spaced. The contrasts t-tests will be more powerful if one or more of the contrasts match the true pattern of group differences. The pairwise t-tests are generally less powerful than the contrast t-tests (because more $G(G-1)/2$ tests are conducted instead of $G-1$) but still may be more powerful than the F-test if one of the pairwise differences is much larger than the others.
One way to choose the contrasts is to test each group mean vs the average of the other group means. This approach is more powerful than the other approaches for detecting one group separate to the others.
After many years of working in biomedical research, I find that I use the F-test less and less and I use approach 3 more and more. The main trouble with the F-test in practice is that, when the null hypothesis is rejected, it gives no guidance as to which groups means are different. So one has to conduct the t-tests anyway, in which case disagreement between the t-tests and the F-tests becomes an interpretation problem. Assuming that the contrasts are appropriately chosen, approach 3 is statistically more powerful as well as simpler to interpret. Approaches 2 and 3 have the added advantage of the F-test of strong familywise error rate control over all tests conducted.
If the anova is balanced (equal numbers in each group) then one can also consider Tukey's honestly significant differences, which is similar to making all possible pairwise comparisons but more powerful because Tukey's method accounts for the dependencies between the pairwise comparisons. My remarks above apply to any anova, balanced or otherwise.
