What is ANOVA?
It is important to note some of the reasons that ANOVA exist, as it can provide some context about how you should consider your omnibus ANOVA and the pairwise tests that accompany them (and largely your question about interpreting both).
The person largely credited for its invention Ronald A. Fisher was hired at an agricultural facility to determine differences in crop yield variation in plots at a farm. The facility had collected nearly a century's worth of data on this (about 70 years by the time Fisher got there) that included fluctuations in weather patterns, which fertilizers were used, soil deterioration, etc. This was a lot of info that seemed to only convey that year-by-year, some fertilizers worked and then some years they didn't. There was a lot of random noise that people simply didn't consider at the time as confounding effects (Fisher, 1921).
Fisher then came up with the solution of finding a way to systematically apportion effects attributed to the actual treatment (the fertilizers) and effects that were simply random (weather, etc.) by tabulating the data and then determining the mean squares between and within groups (see original table below for how simply inspecting the differences here wasn't super informative):
Formulation of Parametric and Nonparametric Versions
The omnibus test of the ANOVA is simply stated as:
$$
H_0: \mu_1 = \mu_2 = \mu_3
$$
where the $\mu$ here is simply a sample mean and can be theoretically extended to any number of groups. Note that for the nonparametric version you use, this is instead a test of medians (though not in an exact way as Robert already notes). In the parametric case, this is achieved by apportioning variance related to fixed factors (our groups) and general variance within these groups, such that the $F$ test used in ANOVA is:
$$
F = \frac{MS_{\text{between}}}{MS_{\text{within}}}
$$
where $MS$ is the "mean squares". The calculation of the mean squares is quite tedious but can be less formally described as (Gravetter et al., 2021):
$$
F = \frac{ \text{variance between groups} }{ \text{variance within groups} }
$$
with the "variance" here being not a test of the actual variance in groups, but variance of sample means. The major difference with the nonparametric Kruskall-Wallis is that testing is done using a rank-based approach instead (as they are less prone to issues with outliers, heterogeneous variance, etc.):
$$
H = \left(\frac{12}{N(N+1)} \sum_{j=1}^k \frac{R_{j}^2}{n_j}\right) - 3(N+1),
$$
where $N$ is the total sample size, $n_j$ is the sample size for a given group, $j$ is the group designator, $R_j$ is the sum of the ranks in a group. See this link for a step-by-step approach to calculating the K-W test. The idea behind it is that you are pooling the observations from the samples into one combined sample, then ranking each group observation from lowest to highest from $1$ to $N$, and then comparing the groups after.
ANOVA and PWCs
We can now determine how omnibus tests like these are not making the same claims as their PWCs, which is a big part of your question. Note at the outset we have a potential problem with how we are testing groups in the omnibus form. We are not testing which groups are different from each other (or even how different they are). The test can be heuristically stated as "is there no difference between the groups I have selected?" The ANOVA test, when flagged for statistical significance, simply states that the evidence of no differences is low. As a simple thought experiment, say we collect reading scores from 100 different schools across some country. If we run an ANOVA, we will almost assuredly get flagged for significance. Why? Because at this point, almost any fluctuation in means will enlarge the $F$ ratio. We are also just pooling the results here without determining how this relates to individual groups. To reiterate, we neither find out where our differences lie or how much of a difference actually exists. Hence the use of t-tests, typically corrected pairwise comparisons (PWCs), which largely developed after Fisher's formulation of ANOVA.
I note this specifically because you ask whether or not we should pay attention to one test statistic (the omnibus test) compared to another (the by-group comparisons). It may be useful to explore Midway et al., 2020, which provides a detailed accounting of what PWCs are and which ones are useful to use contingent upon setting. In their paper, they state the following:
The classic ANOVA (ANalysis Of Variance) is a general linear model that has been in use for over 100 years (Fisher, 1918) and is often used when categorical or factor data need to be analyzed. However, an ANOVA will only produce an F -statistic (and associated p-value) for the whole model. In other words, an ANOVA reports whether one or more significant differences among group levels exist, but it does not provide any information about specific group means compared to each other. Additionally, it is possible that group differences exist that ANOVA does not detect. For both of these reasons, a strong and defensible statistical method to compare groups is nearly a requirement for anyone analyzing data.
However, PWCs have their own problems, particularly when you have several group comparisons to make. As ANOVA already has issues with the school example I presented earlier, so too does a typical PWC (usually a sharp reduction in power with many group comparisons, see Nakagawa, 2004). Thankfully you do not have a considerable number of groups, but I note this because you are already using a nonparametric ANOVA and PWCs, which makes the power even further reduced. Whether that is warranted or not is contingent upon your data, but I simply note that it is something to consider with respect to reporting your results.
What This Means for You
First, you can explain the differences in these NHST test statistics separately. For the omnibus test, you can explain that the test that "there exists no difference between the groups" showed that the probability was low, perhaps lower than chance outcomes, so the idea that the differences are exactly zero is perhaps not well-founded. For the PWCs, you simply state that the null hypothesis was/was not rejected for each group. This can be done independent of the omnibus test for the reasons outlined above. See Wasserstein and Lazar (2018) for proper reporting of $p$ values, and perhaps consider neoFisherian reporting (Hurlbert & Lombardi, 2009) so as to not exaggerate your claims.
Second, and more important, never just report $p$ values for testing. They are largely useless on their own, and have a well-documented relationship with causing real-world harm by making poor claims with them (Hauer, 2004; Ziliak & McCloskey, 2008). This is no less true for ANOVAs or PWCs. It is much more important to report things like the raw mean/SD estimates for each group, visualizations of group differences, any effect sizes from both the model and group comparisons, and perhaps things like confidence intervals as well (see Cumming, 2014 for why). I repeat, do not just report the $p$ values, as your post seems to indicate that this was the only thing being considered.
References
- Nakagawa, S. (2004). A farewell to Bonferroni: The problems of low statistical power and publication bias. Behavioral Ecology, 15(6), 1044–1045. https://doi.org/10.1093/beheco/arh107
- Cumming, G. (2014). The new statistics: Why and how. Psychological Science, 25(1), 7–29. https://doi.org/10.1177/0956797613504966
- Fisher, R. A. (1921). Studies in crop variation. I. An examination of the yield of dressed grain from Broadbalk. The Journal of Agricultural Science, 11(2), 107–135. https://doi.org/10.1017/S0021859600003750
- Hauer, E. (2004). The harm done by tests of significance. Accident Analysis & Prevention, 36(3), 495–500. https://doi.org/10.1016/S0001-4575(03)00036-8
- Gravetter, F. J., Wallnau, L. B., Forzano, L.-A. B., & Witnauer, J. E. (2021). Essentials of statistics for the behavioral sciences (Edition 10). Cengage.
- Hurlbert, S. H., & Lombardi, C. M. (2009). Final collapse of the Neyman-Pearson decision theoretic framework and rise of the neoFisherian. Annales Zoologici Fennici, 46(5), 311–349. https://doi.org/10.5735/086.046.0501
- Midway, S., Robertson, M., Flinn, S., & Kaller, M. (2020). Comparing multiple comparisons: Practical guidance for choosing the best multiple comparisons test. PeerJ, 8, e10387. https://doi.org/10.7717/peerj.10387
- Wasserstein, R. L., & Lazar, N. A. (2016). The ASA statement on p -values: Context, process, and purpose. The American Statistician, 70(2), 129–133. https://doi.org/10.1080/00031305.2016.1154108
- Ziliak, S. T., & McCloskey, D. N. (2008). The cult of statistical significance: How the standard error costs us jobs, justice, and lives. University of Michigan Press.
