If I understand correctly, I can begin my answer by saying that we assume that the sample means differ; if their observed differences were 0, there would be nothing to "test," so to speak. Now what we are actually interested in is if these observed mean differences are merely due to chance (i.e., sampling error) or represent a real effect of the independent variable (i.e., factor) on the dependent variable in the real world. That is, we want to ensure that these observed differences are reliable so that we can be comfortable making an inference to the population (as is the case in inferential statistics in general).  To do so, we partition the variation seen across the observed scores into two sources: *between-group* variance, that is attributable to the factor (i.e., the independent variable), and *within-group* variance, whatever is left over and considered random error. The logic is that the latter (i.e., the denominator) is a representation of the variation we would expect simply by chance. The value of this ratio is distributed as *F* when the null hypothesis of no difference between population means is true. However, when the numerator is large enough to yield an *F*-value that is also sufficiently large (typically reaching the criterion of *p* < .05), then it is often concluded that the observed difference between means is *not* merely due to chance (sampling error). This simply rests on the meaning of the derived *p*-value. Of course, remember that the ANOVA significance test only tests if *one* mean is different from any other; to get at which means differ in particular, one needs to run post-hoc tests. However, to my understanding (and anyone feel free to correct me if I'm wrong), but partitioning the observed variance in this way is all about getting obtaining an *F*-statistic and its associated *p*-value. In other words, it is all about hypothesis testing, not estimation (which is done by calculating the raw difference between means and possibly a confidence interval around it--at least in the frequentist approach).