3
$\begingroup$

To look for differences in more than 2 sample means an ANOVA is used. SST (between groups) and SSE (within groups) is calculated. F = MST/MSE. I get all this, however why can we say that there are differences between sample means when MST > MSE? When the variance between groups is larger than variance within groups?

$\endgroup$

3 Answers 3

3
$\begingroup$

Exactly. When the variance of the means of the groups is greater than the variance within the groups, we have evidence that the population means are spread out and not equal. If the group means have about as much variability as the data do in general, then of course the observed group means are spread out some.

Further explanation: Under the null hypothesis, the group means are the same. Thus, you’re essentially sampling from the same distribution three times. I just sampled 10 observations from a standard normal three times and got means of 0.143, 0.451, and -0.248. There’s variability, but those three values have a variance of 0.116. Sticking those values into the ANOVA equations, we get an F-statistic of 1.65 and a p-value of 0.211. There is no evidence that the three groups have different means, nor should there be.

However, as one of the groups gets a different mean, the spread of the observed means is going to get to be bigger. Imagine that the observed means had been 0.143, 0.451, and 600. Suddenly the spread of those three values is very large, not enough that the sampling variability accounts for the spread, and I would reject the null hypothesis of equal means.

$\endgroup$
6
  • $\begingroup$ Could you intuitively explain this evidence? How does variance between groups > variance within groups "proves" that there is a difference in means? $\endgroup$ Jul 6, 2019 at 20:50
  • $\begingroup$ Please see my edit. $\endgroup$
    – Dave
    Jul 6, 2019 at 21:03
  • $\begingroup$ So could I intuitively say this: "Because the variance between groups is larger than within groups, we can conclude that the distribution of one or more groups doesn't overlap and thus don't have the same mean"? $\endgroup$ Jul 6, 2019 at 21:09
  • $\begingroup$ The groups can overlap. Consider sampling from N(-1,1), N(0,1), and N(1,1). $\endgroup$
    – Dave
    Jul 6, 2019 at 21:17
  • $\begingroup$ Explicitly, it has to do with the distribution of sample means. As you get many observations, your sampling distribution gets very tight. If you deviate from that tightness by having one or more group means out of the mainstream of the sampling distribution, you have evidence that the groups are not the same. $\endgroup$
    – Dave
    Jul 6, 2019 at 21:23
0
$\begingroup$

Here is a very simple explanation.

Basically, if there's more variation between groups than within groups, that's evidence that the groups are different.

$\endgroup$
0
$\begingroup$

I don't have any formal answer here, but I saw some discussion in the comments to the previous answer that wanted a more intuitive sense for why the ANOVA works. I think I might have a way of explaining it that doesn't involve any math or thinking about sampling distributions (but at the cost of not being perfectly nuanced).

Say that we have 3 different classrooms who all took the same math test. Each classroom was taught by the same teacher but using a different method. The natural question would be: did the teaching method impact test performance. This is a fairly typical ANOVA question, and hopefully it's a familiar enough scenario to imagine.

To understand how the ANOVA works, it's useful to think about the problem from the perspective of the null hypothesis. Under the null hypothesis, we assume there is no difference in the means of the three classrooms. We might be tempted to check this by computing the mean test scores of the three classrooms and just looking at whether they are the same. The issue with this, of course, is that we don't really expect the means to be exactly the same. Since each classroom will have its own variation around the true mean, we need to have some method that lets us set a threshold hold for when variation between the classrooms is just error versus when it reflects a true difference in means.

Again, taking the case where the null hypothesis is true, this would mean that the between-group variance (variance in means between the classrooms) is not really any larger than the within-group variance (variance around the mean within each classroom). If we think about this a different way, then this would mean that the groups are just kind of labels (e.g., Classroom A, Classroom B, and Classroom C) but are totally interchangeable (i.e., the teaching method used in Classroom A had no different impact than the teaching method used in Classroom C). So, when the within-group variance = between-group variance, we suspect that the groups all come from the same population and just reflect typical sampling variability. Why is this the case? Well, it's because the within-group variance is basically an empirical point of reference for the sampling variability: no single student in the class is expected to score at exactly that class' mean, so there's variability related to this phenomenon that is described by the within-group variability.

With this in mind, now consider a case where the between-group variability starts to grow. This means that one group is starting to pull away from the others with regard to that mean performance and thus introduces more variability. If we compare this new variability source to our empirical sampling variability in the within-groups case, then we can essentially check whether or not we're observing something that might be sampling error versus something different. Once the between-group variance exceeds some threshold, we have to say that a better explanation is now that the classrooms are not equivalent or interchangeable anymore but instead represent at least two (though maybe more) different groups (i.e., the teaching method produces distinctly different test performance).

To your more implicit point about why this ratio of variances lets us make a distinction about the equivalence of means, this is mostly related to the sampling distributions as noted in the other answer here; however, perhaps the reason this doesn't exactly feel intuitive is due to the nature of frequentism and traditional cut-off points for "significance." There is certainly some ratio of between to within group variances that anyone could look at and say "for sure, there's differences in the means of those groups," but research is not generally filled with those kinds of examples. Instead, we often are looking at cases where the effect sizes are relatively small, where there is considerable overlap between the groups, and we're relying on means as summaries of group behavior. Ultimately, the decision of how large that ratio of variances needs to be is arbitrary, and as a result, may not really ever make great sense for some research interest or goals. Still, it's a helpful heuristic for many cases

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.