Heck et al (2013) write that
Generally, the first step in a multilevel analysis is partitioning the variance (referred to as $\sigma^2$) in an outcome variable into its within- and between-group components. If it turns out that there is little or no variation (perhaps less than 5%) in outcomes between groups, there would be no compelling need for conducting a multilevel analysis.
To that end, they suggest running a Random-Effects ANOVA, from which output like the following gets generated:
This example relates to math test results being predicted from schools. I understand that 66.551 represents variation in individual scores within schools, and 10.642 represents between-school variation. To calculate the intraclass correlation Heck et al take 10.642/(10.642+66.551) = 0.138, and conclude that 13.8% of the variance in achievement lies between schools. They write that "the intraclass correlation provides a sense of the degree to which differences in the outcome variable Y exist between Level 2 units".
Their approach makes some intuitive sense to me. However, what I don't understand is why we can't just run a regular fixed effects ANOVA and compare the between-groups sum of squares with the within-groups sum of squares.
By analogy, I'd take 100708.437/530078.816 = .190, which is not the same value that I got from the Random Effects ANOVA. However, isn't a Fixed-Effects ANOVA also also measuring the "degree to which differences in the outcome Y exist between Level 2 units"?
I understand that output from a Random-Effects ANOVA and a Fixed-Effects ANOVA are not the same thing, but is it wrong (or sub-optimal) to use the results from a Fixed-Effects ANOVA to decide whether to proceed with a multilevel analysis? Why/why not?