What way (ways?) is there to visually explain what is ANOVA?
Any references, link(s) (R packages?) will be welcomed.
Cross Validated is a question and answer site for people interested in statistics, machine learning, data analysis, data mining, and data visualization. It only takes a minute to sign up.Sign up to join this community
Personally, I like introducing linear regression and ANOVA by showing that it is all the same and that linear models amount to partition the total variance: We have some kind of variance in the outcome that can be explained by the factors of interest, plus the unexplained part (called the 'residual'). I generally use the following illustration (gray line for total variability, black lines for group or individual specific variability) :
I also like the heplots R package, from Michael Friendly and John Fox, but see also Visual Hypothesis Tests in Multivariate Linear Models: The heplots Package for R.
Standard ways to explain what ANOVA actually does, especially in the Linear Model framework, are really well explained in Plane answers to complex questions, by Christensen, but there are very few illustrations. Saville and Wood's Statistical methods: The geometric approach has some examples, but mainly on regression. In Montgomery's Design and Analysis of Experiments, which mostly focused on DoE, there are illustrations that I like, but see below
(these are mine :-)
But I think you have to look for textbooks on Linear Models if you want to see how sum of squares, errors, etc. translates into a vector space, as shown on Wikipedia. Estimation and Inference in Econometrics, by Davidson and MacKinnon, seems to have nice illustrations (the 1st chapter actually covers OLS geometry) but I only browse the French translation (available here). The Geometry of Linear Regression has also some good illustrations.
Ah, and I just remember this article by Robert Pruzek, A new graphic for one-way ANOVA.
How about something like this?
Following Crawley (2005). Statistics. An introduction using R: Wiley.
Thank you for your great answer so far. While they where very enlightening, I felt that using them for the course I am currently teaching (well, TA'ing) will be too much for my students. (I help teach the course BioStatistics for students from advanced degrees in medicine sciences)
Therefore, I ended up creating two images (Both are simulation based) which I think are useful example for explaining ANOVA.
I would be happy to read comments or suggestions for improving them.
The first image shows a simulation of 30 data points, separated to 3 plots (showing how the MST=Var is separated to the data that creates MSB and MSW:
The second image shows 4 plots, each one for a different combination of variance and expectancy for the groups while
Since we gather certain types of nice graphs in this post, here is another one that I recently found and may help you understand how ANOVA works and how the F statistic is generated. The graphic was created using the granova package in R.
Great question. You know, I've struggled myself with wrapping my head around ANOVA for a very long time. I always find myself going back to the "between versus within" intuition, and I've always tried to imagine what this would look like in my head. I'm glad this question came up, and I've been amazed by the varied approaches to this in the answers above.
Anyway, for a long time (years, even) I've been wanting to collect several plots in one place where I could see what was happening simultaneously from a lot of different directions: 1) how far apart the populations are, 2) how far apart the data are, 3) how big's the between compared to the within, and 4) how do the central versus noncentral F distributions compare?
In a truly great world, I could even play with sliders to see how sample size changes things.
So I've been playing with the
manipulate command in RStudio, and holy cow, it works! Here is one of the plots, a snapshot, really:
If you have RStudio you can get the code for making the above plot (sliders and all)! on Github here.
After playing with this for awhile, I am surprised at how well the F statistic distinguishes the groups, even for moderately small sample sizes. When I look at the populations, they really aren't that far apart (to my eye), yet, the "within" bar is consistently dwarfed by the "between" bar. Learn something every day, I guess.
To illustrate what is going on with one-way ANOVA I have sometimes used an applet offered by the authors of "Introduction to the Practice of Statistics", which allows the students to play with within and between variances and observe their effect on the F statistic. Here is the link (the applet is the last one on the page). Sample screen shot:
The user controls the top slider, varying the vertical spreads of the three groups of data. The red dot at the bottom moves along the plot of p-values while the F-statistic shown beneath is updated.
It seems the ship has already sailed in terms of an answer, but I think that if this is an introductory course that most of the displays offered here are going to be too difficult to grasp for introductory students... or at the very least too difficult to grasp without an introductory display which provides a very simplified explanation of partitioning variance. Show them how SST total increases with the number of subjects. Then after showing it inflate for several subjects (maybe adding one in each group several times), explain that SST = SSB + SSW (though I prefer to call it SSE from the outset because it avoids confusion when you go to the within subjects test IMO). Then show them a visual representation of the variance partitioning, e.g. a big square color coded such that you can see how SST is made of SSB and SSW. Then, graphs similar to Tals or EDi may become useful, but I agree with EDi that the scale should be SS rather than MS for pedagogical purposes when first explaining things.
Here are some representations of situations in which an ANOVA will conclude to different level of fit between $Y$ and $X$.