What way (ways?) is there to visually explain what is ANOVA?

Any references, link(s) (R packages?) will be welcomed.

  • Given the way responses and comments are going, I added the 'data-viz' tag; hope you don't mind (feel free to remove if it's not appropriate here). – chl Dec 12 '10 at 11:45
  • Sure chl - thanks. – Tal Galili Dec 12 '10 at 18:34
  • In his blog 'A psychologist's endeavors in statistical programming' Kristoffer Magnusson gives a great example of one-way anova visualization using D3.js rpsychologist.com/d3-one-way-anova/#comment-1891 – Epifunky Jan 24 '14 at 8:04
  • I have found this nice visualisation of what analysis of variance is. It not as precise as previous answers, but you can interactively play with the visualisation. Found it pretty intersting : students.brown.edu/seeing-theory/regression/index.html#third – Mike Mar 17 '17 at 15:27
up vote 48 down vote accepted

Personally, I like introducing linear regression and ANOVA by showing that it is all the same and that linear models amount to partition the total variance: We have some kind of variance in the outcome that can be explained by the factors of interest, plus the unexplained part (called the 'residual'). I generally use the following illustration (gray line for total variability, black lines for group or individual specific variability) :

alt text

I also like the heplots R package, from Michael Friendly and John Fox, but see also Visual Hypothesis Tests in Multivariate Linear Models: The heplots Package for R.

Standard ways to explain what ANOVA actually does, especially in the Linear Model framework, are really well explained in Plane answers to complex questions, by Christensen, but there are very few illustrations. Saville and Wood's Statistical methods: The geometric approach has some examples, but mainly on regression. In Montgomery's Design and Analysis of Experiments, which mostly focused on DoE, there are illustrations that I like, but see below

alt text

(these are mine :-)

But I think you have to look for textbooks on Linear Models if you want to see how sum of squares, errors, etc. translates into a vector space, as shown on Wikipedia. Estimation and Inference in Econometrics, by Davidson and MacKinnon, seems to have nice illustrations (the 1st chapter actually covers OLS geometry) but I only browse the French translation (available here). The Geometry of Linear Regression has also some good illustrations.

Edit:

Ah, and I just remember this article by Robert Pruzek, A new graphic for one-way ANOVA.

Edit 2

And now, the granova package (mentioned by @gd047 and associated to the above paper) has been ported to ggplot, see granovaGG with an illustration for one-way ANOVA below.

enter image description here

  • Is the first illustration produced using R? – George Dontas Dec 10 '10 at 9:35
  • @gd047 Yes. Should have the ugly source code somewhere if you want. The second one is done in Metapost. – chl Dec 10 '10 at 10:29
  • 3
    @gd047 All right, as usual it's always when we look for old code that we cannot found it (despite my best effort with grep/find), so I rewrote a quick (still ugly) R script for that. I've also put an example of the MP code. – chl Dec 10 '10 at 13:54
  • The geometry of linear regression link seems to have rotted, sadly. – Silverfish Jan 9 '15 at 22:21

How about something like this? alt text

Following Crawley (2005). Statistics. An introduction using R: Wiley.

  • (+1) I remind me of plot.design() (but yours in an enhanced version :-) – chl Dec 11 '10 at 14:41
  • This is the best. – Curious Oct 16 '12 at 23:32

Thank you for your great answer so far. While they where very enlightening, I felt that using them for the course I am currently teaching (well, TA'ing) will be too much for my students. (I help teach the course BioStatistics for students from advanced degrees in medicine sciences)

Therefore, I ended up creating two images (Both are simulation based) which I think are useful example for explaining ANOVA.

I would be happy to read comments or suggestions for improving them.

The first image shows a simulation of 30 data points, separated to 3 plots (showing how the MST=Var is separated to the data that creates MSB and MSW:

  • The left plot shows a scatter plot of the data per group.
  • The middle one shows how the data we are going to use for MSB looks like.
  • The right image shows how the data we are going to use for MSW looks like.

alt text

The second image shows 4 plots, each one for a different combination of variance and expectancy for the groups while

  • The first row of plots is for low variance, while the second row is for high(er) variance.
  • The first column of plots is for equal expectancy between the groups, while the second column shows groups with (very) different expectancies.

alt text

  • 2
    (+1) I always think that the takeaway messages when teaching ANOVA is: (1) we have an F-ratio which reflects the relative importance of the variance accounted for by our factor of interest wrt. total variance (or MSB/MSW where MSW=MST-MSB), (2) the differences between the group means is a variance, and (3) we explicitly test $H_0:~\mu_1=\mu_2=\ldots=\mu_k$ vs. $H_1:~\exists\ i,j~|~\mu_i\neq\mu_j$ ($H_1\equiv\neg~H_0$). If you're able to convey those ideas into a graphical display--which seems to be the case here--, then I think you're almost done. – chl Dec 10 '10 at 21:21
  • Hi chl, thank you for the positive feedback (and for your previous detailed answer)! I think some of the biggest take home massages I got from preparing the material for this class are: 1) How to describe the transformation of the original data so to get te MSB and MSW variances measures. 2) How the test statistic of the MSB/MSW is actually a one sided (not a two sided) test where the H0 is that MSB<=MSW. Lastly, I just thought to note that it is true that SSW=SST-SSB (but I don't see how it's true for MSW=MST-MSB). – Tal Galili Dec 11 '10 at 8:25
  • 1
    Yep, sorry I wrote quickly. I meant: Consider the model $y_{ij}=\mu + \alpha_i + \varepsilon_{ij}$ or $y_{ij}=\mu_i + \varepsilon_{ij}$ and highlight graphically the following decomposition: $y_{ij}=\bar{y_i}+\varepsilon_{ij}=\bar{y}+(\bar{y}_i-\bar{y})+(y_{ij}-\bar{y}_i)$, that is obs. are expressed as deviations from the grand mean + group mean + fluctuations around group mean. Then, we have $(y_{ij}-\bar{y})=(\bar{y}_i-\bar{y})+(y_{ij}-\bar{y}_i)$ or total variation = between-group variation + within-group variation (which is basically your first picture). – chl Dec 11 '10 at 12:30

Since we gather certain types of nice graphs in this post, here is another one that I recently found and may help you understand how ANOVA works and how the F statistic is generated. The graphic was created using the granova package in R. alt text

  • 1
    (+1) I gave a link to Robert Pruzek's article, but I didn't know it was available in R. – chl Dec 11 '10 at 18:25
  • Thanks gd047, I remembered seeing this package, but didn't remember it's name :) – Tal Galili Dec 11 '10 at 22:52

Check out Hadley Wickham's presentation (pdf, mirror) on ggplot. Starting on pages 23-40 of this document he describes an interesting approach to visualizing ANOVAs.

*Link taken from: http://had.co.nz/ggplot2/

Great question. You know, I've struggled myself with wrapping my head around ANOVA for a very long time. I always find myself going back to the "between versus within" intuition, and I've always tried to imagine what this would look like in my head. I'm glad this question came up, and I've been amazed by the varied approaches to this in the answers above.

Anyway, for a long time (years, even) I've been wanting to collect several plots in one place where I could see what was happening simultaneously from a lot of different directions: 1) how far apart the populations are, 2) how far apart the data are, 3) how big's the between compared to the within, and 4) how do the central versus noncentral F distributions compare?

In a truly great world, I could even play with sliders to see how sample size changes things.

So I've been playing with the manipulate command in RStudio, and holy cow, it works! Here is one of the plots, a snapshot, really:

visualizeANOVA

If you have RStudio you can get the code for making the above plot (sliders and all)! on Github here.

After playing with this for awhile, I am surprised at how well the F statistic distinguishes the groups, even for moderately small sample sizes. When I look at the populations, they really aren't that far apart (to my eye), yet, the "within" bar is consistently dwarfed by the "between" bar. Learn something every day, I guess.

To illustrate what is going on with one-way ANOVA I have sometimes used an applet offered by the authors of "Introduction to the Practice of Statistics", which allows the students to play with within and between variances and observe their effect on the F statistic. Here is the link (the applet is the last one on the page). Sample screen shot:

enter image description here

The user controls the top slider, varying the vertical spreads of the three groups of data. The red dot at the bottom moves along the plot of p-values while the F-statistic shown beneath is updated.

  • the link is broken – thias Mar 15 '17 at 19:43
  • @thias Fixed. It is now redirecting to new publisher's website. – David May 16 '17 at 5:34

It seems the ship has already sailed in terms of an answer, but I think that if this is an introductory course that most of the displays offered here are going to be too difficult to grasp for introductory students... or at the very least too difficult to grasp without an introductory display which provides a very simplified explanation of partitioning variance. Show them how SST total increases with the number of subjects. Then after showing it inflate for several subjects (maybe adding one in each group several times), explain that SST = SSB + SSW (though I prefer to call it SSE from the outset because it avoids confusion when you go to the within subjects test IMO). Then show them a visual representation of the variance partitioning, e.g. a big square color coded such that you can see how SST is made of SSB and SSW. Then, graphs similar to Tals or EDi may become useful, but I agree with EDi that the scale should be SS rather than MS for pedagogical purposes when first explaining things.

Here are some representations of situations in which an ANOVA will conclude to different level of fit between $Y$ and $X$.

enter image description here

enter image description here

enter image description here

Your Answer

 
discard

By clicking "Post Your Answer", you acknowledge that you have read our updated terms of service, privacy policy and cookie policy, and that your continued use of the website is subject to these policies.

Not the answer you're looking for? Browse other questions tagged or ask your own question.