69
$\begingroup$

What way (ways?) is there to visually explain what is ANOVA?

Any references, link(s) (R packages?) will be welcomed.

Improve this question
$\endgroup$
2

9 Answers 9

Reset to default
56
$\begingroup$

Personally, I like introducing linear regression and ANOVA by showing that it is all the same and that linear models amount to partition the total variance: We have some kind of variance in the outcome that can be explained by the factors of interest, plus the unexplained part (called the 'residual'). I generally use the following illustration (gray line for total variability, black lines for group or individual specific variability) :

alt text

I also like the heplots R package, from Michael Friendly and John Fox, but see also Visual Hypothesis Tests in Multivariate Linear Models: The heplots Package for R.

Standard ways to explain what ANOVA actually does, especially in the Linear Model framework, are really well explained in Plane answers to complex questions, by Christensen, but there are very few illustrations. Saville and Wood's Statistical methods: The geometric approach has some examples, but mainly on regression. In Montgomery's Design and Analysis of Experiments, which mostly focused on DoE, there are illustrations that I like, but see below

alt text

(these are mine :-)

But I think you have to look for textbooks on Linear Models if you want to see how sum of squares, errors, etc. translates into a vector space, as shown on Wikipedia. Estimation and Inference in Econometrics, by Davidson and MacKinnon, seems to have nice illustrations (the 1st chapter actually covers OLS geometry) but I only browse the French translation (available here). The Geometry of Linear Regression has also some good illustrations.

Edit:

Ah, and I just remember this article by Robert Pruzek, A new graphic for one-way ANOVA.

Edit 2

And now, the granova package (mentioned by @gd047 and associated to the above paper) has been ported to ggplot, see granovaGG with an illustration for one-way ANOVA below.

enter image description here

Improve this answer
$\endgroup$
4
  • $\begingroup$ Is the first illustration produced using R? $\endgroup$
    – Yorgos
    Dec 10, 2010 at 9:35
  • $\begingroup$ @gd047 Yes. Should have the ugly source code somewhere if you want. The second one is done in Metapost. $\endgroup$
    – chl
    Dec 10, 2010 at 10:29
  • 3
    $\begingroup$ @gd047 All right, as usual it's always when we look for old code that we cannot found it (despite my best effort with grep/find), so I rewrote a quick (still ugly) R script for that. I've also put an example of the MP code. $\endgroup$
    – chl
    Dec 10, 2010 at 13:54
  • $\begingroup$ The geometry of linear regression link seems to have rotted, sadly. $\endgroup$
    – Silverfish
    Jan 9, 2015 at 22:21
25
$\begingroup$

How about something like this? alt text

Following Crawley (2005). Statistics. An introduction using R: Wiley.

Improve this answer
$\endgroup$
2
  • 1
    $\begingroup$ (+1) I remind me of plot.design() (but yours in an enhanced version :-) $\endgroup$
    – chl
    Dec 11, 2010 at 14:41
  • $\begingroup$ This is the best. $\endgroup$
    – Tomas
    Oct 16, 2012 at 23:32
14
$\begingroup$

Thank you for your great answer so far. While they where very enlightening, I felt that using them for the course I am currently teaching (well, TA'ing) will be too much for my students. (I help teach the course BioStatistics for students from advanced degrees in medicine sciences)

Therefore, I ended up creating two images (Both are simulation based) which I think are useful example for explaining ANOVA.

I would be happy to read comments or suggestions for improving them.

The first image shows a simulation of 30 data points, separated to 3 plots (showing how the MST=Var is separated to the data that creates MSB and MSW:

  • The left plot shows a scatter plot of the data per group.
  • The middle one shows how the data we are going to use for MSB looks like.
  • The right image shows how the data we are going to use for MSW looks like.

alt text

The second image shows 4 plots, each one for a different combination of variance and expectancy for the groups while

  • The first row of plots is for low variance, while the second row is for high(er) variance.
  • The first column of plots is for equal expectancy between the groups, while the second column shows groups with (very) different expectancies.

alt text

Improve this answer
$\endgroup$
3
  • 2
    $\begingroup$ (+1) I always think that the takeaway messages when teaching ANOVA is: (1) we have an F-ratio which reflects the relative importance of the variance accounted for by our factor of interest wrt. total variance (or MSB/MSW where MSW=MST-MSB), (2) the differences between the group means is a variance, and (3) we explicitly test $H_0:~\mu_1=\mu_2=\ldots=\mu_k$ vs. $H_1:~\exists\ i,j~|~\mu_i\neq\mu_j$ ($H_1\equiv\neg~H_0$). If you're able to convey those ideas into a graphical display--which seems to be the case here--, then I think you're almost done. $\endgroup$
    – chl
    Dec 10, 2010 at 21:21
  • $\begingroup$ Hi chl, thank you for the positive feedback (and for your previous detailed answer)! I think some of the biggest take home massages I got from preparing the material for this class are: 1) How to describe the transformation of the original data so to get te MSB and MSW variances measures. 2) How the test statistic of the MSB/MSW is actually a one sided (not a two sided) test where the H0 is that MSB<=MSW. Lastly, I just thought to note that it is true that SSW=SST-SSB (but I don't see how it's true for MSW=MST-MSB). $\endgroup$
    – Tal Galili
    Dec 11, 2010 at 8:25
  • 1
    $\begingroup$ Yep, sorry I wrote quickly. I meant: Consider the model $y_{ij}=\mu + \alpha_i + \varepsilon_{ij}$ or $y_{ij}=\mu_i + \varepsilon_{ij}$ and highlight graphically the following decomposition: $y_{ij}=\bar{y_i}+\varepsilon_{ij}=\bar{y}+(\bar{y}_i-\bar{y})+(y_{ij}-\bar{y}_i)$, that is obs. are expressed as deviations from the grand mean + group mean + fluctuations around group mean. Then, we have $(y_{ij}-\bar{y})=(\bar{y}_i-\bar{y})+(y_{ij}-\bar{y}_i)$ or total variation = between-group variation + within-group variation (which is basically your first picture). $\endgroup$
    – chl
    Dec 11, 2010 at 12:30
12
$\begingroup$

Since we gather certain types of nice graphs in this post, here is another one that I recently found and may help you understand how ANOVA works and how the F statistic is generated. The graphic was created using the granova package in R. alt text

Improve this answer
$\endgroup$
1
  • 2
    $\begingroup$ (+1) I gave a link to Robert Pruzek's article, but I didn't know it was available in R. $\endgroup$
    – chl
    Dec 11, 2010 at 18:25
10
$\begingroup$

Check out Hadley Wickham's presentation (pdf, mirror) on ggplot. Starting on pages 23-40 of this document he describes an interesting approach to visualizing ANOVAs.

*Link taken from: http://had.co.nz/ggplot2/

Improve this answer
$\endgroup$
5
$\begingroup$

Great question. You know, I've struggled myself with wrapping my head around ANOVA for a very long time. I always find myself going back to the "between versus within" intuition, and I've always tried to imagine what this would look like in my head. I'm glad this question came up, and I've been amazed by the varied approaches to this in the answers above.

Anyway, for a long time (years, even) I've been wanting to collect several plots in one place where I could see what was happening simultaneously from a lot of different directions: 1) how far apart the populations are, 2) how far apart the data are, 3) how big's the between compared to the within, and 4) how do the central versus noncentral F distributions compare?

In a truly great world, I could even play with sliders to see how sample size changes things.

So I've been playing with the manipulate command in RStudio, and holy cow, it works! Here is one of the plots, a snapshot, really:

visualizeANOVA

If you have RStudio you can get the code for making the above plot (sliders and all)! on Github here.

After playing with this for awhile, I am surprised at how well the F statistic distinguishes the groups, even for moderately small sample sizes. When I look at the populations, they really aren't that far apart (to my eye), yet, the "within" bar is consistently dwarfed by the "between" bar. Learn something every day, I guess.

Improve this answer
$\endgroup$
0
4
$\begingroup$

Here are some representations of situations in which an ANOVA will conclude to different level of fit between $Y$ and $X$.

enter image description here

enter image description here

enter image description here

Improve this answer
$\endgroup$
2
  • $\begingroup$ what a impressive visualization! May I know if there is any link or example code show how to plot it? The anova concept of F value, p_vale, group, df is too difficult to busines people, I think you plot may help them to understand how different factor could be influent target. Most importantly, using visualization way, looking forward you feedback. $\endgroup$
    – Yong Wang
    May 13, 2023 at 0:38
  • $\begingroup$ These two SVGs is all I've been able to recover in my archive (the download link won't probably survive very long): we.tl/t-fElYVO7hX3 $\endgroup$
    – FZS
    May 15, 2023 at 9:47
3
$\begingroup$

To illustrate what is going on with one-way ANOVA I have sometimes used an applet offered by the authors of "Introduction to the Practice of Statistics", which allows the students to play with within and between variances and observe their effect on the F statistic. Here is the link (the applet is the last one on the page). Sample screen shot:

enter image description here

The user controls the top slider, varying the vertical spreads of the three groups of data. The red dot at the bottom moves along the plot of p-values while the F-statistic shown beneath is updated.

Improve this answer
$\endgroup$
0
2
$\begingroup$

It seems the ship has already sailed in terms of an answer, but I think that if this is an introductory course that most of the displays offered here are going to be too difficult to grasp for introductory students... or at the very least too difficult to grasp without an introductory display which provides a very simplified explanation of partitioning variance. Show them how SST total increases with the number of subjects. Then after showing it inflate for several subjects (maybe adding one in each group several times), explain that SST = SSB + SSW (though I prefer to call it SSE from the outset because it avoids confusion when you go to the within subjects test IMO). Then show them a visual representation of the variance partitioning, e.g. a big square color coded such that you can see how SST is made of SSB and SSW. Then, graphs similar to Tals or EDi may become useful, but I agree with EDi that the scale should be SS rather than MS for pedagogical purposes when first explaining things.

Improve this answer
$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.