Timeline for Why does factor appear significant in 2-way ANOVA but not by inspection (mean +/- sd or error plot)?
Current License: CC BY-SA 4.0
16 events
| when toggle format | what | by | license | comment | |
|---|---|---|---|---|---|
| Apr 11 at 14:13 | comment | added | whuber♦ | Agreed about the Matlab docs. But my experience with their tech support has been great -- and in all the cases I was investigating (which included fairly low-level code), it was possible to read the source code and to put hooks into it to monitor what it was doing. But using general-purpose tools like Matlab, Mathematica, etc., for statistical analysis is IMHO best done when you know in detail what you're doing, which means (among other things) having long experience with and access to at least one dedicated statistical analysis platform. | |
| Apr 11 at 8:44 | comment | added | Buck Thorn | @whuber To be fair I was stumbling a bit in the dark. The MATLAB documentation is not great. In general the statistics package hides the computations being performed (R code is not much better but the documentation is; I realize a more advanced textbook might include relevant details). This also got wrapped up in discussion of plots (which was useful). The small effect of factor 2 is obfuscated by the presentation, as this answer points out. A comment suggested I read up on contrasts. I edited to clarify the q: how to determine significance of fact #2, including within individual pairs. | |
| Apr 11 at 8:22 | vote | accept | Buck Thorn | ||
| Apr 11 at 8:23 | |||||
| Apr 10 at 15:29 | comment | added | whuber♦ | Because Peter's post here doesn't answer your original question -- it plays the role of an extended request for clarification -- I would suggest modifying your question here rather than starting a new thread. | |
| Apr 10 at 7:58 | comment | added | Buck Thorn | I'd like to avoid posting a new question, but my remaining uncertainty is how to perform pairwise comparisons to see where differences due to factor #2 are significant (eg between 1 and 6). Is this at all possible? I can edit my original post and try not to alter it too much. | |
| Apr 10 at 7:56 | comment | added | Buck Thorn | Thanks for your answer. Upon review it seems showing boxplots was a mistake. I picked this somewhat out of convenience (it is the default display method in Matlab when performing ANOVA). A scatter plot would have been better as whuber and dipetkov pointed out. Your answer though does answer my actual question, and whuber pointed out why I missed the obvious. Factor #1 has a strong effect and #2 weak (I suspected none but the ANOVA identified one). | |
| Apr 10 at 7:53 | comment | added | Buck Thorn | I suspect the "extra lines" you refer to are the "notches". From the documentation: "Notches display the variability of the median between samples. The width of a notch is computed so that boxes whose notches do not overlap have different medians at the 5% significance level. The significance level is based on a normal distribution assumption, but comparisons of medians are reasonably robust for other distributions. Comparing box plot medians is like a visual hypothesis test, analogous to the t test used for means. In some cases, notches can extend outside the boxes." | |
| Apr 8 at 16:53 | comment | added | Peter Flom | Yes, I know that. But look at YOUR plot. It has more lines than that. | |
| Apr 8 at 16:32 | comment | added | Buck Thorn | The more about section does provide quite a bit of information. Plus marks (+) are outliers, for instance, dashed lines are whiskers. | |
| Apr 8 at 16:30 | comment | added | Buck Thorn | The levels of factor #1 are different chemical reagents applied to equivalent substrates. For factor #2 these are orthogonal to factor #1 (presumed to be non-interacting) and can be assumed to be 2 different solvents. | |
| Apr 8 at 16:27 | comment | added | Peter Flom | First, yes, I know what a box plot is. The notches are clear enough, too (notched box plot) but there are a bunch of extra lines. They are not explained and nothing in the 'more about" section has them. And you still haven't said what the factors are -- e.g. they might be sex and race; or political party and age group; or who knows what. | |
| Apr 8 at 16:25 | comment | added | Buck Thorn | @whuber I apologize, that would be a limitation with my present abilities with Matlab. While a little painful it is not difficult to perform a visual analysis. As mentioned, 1 and 6 differ in level of 2-level categorical factor #2 but are at the same level of categorical factor #1. Similarly 2-7,3-8,4-9,5-10 are paired wrt factor #1. | |
| Apr 8 at 16:22 | comment | added | whuber♦ | I am confident Peter is not asking what a boxplot is. One of the (many) problems with the plots in the question is that they do not clearly reveal the two-factor structure of the analysis (nor do they help at all with the interaction). The use of confidence bands for boxplots that represent only 5 values each is unnecessary and visually confusing, too. (Typically there are more graphical elements in each box then there are data behind it!) You could do much better plotting the data themselves on dot plots. | |
| Apr 8 at 16:22 | comment | added | Buck Thorn | "It would have made this easier to write if you had told us what these factors are". For factor#1 there are 5 levels (eg treatments 1-5 differ in this factor) , it is categorical. The boxplot shows the means +/- sd (and percentiles) for each treatment (corresponding to combination of factors 1 and 2): there are 2x5 possible treatments. | |
| Apr 8 at 16:15 | comment | added | Buck Thorn | Matlab details boxplots here (see the "more about" section). | |
| Apr 8 at 16:12 | history | answered | Peter Flom | CC BY-SA 4.0 |