3
$\begingroup$

I have a mixed-model design with between-subjects factors Group (treatment, control) and Difficulty (easy, hard), and within-subjects factor Timepoint (pre, post). I want to use my statistical package (Statistica) to plot the dependent variable (memory test score) with separate categories for easy&hard, separate lines for the 2 groups, and repeat such a graph for each time point.

However, I was surprised to see that the means & error bars in the graphs produced by the ANOVA window depend on how the ANOVA is defined: 1) if I do each between-subjects ANOVA separately (once for each level of the within factor, i.e. once for pre and once for post), the means & error bars showing the interaction in each ANOVA are different from the graph showing the 3-way interaction (browen down for pre and post) in the mixed-model 3-way ANOVA 2) if I take IQ as a covariate, I get means & error bars that are again different to the no-covariate case 3) if I enter the covariate into the model crossed with the other factors (e.g. if I look for a "Group X Difficulty X IQ" interaction), I get means & error bars that are yet-again different as compared to when the covariate is added to the model separately, i.e. uncrossed with any other factor.

Why is that? Shouldn't the mean and error bars simply be calculated as the average and SEM (or whatver the error bars represent) respectively? And shouldn't those two statistics be independent of what factors are included in the model (ANOVA) and how they are included (see points 1-3 above)?

If it is in fact legitimate that these things influence the means and error bars, then which graph should I believe? In other words, how should I plot my two between-subjects (Group x Difficulty) interactions: from within the large 3-way (Group x Difficulty x Timepoint) ANOVA or from within each of the 2-way ANOVAs (Group x Difficulty) done separately for pre and for post?

Thanks and sorry for the long question! Please let me know if I need to make it more clear.

Improve this question
$\endgroup$
1
  • $\begingroup$ Would very much appreciate getting some expert opinions on this question, thanks a lot in advance! $\endgroup$
    – z8080
    Commented Jun 28, 2014 at 16:29

1 Answer 1

Reset to default
2
+25
$\begingroup$

The error estimates will differ depending on what variables are in your analysis. Remember each time you add a predictor that is not perfectly correlated with another predictor it will soak up some of the error variance. In addition, various statistical packages take any number of different approaches to plotting error bars for within subjects design depending on their theoretical stance as to whether it is appropriate to include subject variance in the error term.

All other things being equal, I prefer to see error bars that reflect the statistical test result you want to highlight for the reader. I also like to see variables like time on the X axis. But those are just general preferences. There is no 'right' answer here IMO... you just have to make the best decision given your circumstances.

Improve this answer
$\endgroup$
5
  • $\begingroup$ Thanks! What I don't understand, though, is why are the means and the error bars affected by whether or not subject variance is included in the error term? Surely the mean will just be the mean, and the error bars are just (typically) +-1SEM. Surely SEM should be independent of the error terms calculated as part of the ANOVA, not to mention the fact that there is no error term if you produce the graph not out of an ANOVA interaction but simply using the stats packages Plot Means option. $\endgroup$
    – z8080
    Commented Jul 1, 2014 at 11:42
  • $\begingroup$ In SPSS (and probably other packages) the means you are plotting in some plots but not others are estimated marginal means (I.e. unweighted by sample size and corrected by covariate if applicable). If your conditions have different sample sizes or the distribution of your covariance is uneven, then the estimated marginal means will not equal the observed means leading to different plots. $\endgroup$
    – russellpierce
    Commented Jul 1, 2014 at 11:53
  • $\begingroup$ So then the error bars will be subject to the same dependence upon, say, the factors of the ANOVA? Also, what I am still not sure about is which is the correct way to plot this graph. I know you said it depends on circumstances, but I really just want to plot the mean and error bar for each "cell", i.e. each combination of conditions. Is it at all true that the graph plotted from a simple Graph window in SPSS/Statistica will be a "cleaner" way to do this than the graph produced by a model (ANOVA) that points to the same data (same conditions, equal number of subjects per condition etc)? $\endgroup$
    – z8080
    Commented Jul 7, 2014 at 22:43
  • $\begingroup$ Yes, the error bars from a fitted model will typically reflect the residual variance after accounting for factors in the model. $\endgroup$
    – russellpierce
    Commented Jul 7, 2014 at 22:46
  • $\begingroup$ @longtalker you can plot the between subjects error bars if you like. Visualization here is a matter of art and tradition more than a matter of what is 'right'. I typically like the plots I see to reflect the salient aspects of the test they are related to. But, there is room for latitude here. $\endgroup$
    – russellpierce
    Commented Jul 7, 2014 at 22:52

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.