# Different means with non-significant anova

I did an ANOVA test summary(aov(m$Y~m$type)) and I get a p-value of 0.067 i.e. the means are equal. But my means by type are these

m$type m$Y
1            1      56.33333
2            2      51.70000
3            3       9.25000


The mean of type 3 is so different, why ANOVA doesn't gives a p-value<0.05?

• What are the group sizes and variances?
– whuber
Dec 16 '14 at 23:17
• @whuber 12,10,8 respectively. And std. desviation are 63.3,54.4,12.9 respectively for types 1,2,3 Dec 16 '14 at 23:30

Note that you're making inferences about unobserved population means. Obviously sample means are generally unequal! You don't need a test to tell you that.

A p-value higher than your significance level doesn't mean that your population means are equal, only that you're unable to tell the difference from what you'd expect to see if they were equal. Quite a different thing; not-clearly-different isn't the same as equal.

Sample means that seem different don't imply that they're so different that you couldn't observe something similar with population means that were equal.

I note that all your standard deviations exceed your means, and that they seem roughly in proportion to your means. I expect that your data are fairly strongly right skew and heteroskedastic. It would help if you could say more about your data.

i) The usual ANOVA assumes the variances are equal, and produces an averaged estimate of the variance. Relative to standard errors for the differences based on the overall standard deviation, the means are not so different that it can tell them all apart.

ii) even if you could tell some means apart in an ANOVA, if there are other comparisons that are not different, it may not be able to distinguish the collection of means overall as different. That is, you could have a non-significant ANOVA while a pairwise comparison done alone (say a t-test) would be different.

The assumptions that underlie the p-value calculation in the usual ANOVA don't seem to hold, so your p-value relies on likely-untenable assumptions and so will not be a reasonable reflection of the actual situation.

If your within-group data are reasonably consistent with normality (this would require negative observations), then you could do a Welch-corrected ANOVA. See ?oneway.test.

If normality seems less likely, a more tenable set of assumptions (depending on your data, you might consider a GLM, say, or perhaps working on the log scale), might be preferable. I'd suggest looking at some visual display, perhaps something like this (note the log-scale; this might be suitable if your values are all positive):

(obviously this isn't your actual data, since we don't have that)

If your data look anything like that, I'd say there's no solid reason to think that the smaller-mean group is necessarily different from the others.