Tell me more ×
Cross Validated is a question and answer site for statisticians, data analysts, data miners and data visualization experts. It's 100% free, no registration required.
up vote 1 down vote favorite

I have a data file with task duration values for three groups, and I want to determine group effect on task duration (tasks were executed by individuals; each group had 7 different individuals; each individual executed the same three tasks; and the data for one individual in group B was not recorded because of a setup problem during the experiment).

I have created from the data file the following box plot (red dots are the means, and "n" is the number of time values in each group):

enter image description here

and also the following histogram (duration given in "min:sec"):

enter image description here

My data sample per group is small, and "Shapiro-Wilk normality test" tells me that group A does not come from a normal distribution, and that groups B and C come from a normal distribution. Because groups are small and one group is non-normal, I decided to run Kruskal-Wallis one-way analysis of variance (non-parametric) and its result was:

Kruskal-Wallis rank sum test
data:  Duration by Group 
Kruskal-Wallis chi-squared = 4.2811, df = 2, p-value = 0.1176

so I should accept that the effect of the groups was not significant (p-value > 0.05).

However, when I run one-way Anova (sanity check just in case Kruskal-Wallis was not the correct choice), Anova's result was:

ANOVA Duration ~ Group 
            Df    Sum Sq   Mean Sq F value  Pr(>F)   
Group        2 0.0003692 1.846e-04   6.473 0.00293 **
Residuals   57 0.0016257 2.852e-05                   
---
Signif. codes:  0 ‘***’ 0.001 ‘**’ 0.01 ‘*’ 0.05 ‘.’ 0.1 ‘ ’ 1 

Tukey multiple comparisons of means
   95% family-wise confidence level

             diff          lwr           upr     p adj
B-A -0.0055763154 -0.009704328 -0.0014483027 0.0054132
C-A -0.0048032407 -0.008769307 -0.0008371744 0.0138750
C-B  0.0007730747 -0.003354938  0.0049010874 0.8943085

That is, Anova returns p-value < 0.05, that is, it is telling that the group effect is significant (in this case, group A was significantly different regarding B and C).

However, because of a small number of samples per group and the fact that one group is not normally distributed, I tend to accept Kruskal-Wallis result, but I am not sure.

So my questions are:

Should I accept Kruskal-Wallis result because this test is more adequate than Anova for this particular case?

How should I have used the box plot and the histogram to decide for the most adequate test?

Thanks

share|improve this question
The assumption is that the residuals are normally distributed, what is the result of the Shapiro-Wilks test for the residuals? – Glen Jul 5 '12 at 1:56
@Glen The Shapiro-Wilk test for the residuals of group A is the same as for group A (the residuals where computed as indicated here) – mljrg Jul 5 '12 at 10:33

3 Answers

up vote 3 down vote accepted

The boxplot and histogram tell you all by themselves that your data are skewed, especially in group A. The Shapiro-Wilk test is kind of pointless. With data thusly skewed the ANOVA isn't really appropriate. The Kruskal-Wallis rank sum test is based on the ranks, not the absolute values and doesn't require normality, either in the measures or residuals. It is the more appropriate test.

A quick Google search will tell you one requires normality and one does not.

One thing you might consider is that durations are an arbitrary representation of time. For example, you can indicate the duration of an event as 2s or you can say the event has a rate 0.5 events/s. It's the exact same thing and both numbers can arbitrarily be interchanged for representation. However, rates tend to be much less skewed and more appropriate for statistical analysis. It's possible your rates are normally distributed and you can use ANOVA in that case.

If you do decide to look at rates keep in mind that the direction of magnitude changes, a higher duration values = a lower rate value. Some people use a negative rate just to avoid that confusion.

share|improve this answer
If I did the transformation into ratios correctly, in R the Shapiro-Wilk results are: shapiro.test(1/A): W = 0.9064, p-value = 0.04657 shapiro.test(1/B): W = 0.9026, p-value = 0.06388 shapiro.test(1/C): W = 0.6018, p-value = 2.057e-06 That is, at alpha=0.05, Shapiro-Wilk normality test fails for groups A and C, so Anova is not adequate again ... – mljrg Jul 5 '12 at 11:50
It's probably not but I already indicated the test isn't the best way to tell, just look at the data. Regardless, you've got a resonable non-parametric solution. To summarize, there is no good "test" for normality because even with very normal data it will always fail if N is high enough. Nevertheless, you have a small N and the test is reflecting what you would see in plots anyway. – John Jul 5 '12 at 13:26
up vote 2 down vote

The Kruskal-Wallis test and the Anova test are testing different hypotheses, both could be correct, the answers differ because they are answering different questions.

share|improve this answer
The tests address slightly different hypothesis and it is clear that the medians are closer than the means. But I think the ANOVA test is still inappropriate to apply to this data and does not provide a correct answer. – Michael Chernick Jul 5 '12 at 3:45
1  
While the data is clearly skewed, with a sample size of 60 (probably 20 per group) the CLT may give a good enough approximation (multiply the p-value by 10 and it will still be significant at the 0.05 level). It would be interesting to look at permutation and/or bootstrap tests comparing the means and the medians of the groups (even though the KW test is not technically a test of medians either). – Greg Snow Jul 5 '12 at 4:04
@GregSnow A is highly skewed because one task took much more time (the values above 21 min) than the others in A. A's effect surely makes individuals take more time in that task compared with the same task in B and C, in which individuals did not do the extra work caused by the effect of group A. However, as Kruskal-Wallis compares medians (centrality), and the durations for all other tasks are close among groups, I tend to accept the result of Kruskall-Wallis, especially as the extra time (bug fix) for the task in group A would be spent later (at bug discovery) by individuals in B and C. – mljrg Jul 5 '12 at 11:11
up vote 1 down vote

The distributions all overlap very much. The Kruskal Wallis test seems to be indicating that the centers of the distributions are nearly the same. The distribution for group is highly skewed do to several very extremely high values. That is what causes the distribution to fail the Shapiro-Wilk test. Th anova F test wrongly interprets group A to have a significantly larger mean because it "ignores" the skewness. The Kruskal Wallis test is giving the appropriate answer while the F test is not.

share|improve this answer
It might be worthy that ANOVA tests are robust against normality assumption. – hbaghishani Jul 5 '12 at 11:03
That is true of t tests but not the anova F test. To Greg Snow's comment it could be enlightening to look at the bootstrap distribution of the group means (particularly group A) to see how skewed it is. – Michael Chernick Jul 5 '12 at 11:13
I think your point about "The anova F test wrongly interprets group A to have a significantly larger mean" makes sense in my setting especially after my comment to @GregSnow's answer, where I analyse why Kruskal Wallis seems to be the most appropriate choice here. – mljrg Jul 5 '12 at 11:19

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.