7
$\begingroup$

I have a data file with task duration values for three groups, and I want to determine group effect on task duration (tasks were executed by individuals; each group had 7 different individuals; each individual executed the same three tasks; and the data for one individual in group B was not recorded because of a setup problem during the experiment).

I have created from the data file the following box plot (red dots are the means, and "n" is the number of time values in each group):

enter image description here

and also the following histogram (duration given in "min:sec"):

enter image description here

My data sample per group is small, and "Shapiro-Wilk normality test" tells me that group A does not come from a normal distribution, and that groups B and C come from a normal distribution. Because groups are small and one group is non-normal, I decided to run Kruskal-Wallis one-way analysis of variance (non-parametric) and its result was:

Kruskal-Wallis rank sum test
data:  Duration by Group 
Kruskal-Wallis chi-squared = 4.2811, df = 2, p-value = 0.1176

so I should accept that the effect of the groups was not significant (p-value > 0.05).

However, when I run one-way Anova (sanity check just in case Kruskal-Wallis was not the correct choice), Anova's result was:

ANOVA Duration ~ Group 
            Df    Sum Sq   Mean Sq F value  Pr(>F)   
Group        2 0.0003692 1.846e-04   6.473 0.00293 **
Residuals   57 0.0016257 2.852e-05                   
---
Signif. codes:  0 ‘***’ 0.001 ‘**’ 0.01 ‘*’ 0.05 ‘.’ 0.1 ‘ ’ 1 

Tukey multiple comparisons of means
   95% family-wise confidence level

             diff          lwr           upr     p adj
B-A -0.0055763154 -0.009704328 -0.0014483027 0.0054132
C-A -0.0048032407 -0.008769307 -0.0008371744 0.0138750
C-B  0.0007730747 -0.003354938  0.0049010874 0.8943085

That is, Anova returns p-value < 0.05, that is, it is telling that the group effect is significant (in this case, group A was significantly different regarding B and C).

However, because of a small number of samples per group and the fact that one group is not normally distributed, I tend to accept Kruskal-Wallis result, but I am not sure.

So my questions are:

Should I accept Kruskal-Wallis result because this test is more adequate than Anova for this particular case?

How should I have used the box plot and the histogram to decide for the most adequate test?

Thanks

Improve this question
$\endgroup$
4
  • $\begingroup$ The assumption is that the residuals are normally distributed, what is the result of the Shapiro-Wilks test for the residuals? $\endgroup$
    – Glen
    Commented Jul 5, 2012 at 1:56
  • $\begingroup$ @Glen The Shapiro-Wilk test for the residuals of group A is the same as for group A (the residuals where computed as indicated here) $\endgroup$
    – mljrg
    Commented Jul 5, 2012 at 10:33
  • 1
    $\begingroup$ What makes you think either test is "wrong"? p-values are random variables, and different hypothesis tests aren't perfectly dependent. It's perfectly reasonable that sometimes one might reject and another fail to, even when all assumptions of both are satisfied. $\endgroup$
    – Glen_b
    Commented Oct 29, 2014 at 17:16
  • $\begingroup$ Two linked side-issues deserve comment. Groups A B C mush together extra sources of variation. If one individual didn't complete B I would omit that individual from the other groups too before analysis. It's a pity that individuals are not identified because minimally differences between individuals need to be checked too, and it's possible that there is dependence there too. I guess this what @Motmot was driving at in an answer (now deleted). $\endgroup$
    – Nick Cox
    Commented Jun 23, 2018 at 15:20

4 Answers 4

Reset to default
10
$\begingroup$

The boxplot and histogram tell you all by themselves that your data are skewed, especially in group A. The Shapiro-Wilk test is kind of pointless. With data thusly skewed the ANOVA isn't really appropriate. The Kruskal-Wallis rank sum test is based on the ranks, not the absolute values and doesn't require normality, either in the measures or residuals. It is the more appropriate test.

A quick Google search will tell you one requires normality and one does not.

One thing you might consider is that durations are an arbitrary representation of time. For example, you can indicate the duration of an event as 2 s or you can say the event has a rate 0.5 events/s. It's the exact same thing and both numbers can arbitrarily be interchanged for representation. However, rates tend to be much less skewed and more appropriate for statistical analysis. It's possible your rates are normally distributed and you can use ANOVA in that case.

If you do decide to look at rates, keep in mind that the direction of magnitude changes, a higher duration values = a lower rate value. Some people use a negative rate just to avoid that confusion.

Improve this answer
$\endgroup$
3
  • $\begingroup$ If I did the transformation into ratios correctly, in R the Shapiro-Wilk results are: shapiro.test(1/A): W = 0.9064, p-value = 0.04657 shapiro.test(1/B): W = 0.9026, p-value = 0.06388 shapiro.test(1/C): W = 0.6018, p-value = 2.057e-06 That is, at alpha=0.05, Shapiro-Wilk normality test fails for groups A and C, so Anova is not adequate again ... $\endgroup$
    – mljrg
    Commented Jul 5, 2012 at 11:50
  • 1
    $\begingroup$ It's probably not but I already indicated the test isn't the best way to tell, just look at the data. Regardless, you've got a resonable non-parametric solution. To summarize, there is no good "test" for normality because even with very normal data it will always fail if N is high enough. Nevertheless, you have a small N and the test is reflecting what you would see in plots anyway. $\endgroup$
    – John
    Commented Jul 5, 2012 at 13:26
  • 1
    $\begingroup$ (+1) The point about considering a transformed scale (here specifically, the reciprocal of duration is a rate or speed) is much more general and often the intermediate, even ideal, solution (b) between (a) working with a normal-based test even though the original data are clearly skewed and (c) jumping to a rank test. $\endgroup$
    – Nick Cox
    Commented Jun 23, 2018 at 9:06
6
$\begingroup$

The Kruskal-Wallis test and the Anova test are testing different hypotheses, both could be correct, the answers differ because they are answering different questions.

Improve this answer
$\endgroup$
3
  • $\begingroup$ The tests address slightly different hypothesis and it is clear that the medians are closer than the means. But I think the ANOVA test is still inappropriate to apply to this data and does not provide a correct answer. $\endgroup$
    – Michael R. Chernick
    Commented Jul 5, 2012 at 3:45
  • 2
    $\begingroup$ While the data is clearly skewed, with a sample size of 60 (probably 20 per group) the CLT may give a good enough approximation (multiply the p-value by 10 and it will still be significant at the 0.05 level). It would be interesting to look at permutation and/or bootstrap tests comparing the means and the medians of the groups (even though the KW test is not technically a test of medians either). $\endgroup$
    – Greg Snow
    Commented Jul 5, 2012 at 4:04
  • $\begingroup$ @GregSnow A is highly skewed because one task took much more time (the values above 21 min) than the others in A. A's effect surely makes individuals take more time in that task compared with the same task in B and C, in which individuals did not do the extra work caused by the effect of group A. However, as Kruskal-Wallis compares medians (centrality), and the durations for all other tasks are close among groups, I tend to accept the result of Kruskall-Wallis, especially as the extra time (bug fix) for the task in group A would be spent later (at bug discovery) by individuals in B and C. $\endgroup$
    – mljrg
    Commented Jul 5, 2012 at 11:11
1
$\begingroup$

Two things to keep in mind: first, ANOVA is robust in the face of non-normality if sample sizes are equal - the larger the difference in sample sizes, the less reliable it is; second, the K-W test is not a test of means or medians - it is really a test of similarity of distributions and, if the distributions are similar, it can be interpreted as a test of location. In my experience, most people ignore that both the Mann-Whitney and K-W test expect (require) the groups being compared to have similar distributions.

Several options are available for your problem. You might try a data transformation (e.g., log) to put the data on a scale that yields normal distributions in each group. Or, you might try running standard ANOVA after replacing the data with their ranks. Both approaches are effective when the assumptions for ANOVA are violated.

Improve this answer
$\endgroup$
1
$\begingroup$

The distributions all overlap very much. The Kruskal Wallis test seems to be indicating that the centers of the distributions are nearly the same. The distribution for group is highly skewed due to several very extremely high values. That is what causes the distribution to fail the Shapiro-Wilk test. The anova F test wrongly interprets group A to have a significantly larger mean because it "ignores" the skewness. The Kruskal Wallis test is giving the appropriate answer while the F test is not.

Improve this answer
$\endgroup$
4
  • $\begingroup$ It might be worthy that ANOVA tests are robust against normality assumption. $\endgroup$
    – hbaghishani
    Commented Jul 5, 2012 at 11:03
  • $\begingroup$ That is true of t tests but not the anova F test. To Greg Snow's comment it could be enlightening to look at the bootstrap distribution of the group means (particularly group A) to see how skewed it is. $\endgroup$
    – Michael R. Chernick
    Commented Jul 5, 2012 at 11:13
  • $\begingroup$ I think your point about "The anova F test wrongly interprets group A to have a significantly larger mean" makes sense in my setting especially after my comment to @GregSnow's answer, where I analyse why Kruskal Wallis seems to be the most appropriate choice here. $\endgroup$
    – mljrg
    Commented Jul 5, 2012 at 11:19
  • $\begingroup$ The general approach here, to look at the data carefully before, and after, looking at the inferential test, is very wise. However, in this case I'm not sure characterizing 7 out of 21 data points as few enough values to argue that the medians should be focused on exclusively. $\endgroup$
    – John
    Commented Aug 27 at 12:55

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.