20
$\begingroup$

I am learning R and have been experimenting with analysis of variance. I have been running both

kruskal.test(depVar ~ indepVar, data=df)

and

anova(lm(depVar ~ indepVar, data=dF))

Is there a practical difference between these two tests? My understanding is that they both evaluate the null hypothesis that the populations have the same mean.

$\endgroup$
28
$\begingroup$

There are differences in the assumptions and the hypotheses that are tested.

The ANOVA (and t-test) is explicitly a test of equality of means of values. The Kruskal-Wallis (and Mann-Whitney) can be seen technically as a comparison of the mean ranks.

Hence, in terms of original values, the Kruskal-Wallis is more general than a comparison of means: it tests whether the probability that a random observation from each group is equally likely to be above or below a random observation from another group. The real data quantity that underlies that comparison is neither the differences in means nor the difference in medians, (in the two sample case) it is actually the median of all pairwise differences - the between-sample Hodges-Lehmann difference.

However if you choose to make some restrictive assumptions, then Kruskal-Wallis can be seen as a test of equality of population means, as well as quantiles (e.g. medians), and indeed a wide variety of other measures. That is, if you assume that the group-distributions under the null hypothesis are the same, and that under the alternative, the only change is a distributional shift (a so called "location-shift alternative"), then it is also a test of equality of population means (and, simultaneously, of medians, lower quartiles, etc).

[If you do make that assumption, you can obtain estimates of and intervals for the relative shifts, just as you can with ANOVA. Well, it is also possible to obtain intervals without that assumption, but they're more difficult to interpret.]

If you look at the answer here, especially toward the end, it discusses the comparison between the t-test and the Wilcoxon-Mann-Whitney, which (when doing two-tailed tests at least) are the equivalent of ANOVA and Kruskal-Wallis applied to a comparison of only two samples; it gives a little more detail, and much of that discussion carries over to the Kruskal-Wallis vs ANOVA.

It's not completely clear what you mean by a practical difference. You use them in generally a generally similar way. When both sets of assumptions apply they usually tend to give fairly similar sorts of results, but they can certainly give fairly different p-values in some situations.

Edit: Here's an example of the similarity of inference even at small samples -- here's the joint acceptance region for the location-shifts among three groups (the second and third each compared with the first) sampled from normal distributions (with small sample sizes) for a particular data set, at the 5% level:

Acceptance regions for location-differences in Kruskal-Wallis and Anova

Numerous interesting features can be discerned -- the slightly larger acceptance region for the KW in this case, with its boundary consisting of vertical, horizontal and diagonal straight line segments (it is not hard to figure out why). The two regions tell us very similar things about the parameters of interest here.

$\endgroup$
  • 2
    $\begingroup$ +1. I dared to edit it slightly just to add emphasis where I thought it necessary. Please see now, whether you agree or not. $\endgroup$ – ttnphns Nov 10 '13 at 8:58
  • $\begingroup$ @ttnphns thanks for the edit. There are some particular reasons why some of the things you changed were in there, so I may edit some of the original back in. However, perhaps I should make it clearer why I wrote it as I had it before. But first I want to think carefully about how best to keep as much of your changes as I can. $\endgroup$ – Glen_b Nov 10 '13 at 9:01
4
$\begingroup$

Yes there is. The anova is a parametric approach while kruskal.test is a non parametric approach. So kruskal.test does not need any distributional assumption.
From practical point of view, when your data is skewed, then anova would not a be good approach to use. Have a look at this question for example.

$\endgroup$
  • 4
    $\begingroup$ I would say that Kruskal-Wallis ANOVA makes relaxed assumptions regarding distributions compared to parametric ANOVA: observations in each group come from populations with similar shape. Heteroskedasticity or highly skewed distributions remain as problematic as with traditional tests. $\endgroup$ – chl Nov 9 '13 at 20:22
  • 2
    $\begingroup$ How so, @chl ? The ranks aren't changed by skew, and KW is rank based. What am I missing? $\endgroup$ – Peter Flom - Reinstate Monica Nov 9 '13 at 20:53
  • 6
    $\begingroup$ @PeterFlom The KW test assumes that sampled populations have identical shape and dispersion, although in most case little departure from those assumptions will not affect the results. When parametric assumptions are met, the test is $3/\pi$ as powerful as one-way ANOVA. Regarding rank-based test statistics, some studies suggest, however, that varying degrees of skewness may inflate nominal type I error rate, see, e.g., Fagerland and Sandvik (2009), or some other references. $\endgroup$ – chl Nov 9 '13 at 21:17
  • $\begingroup$ @chl The $H_0$ hypothesis is the equality of the distributions, thus the identical shape assumption is only related to the power, isn't it ? $\endgroup$ – Stéphane Laurent Nov 9 '13 at 22:54
  • 1
    $\begingroup$ @StéphaneLaurent If the shapes are not identical it may lead to bad inference. see my example here $\endgroup$ – Flask Nov 10 '13 at 5:55
3
$\begingroup$

As far as I know (but please correct me if I'm wrong cause I'm not sure), the Kruskal-Wallis test is constructed in order to detect a difference between two distributions having the same shape and the same dispersion, that is, one is obtained by translating the other by a difference $\Delta$, such as: enter image description here

Let's call $(*)$ this assumption. The KW test tests the null hypothesis $H_0\colon\{\Delta=0\}$ vs $H_1\colon\{\Delta \neq 0\}$. However, the KW test is "valid" without assumption $(*)$ : its signficance level (probability to reject $H_0$ under $H_0)$ is valid because $(*)$ is obviously fulfilled under $H_0\colon\{\text{the distributions are equal}\}$.

But the KW test is "inefficient" if $(*)$ does not hold: it only intend to have a good power to detect $\Delta >0$, and then the test statistic is not appropriate to reflect the difference between the two distributions if there's no such $\Delta$.

Consider the following example. Two samples $x$ and $y$ of size $n=1000$ are generated from two quite different distributions but having the same mean. Then KW fails to reject $H_0$.

set.seed(666)
n <- 1000
x <- rnorm(n)
y <- (2*rbinom(n,1,1/2)-1)*rnorm(n,3)
plot(density(x, from=min(y), to=max(y)))
lines(density(y), col="blue")

enter image description here

> kruskal.test(list(x,y))

    Kruskal-Wallis rank sum test

data:  list(x, y)
Kruskal-Wallis chi-squared = 2.482, df = 1, p-value = 0.1152

As I claimed in the beginning, I'm not sure about the precise construction of KW. Maybe my answer is more correct for another nonparametric test (Mann-Whitney ?..), but the approach should be similar.

$\endgroup$
  • 1
    $\begingroup$ Kruskal-Wallis test is constructed in order to detect a difference between two distributions having the same shape and the same dispersion As mentioned in Glen's answer, the comments and in many other places on this site, it is true but is the narrowed reading of what the test does. same shape/dispersion is actually not an intrinsic but is an additional assumption which is used in some and not used in other situations. $\endgroup$ – ttnphns Nov 10 '13 at 13:59
  • $\begingroup$ P.S. Your 2nd example does not contradict or refute KW test. The H0 of the test is not distributions are equal, it is a mistake to think so. The H0 is only that, figularly, the two points of "condensation of the gravities" do not deviate from each other. $\endgroup$ – ttnphns Nov 10 '13 at 15:25
  • $\begingroup$ @ttnphns I believe you, I don't know. But commonly we consider $H_0$ as the equality (see e.g. the article on wikipedia). $\endgroup$ – Stéphane Laurent Nov 10 '13 at 15:56
  • 1
    $\begingroup$ I just say this is a common belief. According to the help of krusal.test() in R, $H_0$ is the equality of the location parameters of the distribution. In practice I think we often use KW to assess a difference between the distributions. Hence we could assume the same shape (as we do in the Gaussian ANOVA case), and apply KW, this makes sense. $\endgroup$ – Stéphane Laurent Nov 10 '13 at 16:24
  • 1
    $\begingroup$ Yeah. the equality of the location parameters of the distribution is the right formulation (albeit "location" shouldn't be thought of as just a mean or median, in general case). If you assume same shapes, then, naturally, this same H0 becomes "identical distribution". $\endgroup$ – ttnphns Nov 10 '13 at 16:32
0
$\begingroup$

Kruskal-Wallis is rank based, rather than value-based. This can make a big difference if there are skewed distributions or if there are extreme cases

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.