3
$\begingroup$

I need to test for differences between non-normally distributed and heteroscedastic data.

I know that I can use the Kruskal-Wallis test to assess significant differences between groups but one assumption of this test is homoscedasticity. I tested for homoscedasticity (i.e similar variance around groups) with Levene and Bartlett test and my data was found not to be homoscedastic.

One way ANOVA is not the solution because my data aren't normally distributed even when log-transforming the data.

I thought on applying Welch's ANOVA but the data doesn't follow a normal distribution either.

So, is there any test to asses significant differences between groups given a heteroscedastic and non-normally distributed data ?

$\endgroup$
3
  • 1
    $\begingroup$ Why do you say that KW requires homoscedasticity? $\endgroup$
    – Sinval
    Oct 17, 2021 at 13:13
  • 2
    $\begingroup$ Include a histogram of your data. Did you test the residuals or the data (response) for normality? $\endgroup$
    – AdamO
    Oct 17, 2021 at 13:14
  • $\begingroup$ Tests of variance with large sample sizes can (correctly!) flag practically insignificant variance differences as statistically insignificant. When you plot your data, what do you see? $\endgroup$
    – Dave
    Oct 17, 2021 at 13:32

2 Answers 2

6
$\begingroup$

Despite many books saying so, the Krusal-Wallis test does not need to assume equal spread, except under the null; you want it under the null for exchangeability, which is what you need for a permutation test since you need to have all permutations be equally likely under the null.

Sal Magniafico's answer describes Conover's take on this test; in particular I agree with Conover's item 4, but I think additional discussion of the issue there is warranted.

It's perfectly reasonable to posit a sequence of alternatives that have spread and even shape changing as distributions move up or down as long as it's not changing so rapidly that it impacts the tendency for higher ranks to be more predominant as the distribution moves up (and vice versa, to have lower ranks as it moves down).

I'll focus on two groups in the remaining discussion, for simplicity's sake, except for one or two asides.

Typically you'd look for sequences of alternatives where for each one in sequence the cdf is lower than the one before (everywhere $\leq$ and somewhere $<$) as the distribution moves "up"/right along the number line, and vice-versa in the other direction; this situation is unambiguous in relation to the distribution moving "up" and "down".

(However, even that much is not strictly necessary; any increasing tendency to be larger would still work just fine, though it then becomes somewhat easier to come up with situations that are more difficult to interpret [1], and conclusions may need to be more carefully worded. I'll focus for now on sequences where the sequence of cdfs under the alternative moves monotonically)

Consider that the Kruskal-Wallis is unimpacted by any strictly monotonic-increasing transformation (the behavior and properties of ranks are unchanged); if you had constant spread on one scale, other monotonic but non-linear transformations of the same variables would not have have constant spread - but the test would not change.

For example, the Kruskal-Wallis would be perfectly valid for testing a change of scale with gamma-distributed variates (with constant shape parameter); the spread and the mean change in proportion.

By the same token you could consider a sequence of beta variates where the spread and the shape change as the distribution moves up or down (e.g. hold one shape parameter - say $\beta$ - constant and allow the other - $\alpha$ - to vary). Again the Kruskal-Wallis test is perfectly valid for this.

You might add an assumption of pure location-shift if that's what you're interested in, but it's not a necessary part of the test you're using.

Similar arguments can often be made for many other tests -- and not just permutation tests: properties you need under the null (to get a desired significance level) you often don't need under the alternative, as long as the sequence of alternatives is not changing those things too rapidly as you move away from the null; you may well retain most of the properties you're interested in - consistency, good power [2] and so on without insisting on the assumptions you need under the null still holding always and everywhere under the set of alternatives.


Testing assumptions on data

With data, you don't know whether the null is true, so it's pointless checking the data for an assumption that you only require under the null. That's not to say you don't consider your assumptions with care -- they're important and indeed I think they often don't get sufficient attention, but when they do get attention it's entirely misplaced (and often misleading) by focusing on the particulars of the data at hand; this is problematic on multiple fronts.


What if you don't want to assume constant spread under the null

This can be handled, if you're clear about what parameter you're interested in (say, means). One approach would be to construct a bootstrap test. For example, you could construct a bootstrap confidence interval for $\mu_2-\mu_1$ and see if it includes $0$. However, you have to pay attention to the coverage properties of your interval, so typically you will want sample sizes that are not too small.


Parenthetical asides/footnotes

[1]: such as sequences of cyclical dominance (e.g. $P(X_{j+1}>X_{j})>\frac12, j=1,...,k-1$, but $P(X_{1}>X_{k})>\frac12$). The Kruskal-Wallis is not sensitive to this sort of thing, but a collection of pairwise Wilcoxon-Mann-Whitney tests would be.

[2]: the specific power curves will of course change, but if the properties you need under the null don't change too quickly the behavior may still be very sensible. (You might be able to obtain more powerful tests if you know the specific sequence of alternatives, but you don't have to know that much -- if you're prepared to make some fairly reasonable but weaker assumptions about the behavior under the alternative, you may have a perfectly usable test in a broad range of conditions.)

$\endgroup$
3
$\begingroup$

These are the assumptions for the Kruskal-Wallis test given by Conover, 1999, Practical Nonparametric Statistics, 3rd.

  1. All samples are random samples from their respective populations.
  2. In additional to independence within each sample, there is mutual independence among the various samples.
  3. The measurement scale is at least ordinal.
  4. Either the k population distribution functions are identical, or else some of the populations tend to yield larger values than other populations do.

I think some authors will sometimes substitute a homoscedasticity assumption for assumption 4 given here.

Assumption 4 seems like an odd assumption, but I think it's necessary because if you start with two populations that have the same location and the same distribution shape, but different variances, when you sample these, you are likely to get two samples that the K-W test may detect as different, even though they are stochastically similar at the population level. The upshot being that without assumption 4, you can get an inflated type-I error rate in these cases. I'm not entirely sure about this; I'd have to double check with some simulations.

If I understand all this correctly, assumption 4 is superior to having a homoscedasticity assumption. The homoscedasticity assumption is too restrictive because it prevents you from addressing those populations that are heteroscedastic and that have differences in stochastic dominance that the K-W test could detect.

Hopefully that all makes some sense, and I'm not too far from accurate.

$\endgroup$
1
  • 1
    $\begingroup$ hey thanks so much for this analysis! $\endgroup$
    – Valentin
    Oct 17, 2021 at 23:00

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.