0
$\begingroup$

As the title says, what is exactly what is being tested before deciding to use a non-parametric alternative test (as Kruskal-Wallis for ANOVA, or Mann-Whitney's U for student's t)?

Most sources are not specific on this issue, being content with just referring to the "normality of the data" (e.g. statistics how to). This leaves us with two options, either the normality of the residuals or the normality of the response variable, and looking online both arguments are used.

For "data normality" as "response normality" we have other sources (e.g. Boston University School of Public Health, and here). For example, in the Biostatistics Handbook by John McDonald which on his Kruskal-Wallis section refers to the normality assumption, which gives a sequence of examples of normality testing on histograms and distributions of different response variables:

Parametric tests assume that your data fit the normal distribution. If your measurement variable is not normally distributed, you may be increasing your chance of a false positive result if you analyze the data with a test that assumes normality.

Also stating that residuals come as a second option:

If there are not enough observations in each group to check normality, you may want to examine the residuals (each observation minus the mean of its group).

On the other hand, for "residual normality", Wikipedia is on board with it:

Since it is a non-parametric method, the Kruskal–Wallis test does not assume a normal distribution of the residuals, unlike the analogous one-way analysis of variance.

As well as other answers on this website (where in fact the accepted, but not top voted, answer refers to response variable normality)

I am inclined to think it is residual normality what matters, but I am actually curious on why it seems to be such a common error—that actually makes its way into academic contexts, handbooks and "general knowledge"—and if there's some weight or sense into the "response variable normality" position.

Improve this question
$\endgroup$
7
  • 2
    $\begingroup$ Regarding your title, there's no requirement whatever that the data be non-normal to use a nonparametric test. Many nonparametric tests work very well on normally distributed data. The premise of that question is false. $\endgroup$
    – Glen_b
    Aug 2, 2019 at 0:30
  • $\begingroup$ Some of your questions are answered in one of your own links; Peter Flom's answer and clarifications by commenters deal with what is assumed normal in a regression model (& so ANOVA + t tests); as Peter says, it's the errors (not the marginal distribution of Y), and commenters clarify that it's also equivalently an assumption about the conditional distribution of the response. [The conditional-distribution view is more general, since it works in cases where there's no additive error term, e.g. generalized linear models] ... ctd $\endgroup$
    – Glen_b
    Aug 2, 2019 at 1:31
  • $\begingroup$ ctd ... On some other aspects of the Q, you may also get some value from this: What to do when Kolmogorov-Smirnov test is significant for residuals of parametric test but skewness and kurtosis look normal?. $\:$ If anything remains unclear after careful reading of the various threads in the duplicates and other mentioned posts, please post a new question. $\endgroup$
    – Glen_b
    Aug 2, 2019 at 1:38
  • $\begingroup$ The facts of the matter are easily established by simply doing the mathematics of deriving the relevant tests; there's no possible dispute on what is assumed to derive a t-distribution and no merit whatever (despite its popularity in what by now would be literally thousands of books that avoid the mathematics and whose authors appears to fail to understand what is otherwise obvious) in the position that marginal normality matters for regression, ANOVA or t-tests. The question at the very end "why it's in so many books" is not a duplicate but might not be on topic, since that's ... ctd $\endgroup$
    – Glen_b
    Aug 2, 2019 at 1:47
  • $\begingroup$ ctd... more about human behavior and psychologists might be a better source to answer questions about confidence of (and in) non-experts. It may tend to relate to things like the Dunning-Kruger effect and perhaps to some extent to economic behavior (more money for our department if we teach our own stats courses -- and oh, now we need to turn these notes into a textbook -- times thousands of departments across various disciplines and universities). A book's popularity within some discipline may be a poor indicator of its value $\endgroup$
    – Glen_b
    Aug 2, 2019 at 1:50

1 Answer 1

Reset to default
3
$\begingroup$

There is no requirement for data to be 'non-normal' in order to do a nonparametric test. However, if data (or residuals, as appropriate) are normal, it is often simplest and best to do the normal test: two-sample t, paired t, ANOVA, simple linear regression, etc. The assumption of normality provides information. If you have that information, it may be best to use it in your analysis.

Tests for normal data. In the two sample tests below, information that the data are normal, allows use of a Welch 2-sample t test, with a result significant at the 5% level. However, 2-sample Wilcoxon test does not find a significant difference. (To be sure, the power of the t test for normal data is not enormously greater than the power of the Wilcoxon test, but cases like this one can arise--especially in large datasets.)

x = c(101.1, 102.3, 97.5, 102.4, 93.1)
y = c(101.0, 115.3, 102.2, 107.3, 112.9)
wilcox.test(x,y)$p.val
[1] 0.1507937
t.test(x,y)$p.val
[1] 0.04046328

Assumptions of randomness and independence for nonparametric tests. Nonparametric tests do not require normal data, but nonparametric tests do require that some assumptions be met. Examples:

(a) A paired test using the Wilcoxon signed-rank test should obviously use paired data where each pair is independent of others. Such as, differences between before and after assessments of $n$ randomly chosen subjects.

(b) A two-sample Mann-Whitney-Wilcoxon (rank sum test) should use data from two independent samples. Such as, subjects randomly chosen and randomly assigned to Treatment and Control groups. And we are testing whether the treatment may have shifted the data values up or down.

(c) Similarly, a Kruskal-Wallis test should have $g \ge 2$ independent groups, and data for a Friedman test should have been collected according to a block design.

Ties in nonparametric analysis. In addition, the traditional nonparametric tests just mentioned all use ranks of data instead of the original data values. The data values are theoretically continuous numerical variables. Ranking can lose information. Distribution theory for ranks requires adjustments if there are ties (which would be impossible in unrounded continuous data). While implementations of these tests in modern statistical software may make very clever adjustments, the use of these tests when there are hugely many ties in the data may not be appropriate.

Minimal sample sizes for some nonparametric tests. Some nonparametric tests require a minimum number of observations in order to give a significant result. Below we show a two-sample Wilcoxon test with only three observations in each of the two groups. Even with clearly higher values in Group 2 than in Group 1, the test simply is incapable of giving a significant result at the 5% level.

wilcox.test(c(1,2,3), c(101,102,103))

        Wilcoxon rank sum test

data:  c(1, 2, 3) and c(101, 102, 103)
W = 0, p-value = 0.1
alternative hypothesis: true location shift is not equal to 0

Computationally intensive nonparametric procedures. Nonparametric boostrap confidence intervals and simulated permutation tests can be used when data are not normal. Generally, these require massive computation. Before you use these procedures, it is important that you know their theoretical assumptions and make sure you know how to do the computation. These tests are an important part of modern statistical analysis, so it is worthwhile to learn how to use them correctly.

Improve this answer
$\endgroup$
2
  • 1
    $\begingroup$ +1 ... In the case where someone is especially worried about the small power difference at the normal, one may substitute a fully efficient (at the normal) nonparametric test such as a permutation test for a difference in means, or even a permutation test based off the t-statistic itself. $\endgroup$
    – Glen_b
    Sep 23, 2019 at 5:17
  • $\begingroup$ @Glen_b. Just recently did such a permutation test at the end of this Answer. $\endgroup$
    – BruceET
    Sep 23, 2019 at 6:26

Not the answer you're looking for? Browse other questions tagged or ask your own question.