8
$\begingroup$

Andrew Gelman in a recent post in his blog suggests using a rank, transforming the rank to a z-score, and then using parametric tests and tools instead of performing non-parametric tests. I never heard of that before.

A search on Google pointed me to this R function in the package GenABEL, which seems to perform the rank+z-transform of a data vector but I could not find papers that evaluate or discuss the idea of using parametric tests on the transformed data instead of Wilcoxon tests.

Can anyone point me to some literature on this method?

Improve this question
$\endgroup$
2
  • 2
    $\begingroup$ This is not a parametric test. You've just using an asymptotic approximation of the null distribution of a rank-based statistic. $\endgroup$
    – dsaxton
    Jul 21, 2015 at 20:50
  • 1
    $\begingroup$ @dsaxton In many cases it is a nonparametric test (or, rather, an approximation to one). Replacing ranks with any fixed (given the sample sizes) set of scores doesn't alter the lack of dependence on the original distribution shapes under the null. $\endgroup$
    – Glen_b
    Apr 7, 2017 at 11:16

1 Answer 1

Reset to default
10
$\begingroup$

(Pulls Conover [1] off the bookshelf...)

This idea is quite old; it dates back at least to van der Waerden (1952/1953) [2][3], who suggested a test that corresponds to the Kruskal Wallis but with ranks replaced by normal scores. (The idea of using ordered random normal values rather than an approximation of their expectation or their median - is perhaps even a little older.)

According to Conover, Fisher and Yates (1957) [4] suggest replacing observations with expected normal scores (i.e. transformed ranks) in a variety of tests where normality would be assumed.

The asymptotic relative efficiency at the normal will be 1, which makes it sound quite attractive ... however, the advantage over say the Wilcoxon-Mann-Whitney (gain in power) -- even at the normal -- is quite small, and if the distribution is heavier tailed than normal (say logistic), it may be disadvantageous to do this. (Some simulation suggests that it is in fact the case: unless the distribution is close to normal already -- in which case there's no benefit to doing the transformation -- such a transformation may actually lose power.)

Chernoff & Lehmann [5] calculate asymptotic power for a variety of distributions; where there's at least one very short tail (such as the uniform), the normal scores test can have much better ARE for a shift alternative against the Wilcoxon-Mann-Whitney -- better than the t-test itself does. Their results agree with my simulations for heavier tailed cases.

Note that in the two-sample case, as the separation in means becomes large, while the combined sample looks quite normal, the two samples are not normal at all:

enter image description here

As a result, not all properties of the normal test will carry over to the normal scores test, and the behaviour at larger separations (with small samples) may be somewhat counterintuitive.

The tests obtained by this idea are sometimes collectively called normal scores tests, which search term (via Google, say) turns up a number of references.

For example, here, Richard Darlington discusses doing it for the Wilcoxon signed ranks test; he points out there's an advantage over the plain rank test, because it reduces the number of tied values of the test statistic.

Before I end up writing pages on it, I'll leave you to search further.

Conover lists a number of other references and has a fair bit of discussion, so I'd definitely recommend reading that.

Gelman's point, however, seems to be about convenience - not needing to develop a new test each time the situation changes; though if convenience is the main issue there's already the ability to use permutation tests on whatever statistic we like. [With the normal scores approach, the difficulty is we still need a suitable way to rank -- you can't just rank things that aren't comparable under the null and expect the right sort of behaviour. There's a similar problem with the permutation test, since you similarly need exchangeability under the null.]

You mention an R function, but you can rank and convert to normal scores easily in R just using functions that already come with R.

e.g. using the sleep data in R. you'd do a t-test this way:

 t.test(extra ~ group, data = sleep)  # Welch
 t.test(extra ~ group, data = sleep, var.equal=TRUE)  # equal-variance
 t.test(qqnorm(extra,plot=FALSE)$x ~ group, data = sleep)  # normal scores

[1] Conover, W. J. (1980),
Practical Nonparameteric Statistics, 2e.
Wiley. pp. 316–327.

(From the above Wikipedia link it looks like in 3e (1999) the discussion starts on p396)

[2] van der Waerden, B.L. (1952),
"Order tests for the two-sample problem and their power",
Proceedings of the Koninklijke Nederlandse Akademie van Wetenschappen, Serie A 55 (Indagationes Mathematicae 14), 453–458.

[3] van der Waerden, B.L. (1953),
"Order tests for the two-sample problem. II, III",
Proceedings of the Koninklijke Nederlandse Akademie van Wetenschappen, Serie A 56 (Indagationes Mathematicae, 15), 303–310 & 311–316.

(there are also corrections to the 1952 paper on p 80 of that volume)

[4] Fisher R.A. and Yates F. (1957)
Statistical Tables for Biological, Agricultural and Medical Research, 5e, Oliver & Boyd, Edinburgh.

[5] Hodges, J. L.; Lehmann, E. L. (1961),
"Comparison of the Normal Scores and Wilcoxon Tests,"
Proceedings of the Fourth Berkeley Symposium on Mathematical Statistics and Probability, Volume 1: Contributions to the Theory of Statistics, 307--317,
University of California Press, Berkeley, Calif.
http://projecteuclid.org/euclid.bsmsp/1200512171.

Improve this answer
$\endgroup$
8
  • $\begingroup$ Glen, this an interesting topic (which I'd like to learn more about) and a good answer of yours. But can I beg you to add some flesh (discourse/conclusions) to it besides just links? And, possibly, collect some links to similar Q/A on this site? Thank you. $\endgroup$
    – ttnphns
    Jul 22, 2015 at 11:10
  • $\begingroup$ Great topic. Are there any published apples-to-apples comparisons of a nonparametric approach and a normal scores approach (with parametric test) for tests of differences in means? My colleague and I have some for tests of correlations, but that's all I'm aware of. In the correlation context, normal scores lead to slightly better power than the rank test (Spearman correlation) or the untransformed parametric test (Pearson). I'm curious to know if similar results occur in other contexts. $\endgroup$
    – Anthony
    Jul 22, 2015 at 11:40
  • 1
    $\begingroup$ @Antony. The item [5] pointed by Glen, contains an apple-to-apple comparison of normal scores and Wilcoxon based on asymptotic efficiency. Scholar lists 114 citations to that paper, some of them should provide other forms of comparison. $\endgroup$
    – Jacques Wainer
    Jul 22, 2015 at 17:00
  • 1
    $\begingroup$ @Glen_b, for correlations, the normal scores advantage was most prominent when both variables were extremely non-normal (e.g., both ~$\chi^2, df=1$) and n was large (>=20, but even larger showed bigger advantages). Other work suggests both skewness and kurtosis matter. If interested, the reference can be found here: stats.stackexchange.com/questions/131369/… $\endgroup$
    – Anthony
    Jul 22, 2015 at 17:56
  • 1
    $\begingroup$ Sorry for all the comments, but I can't hold back on such a fun thread! One more thought - @Glen_b's figure points to a problem that was called "inheritance" by Zimmerman (2011). Even after transformation, the separate samples may "inherit" some of the properties of the original, untransformed distribution. redalyc.org/articulo.oa?id=16917012005 $\endgroup$
    – Anthony
    Jul 22, 2015 at 18:18

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.