Skip to main content
6 of 11
added 103 characters in body
Glen_b
  • 290.5k
  • 37
  • 652
  • 1.1k

I believe it's legitimate to quote the relevant portion of the paragraph in question:

3. The KS test can not be applied in two or more dimensions. Astronomers often have datasets with points distributed in a plane or higher dimensions, rather than along a line. Several papers in the astronomical literature purport to present a two-dimensional KS test, and one is reproduced in the famous volume Numerical Recipes. However, no EDF-based test (this includes KS, AD and related tests) can be applied in two or higher dimensions, because there is no unique way to order the points so that distances between well-defined EDFs can be computed. One can construct a statistic based on some ordering procedure, and then compute the supremum distances between two datasets (or one dataset and a curve). But the critical values of the resulting statistic are not distribution-free.

This is simply wrong. The bivariate distribution function, which is $F(x_1,x_2) = P(X_1\leq x_1,X_2\leq x_2)$ is a map from $\mathbb{R}^2$ to $[0,1]$. That is, the function takes univariate real values between 0 and 1. Those values are certainly "ordered" already - and this (the value of the function) is the thing we need to make comparisons on for ECDF-based tests.

Similarly, the ecdf, $\hat F$ is perfectly well defined in the bivariate case.

There's no need to "order" the bivariate $X$ values*. You simply compute $F$ and $\hat F$ at every required combination and compute the difference. There's no need to try to turn it into some function of a univariate combined variable.

* at least not beyond the sense of the partial order on subsets of the X-space already used by the definition of cumulative distribution functions, at least.

(My bivariate comments carry over to higher-dimension X-vectors as well, but let's stick with the bivariate case for the moment.)

There's no difficulty, no problem, for all their assertions to the contrary.

This will prove to be somewhat of an embarrassment for the authors once they come to understand their (quite basic) error. I'd be surprised if there's not (eventually) a retraction of the paper from the site. It's perhaps unfortunate that these errors are now committed "in public", but that's something we have to get used to.


That they have put up a paper with an error in it is not a particular problem, necessarily. People make mistakes, and other people pick them up. This would not survive competent peer review -- indeed that's the point of peer review. Not everything that gets put up on a university web site has been through that process, and errors are to be expected (and sometimes occur even with that process). I know I've had working papers that had at least the odd gap or small error; other people read them and ask questions, and they get sorted out.

In this case, that misplaced criticism is kind of the point of the paper, though, so it will probably simply be dropped as there's not much left over to rescue. That happens too.

[We should also note that this doesn't appear to be even a working paper on the way to a formal submission to a journal, but simply something they put up, perhaps as a warning to their colleagues; it may never be subject to any formal review process. I note that the forum they suggest for discussion of the paper only has a couple of posts for this whole year. Something that inactive isn't going to be a source of many critical eyes, and quite possibly has few people with much formal background in statistics, so the necessary nudge they'll need to reconsider their position may not some as quickly as one might hope. I'd happily take it up with them on our chat forum, were they inclined to do so.]

Glen_b
  • 290.5k
  • 37
  • 652
  • 1.1k