Skip to main content
4 of 11
added 419 characters in body
Glen_b
  • 290.5k
  • 37
  • 652
  • 1.1k

I believe it's legitimate to quote the relevant portion of the paragraph in question:

3. The KS test can not be applied in two or more dimensions. Astronomers often have datasets with points distributed in a plane or higher dimensions, rather than along a line. Several papers in the astronomical literature purport to present a two-dimensional KS test, and one is reproduced in the famous volume Numerical Recipes. However, no EDF-based test (this includes KS, AD and related tests) can be applied in two or higher dimensions, because there is no unique way to order the points so that distances between well-defined EDFs can be computed. One can construct a statistic based on some ordering procedure, and then compute the supremum distances between two datasets (or one dataset and a curve). But the critical values of the resulting statistic are not distribution-free.

This is simply wrong. The bivariate distribution function, which is $F(x_1,x_2) = P(X_1\leq x_1,X_2\leq x_2)$ is a map from $\mathbb{R}^2$ to $[0,1]$. That is, the function takes univariate real values between 0 and 1. Those values are certainly "ordered" already.

Similarly, the ecdf, $\hat F$ is perfectly well defined in the bivariate case.

There's no need to "order" the bivariate $X$ values*. You simply compute $F$ and $\hat F$ at every required combination and compute the difference. There's no need to try to turn it into some function of a univariate combined variable.

* at least not beyond the sense of the partial order on subsets of the X-space already used by the definition of cumulative distribution functions, at least.

(My bivariate comments carry over to higher-dimension X-vectors as well, but let's stick with the bivariate case for the moment.)

There's no difficulty, no problem, for all their assertions to the contrary.

This will prove to be somewhat of an embarrassment for the authors once they come to understand their (quite basic) error. I'd be surprised if there's not a retraction of the paper from the site some time very soon. It's perhaps unfortunate that these errors are now committed "in public", but that's something we have to get used to.


That they have put up a paper with an error in it is not a particular problem, necessarily. People make mistakes, and other people pick them up. This would not survive competent peer review -- indeed that's the point of peer review. Not everything that gets put up on a university web site has been through that process, and errors are to be expected (and sometimes occur even with that process). I know I've had working papers that had at least the odd gap or small error; other people read them and ask questions, and they get sorted out.

In this case, that's kind of the point of the paper, though, so it will probably simply be dropped as there's not much left over to rescue. That happens too.

Glen_b
  • 290.5k
  • 37
  • 652
  • 1.1k