4
$\begingroup$

I am reading the Wikipedia page on the Kolmogorov-Smirnov test, specifically the section titled Kolmogorov_distribution.

$x$ is $D_\text{max}$ in the CDF $Pr(k\leq x)$?

My question is the number of points matters too,right?

When $n$ is large, $x$ approximates $\sqrt n D_\text{max}$ ?

Here is a snapshot of the section from Wikipedia:

enter image description here


enter image description here

Q1
How do we obtain the distribution of $D_n$ when $n$ is fixed?

Q2
If I get the value of $D_\text{max}$ and the sample size is $n$, I have to calculate $Pr(K<=x)$, right? Is $D_\text{max}$ the $x$ in the formula on the Wikipedia page?

By formula 14.3.9 of Numerical Recipes, we should calculate a value got from the expression in the brackets - should that be the $x$? The value approximates $\sqrt n D_\text{max}$ when $n$ is large.

I am totally confused here.

Q3
We make tests and get a distribution,right? Could you please explain your figure in a "test" way? Maybe that is easier to follow.


I calculate several values(significance)and compare them with the tableks table

The $x$ in $Pr(K<=x)$ is not $D_{max}$ .If sample size is large,x is $(\sqrt{n}+0.12+0.11/\sqrt{n})*D_{max}$ .That means there is no error in 14.4.9 of NR . The significance we want to get is determined by sample size and $D_{max}$ .

Please let me explain my questions in a "test" way.

Suppose sample size is 30, we obtain a dataset of 30 points and we can get a $D_{max}$ from the comparison between the empirical distribution of the sample and the reference probability distribution. We do it 1000 times and we get 1000 $D_{max}$ . There is a distribution for $D_{max}$ ,right?

From your figure , there should be 1000*0.01 points with a $D_{max}$ larger than 0.29 and 1000*0.05 points with a $D_{max}$ larger than 0.24 ,more or less .


Like you said, I am very confused. Please let me make sure the several statements below are right or wrong first.

When sample size is large,my calculation method is right?
we can take $(\sqrt{n}+0.12+0.11/\sqrt{n})*D_{max}$ as x to input to $Pr(K<=x)$ and in this way we can get the significance when sample size is large,right?

Suppose sample size is 30, we obtain a dataset of 30 points and we can get a $D_{max}$ from the comparison between the empirical distribution of the sample and the reference probability distribution. We do it 1000 times and we get 1000 $D_{max}$ . There is a distribution for $D_{max}$ ,right?

From your figure , there should be 1000*0.01 points with a $D_{max}$ larger than 0.29 and 1000*0.05 points with a $D_{max}$ larger than 0.24 . Is that your figure tells us ?

$\endgroup$
4
  • 1
    $\begingroup$ Using the notation of that page, $D_n$ is often regarded as 'the test statistic' (as shown in this image on the same page), but in the discussion you quote, $\sqrt n D_n$ is being used as a test statistic. Obviously the two statistics would be equivalent (lead to rejection/non-rejection of exactly the same cases). $\endgroup$ – Glen_b Oct 19 '13 at 21:00
  • $\begingroup$ Do exactly what 1000 times? The only meaningful thing to do 1000 times is to draw the sample of a given size from the reference probability distribution to get the simulated approximation to the finite sample distribution of $D_{\max}$ -- which, as @Glen_b explained, can be found exactly, without the need to simulate. $\endgroup$ – StasK Oct 22 '13 at 15:14
  • $\begingroup$ You should carefully define the terms in your update to the question. $\endgroup$ – Glen_b Oct 23 '13 at 22:26
  • $\begingroup$ I have modified several sentences. Because of my poor knowledge of statistics, I have tried my best to explain my confusions in a plain but clear way. Those statements are right? $\endgroup$ – questionhang Oct 24 '13 at 15:20
13
$\begingroup$

Note that the Kolmogorov-Smirnov test statistic is very clearly defined in the immediately previous section:

$$D_n=\sup_x|F_n(x)−F(x)|\,.$$

The reason they discuss $\sqrt{n}D_n$ in the next section is that the standard deviation of the distribution of $D_n$ goes down as $1/\sqrt n$, while $\sqrt{n}D_n$ converges in distribution as $n\to\infty$.

Yes, the number of points, $n$, matters to the distribution; for small $n$, tables are given for each sample size, and for large $n$ the asymptotic distribution is given for $\sqrt{n}D_n$ $-$ the very same distribution discussed in the section you quote.

Without some result on asymptotic convergence in distribution, you'd have the problem that you'd have to keep producing tables at larger and larger $n$, but since the distribution of $\sqrt{n}D_n$ pretty rapidly 'stabilizes', only a table with small values of $n$ is required, up to a point where approximating $\sqrt{n}D_n$ by the limiting Kolmogorov distribution is sufficiently good.

Below is a plot of exact 5% and 1% critical values for $D_n$, and the corresponding asymptotic critical values, $K_\alpha/\sqrt n$.

Kolmogorov-Smirnov critical values

Most tables finish giving the exact critical values for $D_n$ and swap to giving the asymptotic values for $\sqrt n D_n$, $K_\alpha$ (as a single table row) somewhere between $n=20$ and $n=40$, from which the critical values of $D_n$ for any $n$ can readily be obtained.


$\text{Responses to followup questions:}$

1)

How do we obtain the distribution of $D_n$ when $n$ is fixed?

There are a variety of methods for obtaining the distribution of the test statistic for small $n$; for example, recursive methods build the distribution at some given sample size in terms of the distribution for smaller sample sizes.

There's discussion of various methods given here, for example.

2)

If I get the value of $D_\text{max}$ and the sample size is $n$, I have to calculate $Pr(K<=x)$, right?

Your test statistic is your observed sample value of the $D_n$ random variable, which will be some value, $d_n$ (what you're calling $D_\text{max}$, but note the usual convention of upper case for random variables and lower case for observed values). You compare it with the null distribution of $D_n$. Since the rejection rule would be "reject if the distance is 'too big'.", if it is to have level $\alpha$, that means rejecting when $d_n$ is bigger than the $1-\alpha$ quantile of the null distribution.

That is, you either take the p-value approach and compute $P(D_n> d_n)=1-P(D_n\leq d_n)$ and reject when that's $\leq\alpha$ or you take the critical value approach and compute a critical value, $d_\alpha$, which cuts off an upper tail area of $\alpha$ on the null distribution of $D_n$, and reject when $d_n \geq d_\alpha$.

By formula 14.3.9 of Numerical Recipes, we should calculate a value got from the expression in the brackets - should that be the x?

14.3.9 looks like it has a typo (one of many in NR). It is trying to give an approximate formula for the p-value of "observed" (that is, my "$d_n$", your $D_\text{max}$), by adjusting the observed value so you can use the asymptotic distribution for even very small $n$ (in my diagram, that corresponds to changing the $y$-value of the observed test statistic via a function of $n$, equivalent to pushing the circles 'up' to lie very close the dotted lines) but then it (apparently by mistake) puts the random variable (rather than the observed value, as it should) into the RHS of the formula. The actual p-value must be a function of the observed statistic.

3)

We make tests and get a distribution, right?

I don't know what you mean to say there.

Could you please explain your figure in a "test" way?

My figure plots the 5% and 1% critical values of the null distribution of $D_n$ for sample sizes 1 to 40 (the circles) and also the value from the asymptotic approximation $K_\alpha/\sqrt n$ (the lines).

It looks to me like you have some basic issues with understanding hypothesis tests that's getting in the way of understanding what is happening here. I suggest you work on understanding the mechanics of hypothesis tests first.

That means there is no error in 14.4.9 of NR .

(Presumably you mean 14.3.9, since that's what I was discussing.)

Yes there is an error. I think you may have misunderstood where the problem is.

The problem isn't with "$(\sqrt{n}+0.12+0.11/\sqrt{n})$". It's with the meaning of the term they multiply it by. They appear to have used the wrong variable from the LHS in the RHS formula, putting the random variable where its observed value should be.

[When the thing you're reading is confused about that, it's not surprising you have a similar confusion.]

$\endgroup$
5
  • $\begingroup$ Thank you for your kind help.I add a figure got from the book Numerical Recipes.There are questions remained.Please take a look at my updated post and forgive my retard questions. $\endgroup$ – questionhang Oct 20 '13 at 15:32
  • $\begingroup$ er..When you have time,could you please..?I know they are retarded questions. I tried to understand but,sigh. $\endgroup$ – questionhang Oct 21 '13 at 15:47
  • $\begingroup$ I add my reponse in the post. Please take a look when you have time. $\endgroup$ – questionhang Oct 22 '13 at 13:00
  • $\begingroup$ I add my response at the end. That is the first step to help me thinking. $\endgroup$ – questionhang Oct 23 '13 at 15:14
  • $\begingroup$ This answer must be marked as accepted for the great kindness of its author! $\endgroup$ – LE Anh Dung Mar 13 '20 at 20:07

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.