Say you had a two tailed normal distribution A for some dataset, and you then measured the probability that some set of data points B were drawn from A, using a KS test.

Could someone explain to me the process by which you would convert the p-value returned by the KS test into a Z score? (No matter how outlandish or silly the reasoning for doing so)

I realise the first thing you would do is look at a Z table but I'm interested here in knowing how you go about doing the conversion explicitly.

Here is my current understanding/interpretation of how to do this:

  • I understand that the p-value gives you the area under a normal distribution from a position N$\sigma $ away from the mean, outwards.

  • If I understand correctly the Z value is the number of $\sigma$ away from the mean you are at any position along the x axis in a 1D Normal distribution. Therefore, here, N = Z.

A pictorial representation of the above points:

enter image description here

  • So my approach for getting Z given a p-value would be to integrate the normal equation and solve for the lower limit. Then dividing through by $\sigma$ will leave you with Z. Is that correct?

  • So integrate:

    $\int_{a}^{b} \frac{1}{\sigma \sqrt{2\pi}}exp\left(-\frac{1}{2}\left( \frac{x-\mu}{\sigma} \right)^2\right) dx$

    and solve for a, given that the integral should equal the p-value.

However, is there a simpler, better way to do this given that the KS test returns a d-statistic using the cumulative distribution from your data? Perhaps because of that there is a neat trick to make a simpler conversion that I cannot yet understand?

Many thanks for any help in this matter.

  • 1
    $\begingroup$ en.wikipedia.org/wiki/…. Because your understanding of the p-value is incorrect, check out our posts on p-values. $\endgroup$ – whuber Jul 23 at 17:17
  • $\begingroup$ Thanks for the reply! Could you give me a little pointer as to what specifically is incorrect in my understanding of the p-value? I've seen it as the area under a normal distribution when calculating z-scores many times and interpret it as a likelihood (rather than a probability) of drawing samples from a region given some assumption such as the mean of the distribution being true from which the sample is drawn under. I don't understand the Wikipedia article section that you've pointed to I'm sorry- just because it's a little too die hard for me to understand. $\endgroup$ – user8188120 Jul 23 at 17:50
  • $\begingroup$ Extra: To be more specific, I've always interpreted the p-value from a KS test as the likelihood of drawing a sample "at least as extreme as" what is being seen when compared to some other distribution. Hopefully that helps :) $\endgroup$ – user8188120 Jul 23 at 17:54
  • $\begingroup$ (1) The KS test does not "measure the probability that some set of data points were drawn from [another set]." This issue is the topic of the highest-voted threads on p-values. (2) The distribution of the KS statistic is not Normal, so areas under the Normal curve are irrelevant. That's the topic of the Wikipedia page. $\endgroup$ – whuber Jul 23 at 18:00
  • $\begingroup$ A KS test measures the likelihood of drawing samples at least as extreme as the ones being looked at in reference to some proposed underlying distribution no? That's what I read in all texts on the test. But yes for (2) that makes sense as it's a comparison of cdf's but if you are absolutely certain the underlying distribution of the comparison distribution is normal, would it not make logical sense that there could be a conversion to something like a z statistic under that very strict axiom? Thanks again for replying by the way! $\endgroup$ – user8188120 Jul 23 at 18:10

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.