3
$\begingroup$

I have 149 locations that are lined up from east to west. I have the geographical distances between each location and the adjacent location going west. I want to test whether the locations are randomly distributed or not. Therefore, I take the furthest west location and the furthest east location, and generate 145 random locations within this space and find the distance between each consecutive location (again from east to west). I then test the actual distribution of distances against the randomly generated distribution of distances using Kolmogorov Smirnov to get a p value.

However, if I then decide to do 1000 simulations, does it make sense to just calculate the average (or median) p value of the 1000 KS tests and report this?

Improve this question
$\endgroup$
9
  • 1
    $\begingroup$ No, I can't think of a situation where it makes sense to average p-values. How are you randomly generating distances? Couldn't you use the distribution you are using to generate the random distances as the null hypothesis in the KS test? $\endgroup$
    – caburke
    Commented Dec 18, 2012 at 22:48
  • $\begingroup$ I'm generating the distances by randomly sampling a number between 0 and the distance to the furthest location. Essentially, this distribution is my null hypothesis. It's just that I would have thought repeating it would make the comparison between the actual and random distribution more robust. I'm using sample.int in R. Why wouldn't it be sensible to average p-values? $\endgroup$
    – Kaleb
    Commented Dec 18, 2012 at 22:59
  • 1
    $\begingroup$ You said "Essentially this distribution is my null hypothesis." Exactly. In each simulation you are drawing a random sample from a probability distribution. The CDF of that probability distribution should be your null hypothesis in the KS test. $\endgroup$
    – caburke
    Commented Dec 19, 2012 at 3:08
  • $\begingroup$ As for the question "why wouldn't it be sensible to average p-values?", the reason is that the average of p-values doesn't have any reasonable interpretation I can think of. $\endgroup$
    – caburke
    Commented Dec 19, 2012 at 3:17
  • 1
    $\begingroup$ I think, going back a step, we need to clarify your research question "whether the locations are randomly distributed or not". Perhaps you actually mean "distributed in accordance with a uniform random distribution or not". Is the uniform distribution actually the crucial aspect of your null hypothesis? $\endgroup$
    – Peter Ellis
    Commented Dec 19, 2012 at 10:07

2 Answers 2

Reset to default
3
$\begingroup$

See here for the documentation of R's implementation of the Kolmogorov-Smirnov test. The important thing is that you can choose a wide variety of distributions as your null hypothesis in the KS test, like the exponential distribution as you mentioned in a comment.

For your original question, it does not make sense to average p-values because an average p-value has no useful interpretation for your needs (or for anything that I am aware of). What you are trying to do, test whether the distances between locations follow some distribution, is exactly what the KS test does.

As an example, suppose you believe distances between the 145 locations in your data set follow a $U(0,1000)$ distribution. Then if your 145 distances are in a vector named $\tt{x}$ in R, run

ks.test(x, punif, min=0, max=1000)

For the hypothesis that the data follows an exponential distribution with mean 2, the code is

ks.test(x, pexp, rate=0.5)
Improve this answer
$\endgroup$
3
  • $\begingroup$ For the null hypothesis of an exponential distribution, would it be make sense to use the mean of the actual distribution of distances? Similarly, for the uniform, would I use the theoretical max and min distance s (distance between last and first locations; and 0) or the max and min observed distances? $\endgroup$
    – Kaleb
    Commented Dec 19, 2012 at 10:30
  • $\begingroup$ Also, if I want to generate the exponential distribution outside ks.test, how is that done? The quantile argument isn't very clear to me? $\endgroup$
    – Kaleb
    Commented Dec 19, 2012 at 10:39
  • $\begingroup$ if you use estimated parameters in the ks.test(), the p-value becomes invalid, as you can check via a Monte Carlo experiment. $\endgroup$
    – Xi'an
    Commented Dec 19, 2012 at 19:18
0
$\begingroup$

Actually you can combine p values using fisher's method. It's not a straight average, it is described here. http://en.wikipedia.org/wiki/Fisher%27s_method and probably many other places. This is assuming that the p values are independent though.

Improve this answer
$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.