1
$\begingroup$

I'm trying to write a piece of code in R that identifies a set of sample data as belonging to a specific distribution and pull the specific distribution parameters, by performing the K-S test and comparing the resulting p-values.

However, I've run into a bit of a logical problem. I can successfully generate gamma, weibull, logistic, normal and poisson distributions and correctly identify them, but as soon as I try to identify an exponential distribution, the resulting p-values are always lower than the p-values from trying to fit a weibull or gamma distribution. So, to put it another way, I generate a random set of values from an exponential distribtion using

rexp(1000)

but when I pass it through my code, the resulting p-values are, for instance:

[1,] "distribution" "ks pvalue"     
[2,] gamma          "0.850558314129566"
[3,] weibull        "0.833929454438442"
[4,] logistic       "0"                
[5,] normal         "0"                
[6,] exponential    "0.704115316673917"
[7,] poisson        "0"

The section of code that performs the test on the exponential distribution, for instance, is (credited with GREAT thanks to @TinaW from StackOverflow here):

gf_shape    = "exponential"
fd_e        = fitdistr(data, "exponential")
est_rate    = fd_e$estimate[[1]]
ks          = ks.test(data, "pexp", rate=est_rate)
results[i,] = c(gf_shape, est_rate, "NA", ks$statistic, ks$p.value)

Do you guys have any idea what I'm doing wrong? I've tried increasing and decreasing the sample size, toying with the parameters, but the gamma and weibull p-values are always larger than that of the exponential fit. Any ideas?

Improve this question
$\endgroup$
10
  • $\begingroup$ Can you link to the Stack Overflow thread w/ the complete code? $\endgroup$
    – gung - Reinstate Monica
    Apr 27, 2015 at 20:14
  • 2
    $\begingroup$ Note that the exponential is a special case of the Weibull, which is more flexible & would be able to better fit any exponential data despite its exponentiality (is that a word?). Are you taking that fact into account? If not, that may be (part of) the answer. $\endgroup$
    – gung - Reinstate Monica
    Apr 27, 2015 at 20:16
  • $\begingroup$ Hi @gung. Here is the thread where most of the code comes from. stackoverflow.com/questions/29855963/… $\endgroup$
    – Martin
    Apr 27, 2015 at 20:18
  • 3
    $\begingroup$ If you aren't taking that into account, then no. A more flexible model (distribution, here) will always fit better than a less flexible model irrespective of the fact that the latter is correct. $\endgroup$
    – gung - Reinstate Monica
    Apr 27, 2015 at 20:24
  • 4
    $\begingroup$ What's most fundamentally wrong is trying to use p-values to compare distributions. That's not how p-values work. You might be able to use a conceptually similar but theoretically valid approach by comparing likelihood ratios rather than p-values. $\endgroup$
    – whuber ♦
    Apr 27, 2015 at 20:43

1 Answer 1

Reset to default
4
$\begingroup$

Note that the exponential distribution is a special case of the Weibull distribution the Weibull's shape parameter equals $1$. Thus, the Weibull is more flexible and would be able to better fit any exponential data despite the data's exponentiality (is that a word?). If you are not taking that into account, that may be (part of) the answer. A more flexible model (distribution, here) will always fit better than a less flexible model irrespective of the fact that the latter is correct.

Improve this answer
$\endgroup$
1
  • 4
    $\begingroup$ Similarly, the exponential is a special case of the gamma.... $\endgroup$
    – Nick Cox
    Apr 27, 2015 at 20:51

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.

Not the answer you're looking for? Browse other questions tagged or ask your own question.