1
$\begingroup$

I'm trying to write a piece of code in R that identifies a set of sample data as belonging to a specific distribution and pull the specific distribution parameters, by performing the K-S test and comparing the resulting p-values.

However, I've run into a bit of a logical problem. I can successfully generate gamma, weibull, logistic, normal and poisson distributions and correctly identify them, but as soon as I try to identify an exponential distribution, the resulting p-values are always lower than the p-values from trying to fit a weibull or gamma distribution. So, to put it another way, I generate a random set of values from an exponential distribtion using

rexp(1000)

but when I pass it through my code, the resulting p-values are, for instance:

[1,] "distribution" "ks pvalue"     
[2,] gamma          "0.850558314129566"
[3,] weibull        "0.833929454438442"
[4,] logistic       "0"                
[5,] normal         "0"                
[6,] exponential    "0.704115316673917"
[7,] poisson        "0"        

The section of code that performs the test on the exponential distribution, for instance, is (credited with GREAT thanks to @TinaW from StackOverflow here):

gf_shape    = "exponential"
fd_e        = fitdistr(data, "exponential")
est_rate    = fd_e$estimate[[1]]
ks          = ks.test(data, "pexp", rate=est_rate)
results[i,] = c(gf_shape, est_rate, "NA", ks$statistic, ks$p.value)

Do you guys have any idea what I'm doing wrong? I've tried increasing and decreasing the sample size, toying with the parameters, but the gamma and weibull p-values are always larger than that of the exponential fit. Any ideas?

$\endgroup$
  • $\begingroup$ Can you link to the Stack Overflow thread w/ the complete code? $\endgroup$ – gung - Reinstate Monica Apr 27 '15 at 20:14
  • 2
    $\begingroup$ Note that the exponential is a special case of the Weibull, which is more flexible & would be able to better fit any exponential data despite its exponentiality (is that a word?). Are you taking that fact into account? If not, that may be (part of) the answer. $\endgroup$ – gung - Reinstate Monica Apr 27 '15 at 20:16
  • $\begingroup$ Hi @gung. Here is the thread where most of the code comes from. stackoverflow.com/questions/29855963/… $\endgroup$ – Martin Apr 27 '15 at 20:18
  • 3
    $\begingroup$ If you aren't taking that into account, then no. A more flexible model (distribution, here) will always fit better than a less flexible model irrespective of the fact that the latter is correct. $\endgroup$ – gung - Reinstate Monica Apr 27 '15 at 20:24
  • 4
    $\begingroup$ What's most fundamentally wrong is trying to use p-values to compare distributions. That's not how p-values work. You might be able to use a conceptually similar but theoretically valid approach by comparing likelihood ratios rather than p-values. $\endgroup$ – whuber Apr 27 '15 at 20:43
4
$\begingroup$

Note that the exponential distribution is a special case of the Weibull distribution the Weibull's shape parameter equals $1$. Thus, the Weibull is more flexible and would be able to better fit any exponential data despite the data's exponentiality (is that a word?). If you are not taking that into account, that may be (part of) the answer. A more flexible model (distribution, here) will always fit better than a less flexible model irrespective of the fact that the latter is correct.

$\endgroup$
  • 4
    $\begingroup$ Similarly, the exponential is a special case of the gamma.... $\endgroup$ – Nick Cox Apr 27 '15 at 20:51

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.