0
$\begingroup$

From checking the histogram of the distribution I had the intuition that the data could follow a Poisson or an exponential distribtuion.

enter image description here

However, Lilliefors test that the data is exponentially distributed can be rejected

lilliefors(log_filesImportedMining['files_imported'], dist='exp', pvalmethod='table')
(0.35391091160912636, 0.0009999999999998899)

The same goes for the Kolmogorov-Smirnov (K-S) test, which strangely gives a p-value of literally 0.0.

kstest(list(log_filesImportedPredictive['files_imported']), 'poisson', args=(log_filesImportedPredictive.mean()))
KstestResult(statistic=0.34115413210541, pvalue=0.0)

.

Questions:

  1. Could it be that I made a mistake with the kstest, since the p-value is 0.0?
  2. Any other ideas what kind of distribution this could be?
  3. Any other methods to test?

Thanks.

Improve this question
$\endgroup$
13
  • 1
    $\begingroup$ What does this variable represent in the real world? Is it a count? $\endgroup$ – kjetil b halvorsen♦ Aug 9 '20 at 12:49
  • 2
    $\begingroup$ Why do you care? Raw data are rarely found to arise from a simple probability distribution. Do you have reason to think that this data should ? Where does the data come from ? More often we are interested in a conditional distribution. $\endgroup$ – Robert Long Aug 9 '20 at 12:49
  • 3
    $\begingroup$ If I understand your display correctly a Poisson distribution is ruled out absolutely as your data aren't counts. An exponential is implausible. The key point is that made by @Robert Long: although introductory texts or courses sometimes like to imply otherwise, real data often do not fit named distributions well and that doesn't much matter. $\endgroup$ – Nick Cox Aug 9 '20 at 12:53
  • 3
    $\begingroup$ P-values reported as 0 usually just mean too small to print. $\endgroup$ – Nick Cox Aug 9 '20 at 12:54
  • 2
    $\begingroup$ Please add this new information as an edit to the post. And, can you share (a link to) the data? $\endgroup$ – kjetil b halvorsen♦ Aug 9 '20 at 15:36

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.