3
$\begingroup$

I'm really struggling to find a good statistical distribution. I've tried Poisson and Gamma so far, but without success (best I've got was a p-value of 0,00005 with a Pearson Chi-Square test). So I really hope you can send me in the right direction.

The case is as follows: I'm studying the arrival rate for the application for mortgages. I'm trying to determine the arrival rate per hour (thus the number of applications that arrive in a certain hour). This data are the total number of arrivals in a specific hour, in this example between 13:00 AM and 14:00 AM. This is the data: Example data

I'm trying to determine the arrival rate per hour. These data are the total number of arrivals in a specific hour, in this example between 13:00 AM and 14:00 AM. This is the dataset: Example data

As an example I've taken a set with a relative high N.

I got the following metadata of the distribution:

Mean 15,60
St Error 0,32
Median 14,50
Mode 16,00
Standard Deviation 9,27
Variance 85,92
Kurtosis 5,49
Skewness 1,54
Range 68,00
Minimum 1,00
Maximum 69,00
Sum 12853,00
Count 824,00

I also have a histogram: Histogram

I've rejected Poisson, since the variance is not the same as the mean. Furthermore I've tried two-parameter gamma with alpha = mean^2/Variance and beta = Variance/mean, but without success.

|cite|improve this question
$\endgroup$
  • 1
    $\begingroup$ Welcome to Cross Validated! For a start would you explain what you're in fact measuring? The Poisson distribution is for a random variable that takes non-negative integer values (e.g. counts per a fixed time interval); the gamma for a continuous non-negative r.v. (e.g. time in minutes between successive events): they can't both be appropriate. Your summary data suggest the former case. Note also that the horizontal-axis labels are missing from your histogram. $\endgroup$ – Scortchi♦ Jan 11 '16 at 9:28
  • $\begingroup$ Can you provide data sample as an example? $\endgroup$ – Tim♦ Jan 11 '16 at 9:30
  • 1
    $\begingroup$ Please be more precise: You wrtie "best I've got was a p-value of 0,00005", probability of what? You write "I've rejected, ..." What did you reject? $\endgroup$ – Dirk Horsten Jan 11 '16 at 9:44
  • 1
    $\begingroup$ I'm trying to determine the arrival rate per hour. This data are the total number of arrivals in a specific hour, in this example between 13:00 AM and 14:00 AM. This is the data: Example data $\endgroup$ – Stefan Hessels Jan 11 '16 at 9:46
  • 1
    $\begingroup$ Are you trying to predict the number of arrivals for the next day,week or month ? Are you trying to detect an usual value when it arrives ? Similarly we have seen the question "what is the probability that the most recent value comes from (is generated by) the historical/observed hisorical distribution ? Are you trying to find out if the distribution for specific hours are statistically different from each other ? Are you trying to find out if the distribution has changed over time ? $\endgroup$ – IrishStat Jan 11 '16 at 13:26
4
$\begingroup$

Gamma is continuous, so I wouldn't (at least not to begin with) consider it for count data.

When variance tends to be larger than mean, one common choice is the negative binomial; it can be regarded as a mixture of Poissons (where the Poisson rates come from a gamma distribution). As a result it can often be suitable for situations where you have a populations which may be heterogeneous.

A negative binomial with the same mean and variance as your sample looks like this:

![enter image description here

This seems more or less reasonable.

[However, in your case it may be that a different mixture of Poissons could work better, perhaps a finite mixture with only two or three components could work.]

|cite|improve this answer
$\endgroup$
  • $\begingroup$ Thank you very much for your clear and extensive. I could really work with this! $\endgroup$ – Stefan Hessels Jan 11 '16 at 15:37
  • $\begingroup$ How did you determine the parameters for this function? I've used Statsmodels in Python, but got some clearly different distribution $\endgroup$ – Stefan Hessels Jan 12 '16 at 14:55
  • $\begingroup$ The answer to that question is already stated in my answer ("with the same mean and variance as your sample") -- that is, I matched the first two moments (since that information was readily available in your question). If you use MLE you will get different parameter estimates, but with such a large count and a not so far from reasonably negative-binomialish shape it shouldn't be very different. Which negative binomial is statsmodels in Python fitting? What parameter values do you have? $\endgroup$ – Glen_b♦ Jan 12 '16 at 14:58
  • $\begingroup$ I've used Negative Binomial Regressions, with an array of ones as the independent variable. The output I've got is: const 2.747162 alpha 0.283321 If I interpret it right, the p-value of the distribution is the alpha and the constant is the r-value is the constant? Or do I misinterpret this $\endgroup$ – Stefan Hessels Jan 12 '16 at 15:53
  • $\begingroup$ I can't tell what those parameters are representing (surely the documentation tells you?), but it looks unlikely that your interpretation could be right -- that would imply a mean of less than 1.09 $\endgroup$ – Glen_b♦ Jan 12 '16 at 16:12

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.