Testing a vector for gamma distribution

Question

I asked this question on stackoverflow before but I was told that I better ask this question here! So... I have a problem with a certain vector. I'm tying to find out IF it's gamma-distributed and (if so) what the parameters (shape, rate) are. MY vector has 400 entries but lets take e.g.

x <- c(45.94,31.04,17.49,9.81,6.34,4.18,2.93,2.01,1.61,1.27,1.04,0.809)

I read something about fitdistr(). But I didn't quite understand what it actually does! I tried thie following with my real (long) vector:

fitdistr(x, "gamma")
shape         rate    
0.167498708   0.519997226 
(0.008849548) (0.068359517)
Warning messages:
1: In densfun(x, parm[1], parm[2], ...) : NaNs wurden erzeugt
2: In densfun(x, parm[1], parm[2], ...) : NaNs wurden erzeugt
3: In densfun(x, parm[1], parm[2], ...) : NaNs wurden erzeugt
4: In densfun(x, parm[1], parm[2], ...) : NaNs wurden erzeugt
5: In densfun(x, parm[1], parm[2], ...) : NaNs wurden erzeugt
6: In densfun(x, parm[1], parm[2], ...) : NaNs wurden erzeugt
7: In densfun(x, parm[1], parm[2], ...) : NaNs wurden erzeugt

What does the output mean? Are these my fitting parameters? I tested them, but the KS-Test gave me a negative result:

ks.test(anzahl, "pgamma", 0.167498708, 0.519997226)

One-sample Kolmogorov-Smirnov test
data:  anzahl
D = 0.3388, p-value < 2.2e-16
alternative hypothesis: two-sided

So could you maybe tell me how I can find out if my vector is gamma-distributed and what the parameters are? By the way => I think my scaling is wrong. What can I do about it?

Here is the real vector:

anzahl
[1] 4.594979e+01 3.104833e+01 1.749554e+01 9.812764e+00 6.347255e+00 4.181605e+00 2.939465e+00 2.190990e+00
[9] 1.615673e+00 1.272173e+00 1.041295e+00 8.493665e-01 6.724542e-01 5.795401e-01 4.922572e-01 4.021586e-01
[17] 3.913656e-01 3.097137e-01 3.359925e-01 2.543407e-01 2.379165e-01 2.257156e-01 1.914594e-01 1.839512e-01
[25] 1.520413e-01 1.464101e-01 1.398405e-01 1.426560e-01 1.112154e-01 1.145002e-01 1.032379e-01 9.479118e-02
[33] 1.018301e-01 8.259033e-02 8.634444e-02 7.836696e-02 7.742844e-02 6.100422e-02 5.584233e-02 6.522759e-02
[41] 5.396527e-02 4.786485e-02 5.114969e-02 5.068043e-02 4.176443e-02 4.551854e-02 5.161896e-02 5.114969e-02
[49] 3.988738e-02 4.551854e-02 3.801032e-02 3.988738e-02 3.331769e-02 3.190990e-02 3.941811e-02 2.580948e-02
[57] 3.190990e-02 3.003285e-02 2.815580e-02 2.815580e-02 3.097137e-02 2.393243e-02 2.393243e-02 1.923979e-02
[65] 2.346316e-02 2.674801e-02 1.970906e-02 1.360863e-02 1.736274e-02 2.299390e-02 1.642421e-02 2.252464e-02
[73] 1.689348e-02 1.783200e-02 1.736274e-02 1.689348e-02 1.220084e-02 1.595495e-02 1.783200e-02 1.830127e-02
[81] 1.454716e-02 1.407790e-02 1.548569e-02 1.548569e-02 1.501642e-02 1.220084e-02 1.407790e-02 1.548569e-02
[89] 1.032379e-02 1.220084e-02 1.220084e-02 1.173158e-02 1.126232e-02 8.916002e-03 1.032379e-02 8.916002e-03
[97] 8.916002e-03 8.446739e-03 1.032379e-02 6.100422e-03 5.631159e-03 8.446739e-03 8.916002e-03 6.569686e-03
[105] 1.032379e-02 1.079305e-02 7.508212e-03 8.916002e-03 5.161896e-03 6.100422e-03 7.977475e-03 8.916002e-03
[113] 7.508212e-03 5.161896e-03 6.569686e-03 6.569686e-03 8.916002e-03 8.916002e-03 4.692633e-03 5.631159e-03
[121] 5.631159e-03 6.100422e-03 4.692633e-03 3.284843e-03 3.284843e-03 4.223369e-03 3.284843e-03 5.631159e-03
[129] 5.631159e-03 3.284843e-03 3.284843e-03 5.161896e-03 5.631159e-03 6.100422e-03 6.100422e-03 5.161896e-03
[137] 7.977475e-03 4.223369e-03 7.038949e-03 2.346316e-03 4.692633e-03 5.161896e-03 3.284843e-03 5.161896e-03
[145] 6.100422e-03 5.161896e-03 4.692633e-03 3.284843e-03 4.692633e-03 3.754106e-03 3.754106e-03 4.223369e-03
[153] 5.161896e-03 5.161896e-03 3.754106e-03 2.815580e-03 1.877053e-03 1.407790e-03 1.877053e-03 2.815580e-03
[161] 4.692633e-03 7.977475e-03 4.692633e-03 3.754106e-03 4.223369e-03 4.692633e-03 2.346316e-03 4.223369e-03
[169] 3.284843e-03 3.284843e-03 2.346316e-03 4.692633e-03 2.346316e-03 9.385265e-04 2.346316e-03 3.284843e-03
[177] 3.284843e-03 9.385265e-04 2.815580e-03 2.346316e-03 4.223369e-03 2.346316e-03 3.284843e-03 2.346316e-03
[185] 1.407790e-03 1.877053e-03 2.346316e-03 2.346316e-03 9.385265e-04 2.346316e-03 1.877053e-03 9.385265e-04
[193] 2.346316e-03 1.407790e-03 2.346316e-03 2.346316e-03 1.407790e-03 3.754106e-03 1.407790e-03 2.815580e-03
[201] 9.385265e-04 2.346316e-03 2.346316e-03 2.346316e-03 1.877053e-03 1.877053e-03 9.385265e-04 1.877053e-03
[209] 1.407790e-03 3.754106e-03 9.385265e-04 1.407790e-03 2.815580e-03 1.877053e-03 1.877053e-03 4.692633e-04
[217] 1.407790e-03 2.346316e-03 2.815580e-03 1.877053e-03 1.407790e-03 1.877053e-03 9.385265e-04 9.385265e-04
[225] 1.877053e-03 9.385265e-04 4.692633e-04 1.407790e-03 1.877053e-03 1.407790e-03 4.692633e-04 1.877053e-03
[233] 9.385265e-04 1.877053e-03 1.407790e-03 9.385265e-04 1.407790e-03 1.877053e-03 4.692633e-04 4.692633e-04
[241] 9.385265e-04 1.407790e-03 1.877053e-03 9.385265e-04 1.877053e-03 2.346316e-03 2.815580e-03 3.284843e-03
[249] 4.692633e-04 1.877053e-03 9.385265e-04 9.385265e-04 2.815580e-03 1.877053e-03 1.877053e-03 2.815580e-03
[257] 1.407790e-03 9.385265e-04 1.407790e-03 9.385265e-04 4.692633e-04 9.385265e-04 1.407790e-03 4.692633e-04
[265] 9.385265e-04 2.346316e-03 9.385265e-04 4.692633e-04 4.692633e-04 4.692633e-04 4.692633e-04 9.385265e-04
[273] 2.815580e-03 1.877053e-03 1.877053e-03 9.385265e-04 4.692633e-04 1.407790e-03 1.877053e-03 9.385265e-04
[281] 1.407790e-03 9.385265e-04 1.877053e-03 1.877053e-03 2.346316e-03 9.385265e-04 1.407790e-03 4.692633e-04
[289] 1.407790e-03 1.407790e-03 4.692633e-04 1.877053e-03 1.877053e-03 4.692633e-04 1.407790e-03 4.692633e-04
[297] 4.692633e-04 9.385265e-04 4.692633e-04 9.385265e-04 4.692633e-04 1.407790e-03 9.385265e-04 4.692633e-04
[305] 9.385265e-04 4.692633e-04 4.692633e-04 4.692633e-04 9.385265e-04 9.385265e-04 4.692633e-04 4.692633e-04
[313] 1.407790e-03 9.385265e-04 4.692633e-04 4.692633e-04 9.385265e-04 4.692633e-04 4.692633e-04 4.692633e-04
[321] 4.692633e-04 9.385265e-04 1.407790e-03 9.385265e-04 9.385265e-04 9.385265e-04 4.692633e-04 1.877053e-03
[329] 4.692633e-04 9.385265e-04 9.385265e-04 1.407790e-03 4.692633e-04 4.692633e-04 4.692633e-04 1.407790e-03
[337] 4.692633e-04 4.692633e-04 4.692633e-04 4.692633e-04 9.385265e-04 4.692633e-04 4.692633e-04 4.692633e-04
[345] 4.692633e-04 4.692633e-04 4.692633e-04 4.692633e-04 4.692633e-04 1.407790e-03 1.407790e-03 4.692633e-04
[353] 9.385265e-04 4.692633e-04 9.385265e-04 9.385265e-04 4.692633e-04 4.692633e-04 4.692633e-04 4.692633e-04
[361] 4.692633e-04 4.692633e-04 4.692633e-04 4.692633e-04 9.385265e-04 9.385265e-04 4.692633e-04 4.692633e-04
[369] 4.692633e-04 9.385265e-04 4.692633e-04 1.877053e-03 9.385265e-04 4.692633e-04 4.692633e-04 4.692633e-04
[377] 4.692633e-04 4.692633e-04 4.692633e-04 4.692633e-04 4.692633e-04 4.692633e-04 4.692633e-04 4.692633e-04
[385] 4.692633e-04 4.692633e-04 4.692633e-04 4.692633e-04 4.692633e-04 4.692633e-04 4.692633e-04 9.385265e-04
[393] 4.692633e-04 9.385265e-04 4.692633e-04 4.692633e-04 4.692633e-04 4.692633e-04 4.692633e-04 4.692633e-04
[401] 4.692633e-04 4.692633e-04 4.692633e-04 4.692633e-04 4.692633e-04 4.692633e-04 4.692633e-04 4.692633e-04
[409] 4.692633e-04 4.692633e-04 4.692633e-04 4.692633e-04

Answer 1

General comments:

1. The situations where you actually need to know if something is from a given distribution are very very rare -- which is handy, because you really can't know that a particular distribution actually is the case.

2. Your data are almost certainly not from any specific distribution on a laundry list of common distributions.

3. Several of those distributions might nevertheless be adequate approximations for some particular purpose or other.

4. Even if they are perfectly fine approximations, with a large enough sample size, a goodness of fit test would be sure to reject; goodness of fit tests can't rule a distribution in -- sometimes they can rule one out ... but that doesn't necessarily imply that you shouldn't use that as a model.

5. You're using ks.test with a fitted distribution. The Kolmogorov-Smirnov is for a fully-specified distribution. (If you account for the effect of that, though, you'll still reject.)

Specific comments, looking at your data:

1. You only have 12 observations! You're not going to be able to tell a lot about what the distribution might be, since you won't have a very good idea of the distribution of the data -- the uncertainty in an estimate of the cdf is pretty wide.

2. Nevertheless, the data are pretty clearly not consistent with a gamma (nor will they be consistent with a variety of other distributions)

3. A mixture of two gammas might well serve as a good approximation (but that leads us to 5 parameters! For 12 observations, that's pretty crazy). You might be able to restrict the shape parameters to be the same, but that would still leave 4 parameters.

4. However, now if you do any testing, you'll also have to account in some way for the statistician degrees of freedom from me looking at the data.

One question:

1. Why do you actually need to specify a distribution for these data?

Answer 2

The data seems to be from integers:

 {97919, 66164, 37283, 20911, 13526, 8911, 6264, 4669, 3443, 2711,            
 1810, 1433, 1235, 1049, 857, 834, 660, 716, 542, 507, 481, 408,
 392, 324, 312, 298, 304, 237, 244, 220, 202, 217, 176, 184, 167, 165, 
 130, 119, 139, 115, 102, 109, 108, 89, 97, 110, 109, 85, 97, 81, 85, 
 71, 68, 84, 55, 68, 64, 60, 60, 66, 51, 51, 41, 50, 57, 42, 29, 37, 
 49, 35, 48, 36, 38, 37, 36, 26, 34, 38, 39, 31, 30, 33, 33, 32, 26, 
 30, 33, 22, 26, 26, 25, 24, 19, 22, 19, 19, 18, 22, 13, 12, 18, 19, 
 14, 22, 23, 16, 19, 11, 13, 17, 19, 16, 11, 14, 14, 19, 19, 10, 12, 
 12, 13, 10, 7, 7, 9, 7, 12, 12, 7, 7, 11, 12, 13, 13, 11, 17, 9, 15, 
 5, 10, 11, 7, 11, 13, 11, 10, 7, 10, 8, 8, 9, 11, 11, 8, 6, 4, 3, 4, 
 6, 10, 17, 10, 8, 9, 10, 5, 9, 7, 7, 5, 10, 5, 2, 5, 7, 7, 2, 6, 5, 
 9, 5, 7, 5, 3, 4, 5, 5, 2, 5, 4, 2, 5, 3, 5, 5, 3, 8, 3, 6, 2, 5, 5, 
 5, 4, 4, 2, 4, 3, 8, 2, 3, 6, 4, 4, 1, 3, 5, 6, 4, 3, 4, 2, 2, 4, 2, 
 1, 3, 4, 3, 1, 4, 2, 4, 3, 2, 3, 4, 1, 1, 2, 3, 4, 2, 4, 5, 6, 7, 1, 
 4, 2, 2, 6, 4, 4, 6, 3, 2, 3, 2, 1, 2, 3, 1, 2, 5, 2, 1, 1, 1, 1, 2, 
 6, 4, 4, 2, 1, 3, 4, 2, 3, 2, 4, 4, 5, 2, 3, 1, 3, 3, 1, 4, 4, 1, 3, 
 1, 1, 2, 1, 2, 1, 3, 2, 1, 2, 1, 1, 1, 2, 2, 1, 1, 3, 2, 1, 1, 2, 1, 
 1, 1, 1, 2, 3, 2, 2, 2, 1, 4, 1, 2, 2, 3, 1, 1, 1, 3, 1, 1, 1, 1, 2, 
 1, 1, 1, 1, 1, 1, 1, 1, 3, 3, 1, 2, 1, 2, 2, 1, 1, 1, 1, 1, 1, 1, 1, 
 2, 2, 1, 1, 1, 2, 1, 4, 2, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 
 1, 1, 1, 1, 2, 1, 2, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 
 1, 1}

There are an awful plurality of baseline 1's and no 0's. Taking the square root of the logarithms we can construct a histogram, otherwise, the values are too extreme to visualize.

Mixture of normal distributions shown in red. Think about the data, and maybe explain better how it arose.