0
$\begingroup$

Suppose I have the following data:

 [1]  3631  1681   188  1065   733   643  2001   714   180  5147  2541  1048   643  1356   270  4396   358  4025  2004  1879  2342  4138   616  3161  4904  4320   215
 [28]    79  5431  6551    97   889  6009   992  1487   336   840   612   769   680  5840   603  2581  4087  2241   129  2366  3856   980  1315  1050  7002    36   511
 [55]   529   534  3037  1123  3889  4611  2577  3953   517   774   923   295  3152   524   714  5135  2529  1561  6105  4305  3633  1164  1663   791    11   225   940
 [82]   172  4936   348  3410  3205  6827  3846  1809  6580    61   892  4525   523   595  3594  2245   999   343   856   106  1513   224   324  6725   323  2221  6455
[109]  3955  3580   532   775  3022  3049  1086   613  2866  4799   158   869  2510   149  1809  2772  5474  1096  5668   381  2428   428   308   932  1868   490 10163
[136]    12   671  1676   536  1940   686  1590  5749  1257  1389  3209   562  2504   129  1617  4058   521  2541    57 10747  1795   566  3290   372  5624  1229   252
[163]   257  1971   707  8036  2934   466   378   675  1551  2320   248  2871  4747  2987  6555   369   378   443   397  7653  1471   174   764   585   291   703   440
[190]  1808    83  3346  2384  2693    52   678  1320  7359  5367  1527  5789   300   101  1749  4265  4095  2134   326   326  1266   424   379  2275   206  1740  1593
[217]  1448  7488  1862  4304   436  2609   929  1583   325  5153   371   572   884   422 10905  2406  1873  4371    75   150  1538  1617  7756   630   691   200  1000
[244]   964   693   444    59  2059  1130  1276  1847   367  1533   875  2434   495  2087  1777  3709   335   156   280  2528  2401  1978   511  4999  2568  1398  2637
[271]  1668  2077  2993    69  3699  1667  2584  1915   679  4078  3014   555  2690    69   930  1026   324   991   973   566   459  2338   509   785   467   355  3186
[298]  5202  1122  5077  4945  1973  3029   377  4871  5481   284   801   444  1196   661    25   318  1137  1317  2841   143  1139  1662  1012    88  1764  1203  3618
[325]   713  8657  1274  2255    96   784  1687    62  1211   952   125  3260   879   430  3096   499   699  2395  1704  5818  2754  2012  6724  2891   959  1730   962
[352]   182   210  6051   902  3759  2211   206  1408  1472   883   773  2479   529  2932  1421  2111  1829   847  2761  1060  1805  1348  2049  2507   809   502  5877
[379]  1621  2254  1329  2752  1657   167   526   616   198  1648  1329  1643   360  1028   923  2819  1856  4562  2547  2517  6200  1704    10  1838   333  1643  1561
[406]   985  2763  4939  3116   855  1405   891  4503   210  4406  4836    33    97  4957  2202  1709  6048   123  1193 13006   271   781  2005  2970   352  1600  1862
[433]  2945  5234   502  2943  1666   167  4473   468   795  3175  1114     6   579    42   995  2705  1929   594   809   572   871   470   802   773  7028   430   139
[460]   215  1328   117  1324  1451  1455   157  1347  1049  2391   301  1587   602  1197   199  4400  1883  1971  1849  8050  7730  2527  4066  4443    54   838    16
[487]   584   261   579   729   226   292  1367  1608   749  1351  3710  2219   369   628

When inspecting this data, it is evident that is data is clearly non-normal:

enter image description here

My Question:

In such cases, how do we "summarize" the data? We can see that the "mean" will not to have the same "descriptive power" had the data been Normally Distributed. In such cases, is the standard approach to try and fit different probability distributions to this data (e.g. Exponential, Gamma, Poisson) with varying parameters (e.g. different values of lambda), see which distribution fits the data best - and then use the appropriate statistic for summarizing the data (e.g. if the data came from an exponential distribution, the mean = n/sum(yi) )?

Within the R programming language, are there any standard routines for fitting several probability distributions to some data and finding the optimal model parameters? Or does this need to be done by hand (e.g. manually define the likelihood equation for each probability distribution)?

Thanks!

Note:

I tried the following code (I placed the above data into a data frame) to estimate model parameters for popular probability distributions, but it didn't work:

library(fitdistrplus)
library(patchwork)
library(ggplot2)



 fg <- fitdist(d$df, "gamma")
 fln <- fitdist(d$df, "lnorm")
fg <- fitdist(d$df, "gamma")
 fw <- fitdist(d$df, "weibull")

 par(mfrow = c(2, 2))
 plot.legend <- c("Weibull", "lognormal", "gamma")

a <- denscomp(list(fw, fln, fg), legendtext = plot.legend, plotstyle = "ggplot")
b <- qqcomp(list(fw, fln, fg), legendtext = plot.legend, plotstyle = "ggplot")
c <- cdfcomp(list(fw, fln, fg), legendtext = plot.legend, plotstyle = "ggplot")
d <- ppcomp(list(fw, fln, fg), legendtext = plot.legend, plotstyle = "ggplot")

a+b+c+d

References:

$\endgroup$
4
  • 3
    $\begingroup$ Or simply en.wikipedia.org/wiki/Five-number_summary. $\endgroup$ Nov 24 at 6:05
  • $\begingroup$ @user2974951: do you want to post your comment(s) as an answer? Better to have a short answer than no answer at all. Anyone who has a better answer can post it. $\endgroup$ Nov 24 at 7:13
  • 1
    $\begingroup$ What is the purpose of such a summary? Five number summary (or, almost equivalently, a boxplot of the data) might be a good start. But what comes next? Comparison with another similar population/process? Make a CI for population mean or median? *Maybe with bootstrap.) Distribution ID for future modeling? // Because you're wondering if data might be exponential, are sample mean and SD roughly the same? $\endgroup$
    – BruceET
    Nov 24 at 7:36
  • $\begingroup$ See the fitdistrplus package. $\endgroup$
    – Dave2e
    Nov 26 at 1:54
2
$\begingroup$

If possible I would show a plot of the distribution, rather than just some summary statistics, since it is more informative. Alternatively, if some summary statistics are required, I would use a five-number summary or similar, rather than trying to find a distribution which fits the data. Fitting a distribution might look like a more elegant solution, since you can then only show the parameters of the distribution, but in all likelihood the fitted distribution will not be a great fit, and so the results will be misleading.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.