# Fitting data sample to a distribution

I'm trying to fit a data sample to a distribution. So far I have created a histogram and fitted the data with a lognormal distribution in R and made a Q-Q plot in excel (of log(benefits paid) against theoretical normal quantiles).

Here are my histogram and Q-Q plot:

However, I don't think this distribution is fitting the data closely enough.

I was hoping for an opinion on whether there are any distributions which should fit the data better. Or is this as closer fit as I'm going to be able to get?

-
This may be a moot point, but why did you use excel to create the q-q plot, instead of using R's qqplot()? – Mark Ramotowski Apr 18 '14 at 12:55
I was trying to avoid using R because I didn't know how but now I do so I can do that now! – user135784 Apr 21 '14 at 11:17
It is so much easier! Happy days! – Mark Ramotowski Apr 21 '14 at 11:19
lol true thanks!! – user135784 Apr 21 '14 at 11:20
What exactly is the relationship between this histogram and the Q-Q plot? The distribution depicted in the latter (which is approximately symmetric and very short-tailed) very clearly is not at all like the former (which is highly asymmetric with a long right tail). Although you indicate the Q-Q plot concerns log benefits paid, the values (which range approximately between -2 and 2) are not logarithms of the data in the histogram. – whuber Jun 5 '14 at 13:19

The bins in your histogram are a bit too wide (the default has too few bins), making it hard to discern the shape clearly.

The QQ-plot suggests that a finite mixture of perhaps two or three components, possibly of something right skew, perhaps like gamma distributions.

-
Thanks Glen! Yes I see what you mean I tried to set the number of bins lower however unfortunately R automatically altered this and I'm not skilled enough to change this. I had a look on Wikipedia and found a distribution called the double-pareto lognormal which looks as though it would be appropriate but I'm not really skilled enough to use that either. Thanks for the further insight though I will keep this in mind! – user135784 Apr 18 '14 at 10:14
(1) I wanted more, not fewer bins! (2) If you're using the hist command I suspect you made an error. I use it often, and the number of bins is trivial to change: try x=rgamma(999,2.5,.25) to make some data, and then hist(x, n=40) which will give close to 40 bins, as an example. If you're using some other command to make histograms you'll need to look at the help. – Glen_b Apr 19 '14 at 0:04
Your data has a hint of several right skew modes, which is why I suggested a mixture of unimodal distributions. Few right skew distributions that you can easily find code to generate are shaped like your sample. – Glen_b Apr 19 '14 at 0:07
Thanks Glen! I'll try to adjust the number of bins up then. Sorry about that! Ah ok I'll try fitting the data like you said with multiple unimodal distributions although I'm pretty new to R so if you have any tips on how I can do this that would help. Thanks!! – user135784 Apr 19 '14 at 2:44
Depending on your needs, in simple cases it can be done reasonably well by hand. There are some packages that are useful for mixture models, some use EM some use other approaches. A number of packages for mixtures of Gaussians exist, so you could take logs and try one of those first. Indeed if I was fitting gammas I'd do the same to try to spot the bumps in the log-density to figure out where to initialize my fits. – Glen_b Apr 19 '14 at 5:29

Have you tried a gamma distribution? That is a little more flexible than a log-normal and typically has thinner tails, which appear to be where your fit is lacking.

If that doesn't work, you might want to click through this list on wikipedia. There are many choices for distributions with positive support, and of course each one has it's pluses and minuses depending on your application. It looks like you're doing something with financial data, so the Pareto distribution may also be worth exploring. That tends to have thick tails though so maybe try the gamma first.

A piece of advice though: you're not going to find a closed-form distribution that fits your data perfectly. Different distributions will miss in different ways, so you'll need to ask yourself exactly how good and where in the distribution the fit has to be. The log-normal fit in your pictures may not actually be that bad depending on your priorities. If you're doing regression analysis, then maybe it works. If you're modeling tail-risk, then maybe not. Also, do you really need a closed form distribution? If distribution fit qua fit is your goal, then maybe the empirical distribution is the way to go.

-
Out of curiosity, what is the basis for saying that gamma distributions are more flexible than lognormal? – Glen_b Apr 18 '14 at 7:46
Well really I mean that its shape can take more obviously different forms depending on that shape parameter than the log-normal, which is at least built from a self-similar location-scale family (the normal). Technically they both have two parameters so it's not that much more flexible, but in a narrow sense... There is also a distribution with an additional parameter, which I forget right now, that includes the gamma as a special case. – Mike Nute Apr 18 '14 at 8:03
Thanks Mike! Your answer is very helpful. I'll try out the Gamma and Pareto distributions and skim through wikipedia and decide which one is best based on your advice! – user135784 Apr 18 '14 at 8:35