Cross Validated is a question and answer site for people interested in statistics, machine learning, data analysis, data mining, and data visualization. Join them; it only takes a minute:

Sign up
Here's how it works:
  1. Anybody can ask a question
  2. Anybody can answer
  3. The best answers are voted up and rise to the top

enter image description here

I am trying to fit a model for the values plotted above. The explanatory variable represents amounts of compensation claim in an earthquake, and the response variable represents amounts of compensation awarded. Can someone tell me what probability distributions would be my possible options for these data? Thanks in advance.

share|improve this question
looks like fanning out at the higher $Claimed$ values – Underminer Mar 28 '14 at 14:56
heteroscedasticity! – Nick Stauner Mar 28 '14 at 15:33
Clearly not a linear regression for me. The scatterplot "starts" at $(0,0)$ and the data are positive. The line $y=x$ looks like a "boundary". – Stéphane Laurent Mar 28 '14 at 22:40
it would nearly always be the case that amount awarded will be less than amount claimed, so that tendency to have a 'boundary' at the $y=x$ line will impact the distribution of the response. If you look instead at the 'relative shortfall' $y^*=(y-x)/x$ vs $x$, it might be in some ways easier to model (assuming no x's are actually zero; otherwise y-x might still be better). – Glen_b Mar 29 '14 at 1:02
possible duplicate of Which distribution fits data better? – Waldir Leoncio Apr 2 '14 at 19:23
up vote 4 down vote accepted

Looks like a fairly linear relation to me. I would just start with a linear regression and inspect the residuals to see if I need to do any more, e.g. do something with the outlier claimed $\approx$ 7.5 milion, awarded $\approx$ 1 milion.

share|improve this answer
Thank you for the quick reply. I just realized that I put on the wrong question. This edited question was actually what I needed help with. Any ideas on this one? – user42698 Mar 28 '14 at 14:31

The relationship looks linear so you might consider a linear regression of received on claimed.

But I think you are going to have a problem with the distribution of residuals. There is almost a ceiling effect (very few people get more than they claim) and thus, if the residuals have 0 mean they probably won't be normally distributed.

It also looks like there will be some influential points, even on the log scale (e.g the $10,000,000 claim/award).

I would consider quantile regression models, which make no assumptions about the distribution of the residuals; also, interest may focus on the upper quantiles.

share|improve this answer
Thank you for the quick reply. I just realized that I put on the wrong question. This edited question was actually what I needed help with. Any ideas on this one? – user42698 Mar 28 '14 at 14:31
Please go back to the original question and then re-post the correct problem as a new one. The original problem got good answers and may be helpful for other people. As is, the answers are nonsensical – Peter Flom Mar 28 '14 at 14:37
is it logarithmic? It seems to me that the increments are just a constant 2 milion. – Maarten Buis Mar 28 '14 at 14:57
@EngrStudent My first source check for all things statistical is usually North Carolina State (or SAS or Duke University ;o) This quantile regression modeling page might be helpful. – Ellie Kesselman Mar 29 '14 at 4:23
I wrote a paper on quantile regression in SAS much of it will apply even if you don't use SAS. – Peter Flom Mar 29 '14 at 10:24

You have two parts to your answer: data and noise.

Someone could fit a line, or a spline to the mean of the data and if they were honest they would say something like "given this model, the mean tendency is ...".

There is variation from the mean. It looks like a 3 million dollars claim got 0. If you had a line fit to predict the amount you might get a number like 3 million. When someone has a claim for anything above 7.5 million dollars the departure from the linear fit collapses.

First things first - I don't like your coordinates. They should be log-log. This would more clearly show the regions with the majority of the data. Your y-axis should be the claim de-rating. $$ y = claim - actual $$ or the log base 10 thereof.

Second - you want to account for variation. You should be able to put a +/- 95% confidence interval around whatever someone asserts as the mean de-rating given a particular value of claim.

Third - there is a clear difference between the 7.5 million dollar claim and the 3.0 million dollar claim. Access to a variable like "how much did you pay your lawyer" or "which firm did you use" might allow clustering of the claims into higher and lower yield buckets. You would then say "if you use x as lawyer and you agree to pay them y, then there is a 95% chance that if you claim z then you will actually get z-thisvalue."

A plot in the form requested (log-log, and transformed y-coordinate) is more likely to result in a resonable distribution. Could you make it one that looks like this? (link)

Best of luck.

PS: I bet a power law governs here. (link)

PPS: And Peter is a genius. If he suggests something and you can figure out how to do it - he is very likely to have given an insightful and informative answer.

share|improve this answer

Looks and sounds to me like these variables follow a , basically. Following this idea, I simulated some data in R to emulate yours:

I used shape parameter $k=.5$ for the random gamma distributions; $x$ is $y\cdot5$ for the strong linear relationship, plus a little random noise that is gamma-distributed (mostly to represent the group of people who claim something but get nothing) and normally-distributed (to make awards just a little higher or lower than claims).

share|improve this answer
Thank you for your reply. I am a little confused though as others tell me a linear regression should fit these data. However I tried your commands in R and they do make the plot look a lot nicer. If I go with a gamma distribution for these with k=0.5, what would be my θ for the function? – user42698 Mar 28 '14 at 14:57
The default scale is $\theta=1$, which is what I used. You'd probably want to figure out your distributions' parameters empirically though. I just simulated these data to resemble the relationship in your scatterplot, but of course the scale is all wrong, to say the least. – Nick Stauner Mar 28 '14 at 15:03
So if I use some commands in the format of your example above, I input something like y=rgamma(2617,0.2); x=5*y+rgamma(2617,0.2)+0.3*rnorm(2617); plot(x,y); abline(0,0.2). The plot shows a nice linear relationship. Then when I use the model to estimate y, do I input x as x=5*y+rgamma(2617,0.2)+0.3*rnorm(2617)? – user42698 Mar 28 '14 at 15:31
Sounds like you're trying to ask a new question about a generalized linear model here in your comment...I think you want glm(y~x,family=Gamma), but it doesn't seem to work with these data. Not sure why, but it produces non-numeric values. – Nick Stauner Mar 28 '14 at 19:24

Your Answer


By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.