1
$\begingroup$

I am trying to build a model that will estimate people's willingness to pay for a certain good.

My dataset is comprised of more than 1000 observations and 30 variables. This is how my distribution looks like if I plot the frequencies of my dependent variable (the right figure only shows values up to 1000 for better visibility):

The values represent the price people are willing to pay for a certain good. As you can see, the willingness to pay varies greatly. Some people would not even pay a single Euro while others would be willing to pay up to 2000 Euro. Now I want to model, what influences the willingness of these people to pay something. I have collected a set of theory-based explanatory variables such as age and income etc.

As a first step, I have used the descdist function of fitdistrplus package in R to produce a Cullen and Frey Plot and see which distribution fits my data best and to start model building from there.

I have tried this with just my collected data and bootstrapping and the plot indicates a Beta distribution.

The code

library(fitdistrplus)
descdist(mydata[complete.cases(mydata),"var1"], discrete= FALSE) # left
descdist(mydata[complete.cases(mydata),"var1"], discrete= FALSE, boot = 1000) # right

yields the graphs

I am a bit puzzled by this since my data takes on values above 1 as it is not a probability distribution. I do have a lot of true zeros which indicate people's unwillingness to pay for the good in question, but I also have values as high as 2000 €.

Question: Does anyone have an idea what could have gone wrong and how to proceed from here?

$\endgroup$
5
  • 2
    $\begingroup$ Being above 1 doesn't mean that you don't have a probability distribution; other way round if you don't have a probability distribution why are you doing this at all? The deeper reason here is that square of skewness and kurtosis do not tie a distribution down. It's perfectly possible to have different distributions with the same squared skewness and kurtosis; an easy example is each distribution and its negation. $\endgroup$
    – Nick Cox
    Commented Apr 23, 2020 at 9:47
  • 1
    $\begingroup$ A beta distribution is for bounded variables and utterly unsuitable for any distribution that lacks sharp bounds. Spikes of zeros are always problematic and not really compatible with this kind of exercise. $\endgroup$
    – Nick Cox
    Commented Apr 23, 2020 at 9:49
  • 2
    $\begingroup$ None of the named distributions ever has a spike anywhere and although this kind of graph is intended, or rather hoped, to be widely helpful it can't deal with awkward cases. $\endgroup$
    – Nick Cox
    Commented Apr 23, 2020 at 10:08
  • $\begingroup$ Hello Nick, thank you for taking the time to read the question. I am aware, that a sample can fit different distributions. However, this not the case here as the cullen and frey graph shows. I do know that I am not dealing with a probability distribution because what the data shows is the amount people are willing to pay in Euro. Thus, I concluded that the beta distribution is an inadequate fit. My question is: How to proceed from here. Do you have any suggestions? Best, Anca $\endgroup$
    – Anca
    Commented Apr 23, 2020 at 12:15
  • 2
    $\begingroup$ Sorry, but your comment does not clarify much for me. I have already commented to the effect that the beta distribution isn't even worth considering here, let alone poor in practice. It seems that you have a mixture distribution of some kind. The literature doesn't really prepare users of statistics well for the fact that named distributions, despite their flexibility and abundance, are often poor fits to real data. I am at a loss at your denial that you have a probability distribution at all, as if true it makes the entire question nonsense. $\endgroup$
    – Nick Cox
    Commented Apr 23, 2020 at 12:24

1 Answer 1

2
$\begingroup$

You need to give more context for this question, preferably a plot of your distribution, what the values represent, and maybe (a link to) the data itself.

You should note that the Cullen and Frey graph is based on skewness and kurtosis, which are normalized values. That is, any (affine) transformation of your data $$ y \mapsto a + b\cdot y $$ will leave (squared) skewness and kurtosis invariant.

(If $b$ is negative, skewness will change sign.)

But it will transform, say, a beta distribution on the interval $[0,1]$ to a beta distribution on some other interval. So, in this context, a beta distribution should not be interpreted as a distribution on the unit interval!

Note also that while the other named distributions in the plot are represented by lines (or even just a point, as the normal) the beta is indicated by a much larger region, hinting at the fact that it can represent far more different shapes. If you have a large spike of zeros, probably that must be represented as a point mass (look into zero-inflated models). For the positive part of the distribution, to study its form, just remove the zeros from the data, and redo the plot. Show us the result of that!

$\endgroup$
3
  • $\begingroup$ thank you for your nice comment. I am only able to answer just now. I did as you suggested and removed all zeros, then redid the plot. Surprisingly it looks the same as the above. Do you have any suggestions about how to proceed from here? $\endgroup$
    – Anca
    Commented Jul 13, 2020 at 19:30
  • $\begingroup$ Maybe go back to look at the histogram, you have nonnegative values, with a mode at zero and a long tail, maybe heavy. So you need some distribution family with mode at zero, unimodal, and varying heaviness of tail. The Cullen-Frey graph does not include such families, so try some directly. I would try out Pareto families, type II, III or IV (type I cannot use zero values). See en.wikipedia.org/wiki/Pareto_distribution $\endgroup$ Commented Jul 16, 2020 at 23:19
  • $\begingroup$ Also: stats.stackexchange.com/questions/329280/… $\endgroup$ Commented Jul 18, 2020 at 0:42

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.