0
$\begingroup$

I am trying to build a model that will estimate people´s willingness to pay for a certain good.

My dataset is comprised of more than 1000 observations and 30 variables. This is how my distribution looks like if I plot the frequencies of my dependent variable:
Frequency distribution of dependent variable
For better visibility for all values up until 1000:
Frequency distribution excluding values over 1000
The values represent the price people are willing to pay for a certain good. As you can see, the willingnes to pay varies greatly. Some people would not even pay a single Euro while others would be willing to pay up to 2000 Euro. Now I want to model, what influences the willingnes of these people to pay something. I have collected a set of theory-based explanatory variables such as age and income ect.
As a first step, I have used the descdist function of fitdistrplus package in R to produce a Cullen and Frey Plot and see which distribution fits my data best and to start model building from there.
I have tried this with just my collected data and bootstrapping and the plot indicates a Beta distribution.

This code:

library(fitdistrplus)
descdist(mydata[complete.cases(mydata),"var1"], discrete= FALSE)

Yields the following graph:

Cullen and Frey graph

And this code: library(fitdistrplus) descdist(mydata[complete.cases(mydata),"var1"], discrete= FALSE, boot = 1000)`

Yields this graph:

Cullen and Frey with bootstrapping

I am a bit puzzled by this since my data takes on values above 1 as it is not a probability distribution.
I do have a lot of true zeros which indicate people's unwillingness to pay for the good in question, but I also have values as high as 2000 €.

Does anyone have an idea what could have gone wrong and how to proceed from here?

$\endgroup$
5
  • 2
    $\begingroup$ Being above 1 doesn't mean that you don't have a probability distribution; other way round if you don't have a probability distribution why are you doing this at all? The deeper reason here is that square of skewness and kurtosis do not tie a distribution down. It's perfectly possible to have different distributions with the same squared skewness and kurtosis; an easy example is each distribution and its negation. $\endgroup$ – Nick Cox Apr 23 '20 at 9:47
  • $\begingroup$ A beta distribution is for bounded variables and utterly unsuitable for any distribution that lacks sharp bounds. Spikes of zeros are always problematic and not really compatible with this kind of exercise. $\endgroup$ – Nick Cox Apr 23 '20 at 9:49
  • 1
    $\begingroup$ None of the named distributions ever has a spike anywhere and although this kind of graph is intended, or rather hoped, to be widely helpful it can't deal with awkward cases. $\endgroup$ – Nick Cox Apr 23 '20 at 10:08
  • $\begingroup$ Hello Nick, thank you for taking the time to read the question. I am aware, that a sample can fit different distributions. However, this not the case here as the cullen and frey graph shows. I do know that I am not dealing with a probability distribution because what the data shows is the amount people are willing to pay in Euro. Thus, I concluded that the beta distribution is an inadequate fit. My question is: How to proceed from here. Do you have any suggestions? Best, Anca $\endgroup$ – Anca Apr 23 '20 at 12:15
  • 2
    $\begingroup$ Sorry, but your comment does not clarify much for me. I have already commented to the effect that the beta distribution isn't even worth considering here, let alone poor in practice. It seems that you have a mixture distribution of some kind. The literature doesn't really prepare users of statistics well for the fact that named distributions, despite their flexibility and abundance, are often poor fits to real data. I am at a loss at your denial that you have a probability distribution at all, as if true it makes the entire question nonsense. $\endgroup$ – Nick Cox Apr 23 '20 at 12:24
1
$\begingroup$

You need to give more context for this question, preferably a plot of your distribution, what the values represent, and maybe (a link to) the data itself.

You should note that the Cullen and Frey graph is based on skewness and kurtosis, which are normalized values. That is, any (affine) transformation of your data $$ y \mapsto a + b\cdot y $$ will leave (squared) skewness and kurtosis invariant.

(If $b$ is negative, skewness will change sign.)

But it will transform, say, a beta distribution on the interval $[0,1]$ to a beta distribution on some other interval. So, in this context, a beta distribution should not be interpreted as a distribution on the unit interval!

Note also that while the other named distributions in the plot are represented by lines (or even just a point, as the normal) the beta is indicated by a much larger region, hinting at the fact that it can represent far more different shapes. If you have a large spike of zeros, probably that must be represented as a point mass (look into zero-inflated models). For the positive part of the distribution, to study its form, just remove the zeros from the data, and redo the plot. Show us the result of that!

$\endgroup$
3
  • $\begingroup$ thank you for your nice comment. I am only able to answer just now. I did as you suggested and removed all zeros, then redid the plot. Surprisingly it looks the same as the above. Do you have any suggestions about how to proceed from here? $\endgroup$ – Anca Jul 13 '20 at 19:30
  • $\begingroup$ Maybe go back to look at the histogram, you have nonnegative values, with a mode at zero and a long tail, maybe heavy. So you need some distribution family with mode at zero, unimodal, and varying heaviness of tail. The Cullen-Frey graph does not include such families, so try some directly. I would try out Pareto families, type II, III or IV (type I cannot use zero values). See en.wikipedia.org/wiki/Pareto_distribution $\endgroup$ – kjetil b halvorsen Jul 16 '20 at 23:19
  • $\begingroup$ Also: stats.stackexchange.com/questions/329280/… $\endgroup$ – kjetil b halvorsen Jul 18 '20 at 0:42

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.