Generating functional-form PDF from Max Likelihood Estimation

Question

For the purpose of this question, please consider me a stats newbie.

I'm working on a (very fun!) research project which involves estimating a pdf of "personal values" -- i.e. how much a certain person values a certain good they can buy.

I wanted to find the functional form of the PDF so that I could generate data according to the distribution I saw in the empirical data I've been working with.

But I received a suggestion to follow Maximum Likelihood Estimation, which isn't something I've done in the past. I've been experimenting but my data doesn't seem to likely fit any typical distribution... it's more of a "bactrian camel" distribution.

So what I'm interested in knowing is:

1. Is my current approach correct? (see below)
2. How might I approach this with Maximum Likelihood Estimation?

Here's what I've done so far:

1. Constructed a CDF of the data I have (taking them as "valuations")
2. Literally... found the OLS estimator of the CDF, in functional form, so now I have a polynomial estimation of the CDF
3. Took the derivative of that function (which should equal the PDF)
4. Generated data according to that derivative.

Overall, this approach has performed fairly well. I don't have any metrics to compare my generated data with the actual data -- I don't know any! -- but from comparing the two datasets, they're fairly close in mean and variance.

Much appreciated!

-- An enthusiastic college kid

Edit: Adding details:

Here's what my density function looks like. I'd like to approximate a functional form of this, because I need to do two things:

1. Infer the distribution of data above 150; according to theory, there exist data that is above 150, but it's not in this distribution.

2. Generate random data (say a sample size of 5,000 -> 10,000)

The distribution I approximated through my take-derivative-of-CDF-estimation looks like this (ignore the improper scaling of y-axis -- this is scaled up)

Answer 1

MLE is used for estimating the parameters of known distribution. Suppose you have a sample $\{X_i,i=1,...,n\}$ and you know the distribution function $F$ of $X_i$, that is $F(x,\theta)=P(X_i<x)$, where $\theta$ are the parameters. Suppose $F$ has a density $f$, i.e. $f(x,\theta)=F'(x,\theta)$. Then the MLE estimate of $\theta$ is the value $\hat\theta$ which maximises the quantity:

$$L(\theta) = \sum_{i=1}^n\log f(X_i,\theta).$$

The main drawback of MLE is that you need to know $f$ in advance. Only then you can apply the MLE. In your case you want to find the $f$. One way of doing that is to pick a suitably looking functional form $g$ (which must be a distribution (for densities it must be positive and the integral over its domain must be one)) and try to estimate its parameters from your data. If the resulting fit is ok, you can say that your data distribution is approximately $g$, or alternatively you can say that you cannot reject the hypothesis that your data comes from $g$.

Hopefully there are lots and lots of distributions out there which you can try on your data. The best way to look for it is in the literature. Find an article which uses data similar to yours (in a sense that the process which generates the data in the article is similar to yours) and see which distribution is used in that article.

Answer 2

You can simulate from an unknown distribution like this pretty easily (and you won't have to specify a distribution):

1) Calculate empirical cdf function $\hat{F}$. In R, see ecdf function.

2) Generate vector of random uniforms $u$. In R, see runif function.

3) Calculate $\hat{F}^{-1}(u)$ for each $u$. See R quantile funtion.

Here's link for why this works: https://en.wikipedia.org/wiki/Inverse_transform_sampling