# Generating functional-form PDF from Max Likelihood Estimation

For the purpose of this question, please consider me a stats newbie.

I'm working on a (very fun!) research project which involves estimating a pdf of "personal values" -- i.e. how much a certain person values a certain good they can buy.

I wanted to find the functional form of the PDF so that I could generate data according to the distribution I saw in the empirical data I've been working with.

But I received a suggestion to follow Maximum Likelihood Estimation, which isn't something I've done in the past. I've been experimenting but my data doesn't seem to likely fit any typical distribution... it's more of a "bactrian camel" distribution.

So what I'm interested in knowing is:

1. Is my current approach correct? (see below)
2. How might I approach this with Maximum Likelihood Estimation?

Here's what I've done so far:

1. Constructed a CDF of the data I have (taking them as "valuations")
2. Literally... found the OLS estimator of the CDF, in functional form, so now I have a polynomial estimation of the CDF
3. Took the derivative of that function (which should equal the PDF)
4. Generated data according to that derivative.

Overall, this approach has performed fairly well. I don't have any metrics to compare my generated data with the actual data -- I don't know any! -- but from comparing the two datasets, they're fairly close in mean and variance.

Much appreciated!

-- An enthusiastic college kid

Here's what my density function looks like. I'd like to approximate a functional form of this, because I need to do two things:

1. Infer the distribution of data above 150; according to theory, there exist data that is above 150, but it's not in this distribution.

2. Generate random data (say a sample size of 5,000 -> 10,000) The distribution I approximated through my take-derivative-of-CDF-estimation looks like this (ignore the improper scaling of y-axis -- this is scaled up) • It might be useful to give us more details, show plots, etc. to judge your approach. I'm still not quite certain what you've done exactly. – user44764 May 21 '14 at 23:30
• Hi @Matthew, thanks! I added some details that I hope are helpful. – metakai May 22 '14 at 18:01
• You might want to look into Kernel Density Estimation, which would give you a mixture model PDF for your data. At the very least this would be a way to compare to your already existing approach (which has a bad left-tail in my opinion). – user44764 May 22 '14 at 18:04
• It looks like a mixture of Normals would fit your data quite well (this is a type of KDE). – user44764 May 22 '14 at 18:13
• Awesome. What's the right way to figure out the parameters of the (I'm assuming four) normals? (Mean, sd, etc? – metakai May 22 '14 at 21:45

MLE is used for estimating the parameters of known distribution. Suppose you have a sample $\{X_i,i=1,...,n\}$ and you know the distribution function $F$ of $X_i$, that is $F(x,\theta)=P(X_i<x)$, where $\theta$ are the parameters. Suppose $F$ has a density $f$, i.e. $f(x,\theta)=F'(x,\theta)$. Then the MLE estimate of $\theta$ is the value $\hat\theta$ which maximises the quantity:

$$L(\theta) = \sum_{i=1}^n\log f(X_i,\theta).$$

The main drawback of MLE is that you need to know $f$ in advance. Only then you can apply the MLE. In your case you want to find the $f$. One way of doing that is to pick a suitably looking functional form $g$ (which must be a distribution (for densities it must be positive and the integral over its domain must be one)) and try to estimate its parameters from your data. If the resulting fit is ok, you can say that your data distribution is approximately $g$, or alternatively you can say that you cannot reject the hypothesis that your data comes from $g$.

Hopefully there are lots and lots of distributions out there which you can try on your data. The best way to look for it is in the literature. Find an article which uses data similar to yours (in a sense that the process which generates the data in the article is similar to yours) and see which distribution is used in that article.

You can simulate from an unknown distribution like this pretty easily (and you won't have to specify a distribution):

1) Calculate empirical cdf function $\hat{F}$. In R, see ecdf function.

2) Generate vector of random uniforms $u$. In R, see runif function.

3) Calculate $\hat{F}^{-1}(u)$ for each $u$. See R quantile funtion.

Here's link for why this works: https://en.wikipedia.org/wiki/Inverse_transform_sampling

• I think he's looking for a functional form for this, not a way to simulate more data. – user44764 May 22 '14 at 18:08
• Is not the reason why he wants to find a functional form so that he can simulate from it?...though maybe I'm misunderstanding. – bmciv May 22 '14 at 18:13
• Also, I don't think there's going to be much you can do to "infer the distribution" of data above 150 when there are no observations that large. – bmciv May 22 '14 at 18:14
• You're absolutely right, but he also states that he wants the explicit functional form for the PDF. Perhaps he care about generating the data more than the PDF, in which case your approach would work fine. – user44764 May 22 '14 at 18:15
• Thanks for your help. As per your questions: 1. According to the theory we're working with, it's possible to infer values above 150, if we have a maximum/limit -- so the reasoning is if we understand the distribution below 150, then past 150 should follow a similar distribution. 2. I need to translate the generator into PHP code, so that in the code I'm writing I can generate an arbitrarily large sample according to other functions I've written. So in that case, I want PHP to generate the data for me, and I figure having a functional form would be easier. – metakai May 22 '14 at 18:28