1
$\begingroup$

Given a set of data with their frequency in a data set, like: $$x_1, f_{x_1}, x_2, f_{x_2}, x_3, f_{x_3}, \cdots, x_n, f_{x_n}$$, where $f_i$ is the frequency of value $x_i$ in a data set $\{x_i\}$.The definition of frequency here is: $$f_i =\frac{n_i}{N} =\frac{n_i}{n_1+n_2+n_3+\cdots}$$ If we only know a limited number of data values and their frequency, how can we estimate the density distribution of this dataset?

An example question:

If our observation is $x_1, f_{x_1}, x_5, f_{x_5}, x_8, f_{x_8}$. How can we estimate a density function $\hat{f}$, such that $\hat{f}(x_i)$ is as close to the ground truth $f_i$ as possible.

Further, if we have the ability to decide which points to be observed, how can we decide which points to observe to get the best density estimation? Example: we can observe $5$ data value and their frequency, to get the best density estimation, which $5$ points should we observe?

I search the internet and didn't find much useful resources. I do not need a precise solution (I know my question is not well defined mathematically), can you just give me some insight about which field I should look into to solve this problem, or is there any similar problems that have been discussed before?

Improve this question
$\endgroup$

1 Answer 1

Reset to default
0
$\begingroup$

You are looking for "nonparametric density estimation" and a very good resource is Larry Wasserman's "All of nonparametric statistics". The task is to estimate a probability density function from a set of (iid) samples.

You can also have a look at kernel density estimation. If you are familiar with histograms, they are just one particular choice of kernel. The data you provide is basically a histogram with bin size 1.

Improve this answer
$\endgroup$
2
  • $\begingroup$ Thank you, kernel density estimation solves the first part of my question, but when we can choose which points to observe, how do we make the decision? Say we know all the data frequency in a data set $f_1,f_2,...f_n$, but we can only observe $k$ points in the dataset to do the density estimation, what should be our strategy? Should we choose the data points with the highest frequency? $\endgroup$
    – llxxee
    Commented Dec 18, 2017 at 16:38
  • $\begingroup$ Intuitively, I'd say that if your target distribution doesn't have fat tails, that is a good ansatz, but I don't have any theory to back that up. I guess you could try different strategies and cross-validate them across as much data as you can get your hands on. $\endgroup$
    – Miguel
    Commented Dec 18, 2017 at 17:24

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.