# Monte Carlo integration with density unknown

If I want to find the integral $\int f(x)dx$, I want to use the Monte Carlo method to calculate it. What I have is the data $x_1, \cdots, x_n$ follows $p(x)$. (In my application, $f$ is some function of $p$, $p$ is unknown, and I only have $n$ data points.)

If $p(x)$ is known, then I can do $$\int\frac{f(x)}{p(x)}p(x)dx\approx\frac{1}{n}\sum_{i=1}^n\frac{f(x_i)}{p(x_i)}$$

Now, I don't know $p(x)$ and only have $n$ data points. What I intend to do is to use kernel density estimation of $p(\cdot)$ based on $x_1,\cdots,x_n$, and plug back in the above equation. A technical treatment is that: for $\hat{p}(x_i)$, I leave $x_i$ out and use the other $n-1$ data points to make the kernel density estimation, so it's $\hat{p}_{-i}(x_i)$.

• My question is that is there any result in Monte Carlo theory to prove that $\frac{1}{n}\sum_{i=1}^n\frac{f(x_i)}{\hat{p}_{-i}(x_i)}$ is consistent, or other large sample property, say, minimaxity?

• Or is there any other better way to approximate $\int f(x)dx$ if I only have the data $x_i$ from unknown $p(x)$?

For example, if I want to use $\{x_i\}_{i=1}^n$ to estimate entropy $H(p)=-\int p\log(p)$, I can use $\frac{1}{n}\sum_{i=1}^n\log\hat{p}_{-i}(x_i)$. Note:

• In my application, $f(x)$ is also some function (in term) of $p(x)$ ($p$ is the density), like entropy.
• Entropy is just an example, this estimator has been proved to be minimax optimal in other setting.
• $x_i$ may be multivariate. But any result of the consistency of $\frac{1}{n}\sum_{i=1}^n\frac{f(x_i)}{\hat{p}_{-i}(x_i)}$ or the related is welcome.

Thanks!

• Interesting attempt. I have never seen anything of the kind and I would suspect the solution is rather poor, as the kernel estimate converges much more slowly than the Monte Carlo estimate. It is presumably convergent but at non-practical speeds.... – Xi'an Apr 24 '16 at 18:16
• Thanks @Xi'an ! Frankly speaking, I don't think this is a (Monte Carlo) integration problem, since (p unknown implies) f is unknown. So I actually don't even know what is my integrand. Thus, from a numerical integration research point of view, this is not a well-post integration problem, I think. However, this is actually from the estimation problem: estimating statistical functional with iid data $x_i$. If I propose the average estimator, the form is very like the Monte Carlo type integration, and that is why I am asking this question(any result from the Monte Carlo simulation point of view). – breezeintopl Apr 25 '16 at 17:19