Suppose that we want to compute $$\int_{\mathbb{R}} f(x) dx$$ using monte carlo integration so that we can normalize $f(x)$ and make it a pdf. The examples you typically see involve integrals over a finite domain such as $[a,b]$. In such cases we can sample from the uniform distribution on $[a,b]$. In the case we integrate over $\mathbb{R}$ we need to sample from a pdf whose support is $\mathbb{R}$, say the standard normal pdf which we represent as $p(x)$. We can then compute $$\frac{1}{n} \sum_{i=1}^{n} \frac{f(x_i)}{p(x_i)}$$ as our unbiased estimate of $$\int_{\mathbb{R}}f(x)dx=\int_{\mathbb{R}}\frac{f(x)}{p(x)}p(x)dx=\mathbb{E}\left[\frac{f(x)}{p(x)}\right]$$ so long as $$(1)\;\;\;\;\int_{\mathbb{R}}\frac{f(x)^2}{p(x)}dx < \infty$$ We can also rely on estimates of the variance of the estimator as well as employ the central limit theorem to construct confidence intervals so long as $$(2)\;\;\;\;\int_{\mathbb{R}}\frac{f(x)^4}{p(x)^3}dx < \infty$$ My problem is that I struggle to construct and example where (1) and (2) hold. Can someone give examples of monte carlo integration techniques used to normalize a function over $\mathbb{R}$ which satisfy the necessary assumptions so that the estimate is reliable?
$\begingroup$
$\endgroup$
5
-
$\begingroup$ @Tim: I am afraid I do not see your point. If one seeks an approximation to $\int_a^b f(x(\text{d}x$, it can be interpreted as a uniform integral, no matter what the original problem is. Importance sampling shows us that the reference probability measure is arbitrary. $\endgroup$– Xi'anCommented Aug 26, 2016 at 1:04
-
$\begingroup$ @Wintermute: It is possible to use a first importance sampler to estimate the integral and a second one to estimate the variance term. The condition (2) is thus not necessary for estimating $\int f(x)\text{d}x$. Note also that the distinction between a support of $[1,b]$ and $\mathbb{R}$ is not connected with the existence of a second or a fourth moment. Take for instance $f(x)=1/sqrt{x}$ and$ [a,b]=[0,1]$. $\endgroup$– Xi'anCommented Aug 26, 2016 at 1:08
-
$\begingroup$ @Xi'an ok, I may have misunderstood something. $\endgroup$– TimCommented Aug 26, 2016 at 7:32
-
$\begingroup$ @Xi'an Condition (2) is necessary to get a reliable estimate of the error of your estimator. I would think that using a different sampler to estimate the variance will not give you a reliable notion of the error of your estimator. Condition (2) just ensures that your estimator has enough moments to get a reliable estimate of the error. $\endgroup$– WintermuteCommented Aug 26, 2016 at 15:28
-
$\begingroup$ The variance is another integral that can be estimated separately based on the same sample or another sample. $\endgroup$– Xi'anCommented Aug 27, 2016 at 0:04
Add a comment
|
1 Answer
$\begingroup$
$\endgroup$
4
Extreme example: $f$ a standard normal distribution, and $p$ a Cauchy distribution (just remove a constant from $f$ to make it more similar to your context). Then,
$$\frac{f(x)^2}{p(x)} = \dfrac{\pi(1+x^2)}{(2\pi)}\exp(-x^2),$$
and
$$\frac{f(x)^4}{p(x)^3} = \dfrac{\pi^3(1+x^2)^3}{(2\pi)^2}\exp(-2x^2),$$
which are clearly integrable since they are related to the moments of normal distributions, which always exist.
-
1$\begingroup$ Your formulas do not seem to agree with your descriptions. For instance, if $p$ is a Cauchy density, then $1/p(x)$ is proportional to $1+x^2$, not $1/(1+x^2)$. $\endgroup$– whuber ♦Commented Aug 25, 2016 at 16:36
-
1$\begingroup$ @whuber well spotted, I have corrected the expressions and the argument. Thanks. $\endgroup$– EsoCommented Aug 25, 2016 at 16:40
-
$\begingroup$ So is the kurtosis playing a role here? Should the pdf sampled from have a higher kurtosis than the one we are trying to normalize? $\endgroup$ Commented Aug 25, 2016 at 16:42
-
2$\begingroup$ @Wintermute It has to do with the tails. The variance of this estimator exists when the tails of the denominator are heavier. That's the rule of thumb when constructing importance sampling estimators. $\endgroup$– EsoCommented Aug 25, 2016 at 16:44