Suppose that we want to compute $$\int_{\mathbb{R}} f(x) dx$$ using monte carlo integration so that we can normalize $f(x)$ and make it a pdf. The examples you typically see involve integrals over a finite domain such as $[a,b]$. In such cases we can sample from the uniform distribution on $[a,b]$. In the case we integrate over $\mathbb{R}$ we need to sample from a pdf whose support is $\mathbb{R}$, say the standard normal pdf which we represent as $p(x)$. We can then compute $$\frac{1}{n} \sum_{i=1}^{n} \frac{f(x_i)}{p(x_i)}$$ as our unbiased estimate of $$\int_{\mathbb{R}}f(x)dx=\int_{\mathbb{R}}\frac{f(x)}{p(x)}p(x)dx=\mathbb{E}\left[\frac{f(x)}{p(x)}\right]$$ so long as $$(1)\;\;\;\;\int_{\mathbb{R}}\frac{f(x)^2}{p(x)}dx < \infty$$ We can also rely on estimates of the variance of the estimator as well as employ the central limit theorem to construct confidence intervals so long as $$(2)\;\;\;\;\int_{\mathbb{R}}\frac{f(x)^4}{p(x)^3}dx < \infty$$ My problem is that I struggle to construct and example where (1) and (2) hold. Can someone give examples of monte carlo integration techniques used to normalize a function over $\mathbb{R}$ which satisfy the necessary assumptions so that the estimate is reliable?

  • $\begingroup$ @Tim: I am afraid I do not see your point. If one seeks an approximation to $\int_a^b f(x(\text{d}x$, it can be interpreted as a uniform integral, no matter what the original problem is. Importance sampling shows us that the reference probability measure is arbitrary. $\endgroup$ – Xi'an Aug 26 '16 at 1:04
  • $\begingroup$ @Wintermute: It is possible to use a first importance sampler to estimate the integral and a second one to estimate the variance term. The condition (2) is thus not necessary for estimating $\int f(x)\text{d}x$. Note also that the distinction between a support of $[1,b]$ and $\mathbb{R}$ is not connected with the existence of a second or a fourth moment. Take for instance $f(x)=1/sqrt{x}$ and$ [a,b]=[0,1]$. $\endgroup$ – Xi'an Aug 26 '16 at 1:08
  • $\begingroup$ @Xi'an ok, I may have misunderstood something. $\endgroup$ – Tim Aug 26 '16 at 7:32
  • $\begingroup$ @Xi'an Condition (2) is necessary to get a reliable estimate of the error of your estimator. I would think that using a different sampler to estimate the variance will not give you a reliable notion of the error of your estimator. Condition (2) just ensures that your estimator has enough moments to get a reliable estimate of the error. $\endgroup$ – Wintermute Aug 26 '16 at 15:28
  • $\begingroup$ The variance is another integral that can be estimated separately based on the same sample or another sample. $\endgroup$ – Xi'an Aug 27 '16 at 0:04

Extreme example: $f$ a standard normal distribution, and $p$ a Cauchy distribution (just remove a constant from $f$ to make it more similar to your context). Then,

$$\frac{f(x)^2}{p(x)} = \dfrac{\pi(1+x^2)}{(2\pi)}\exp(-x^2),$$


$$\frac{f(x)^4}{p(x)^3} = \dfrac{\pi^3(1+x^2)^3}{(2\pi)^2}\exp(-2x^2),$$

which are clearly integrable since they are related to the moments of normal distributions, which always exist.

  • 1
    $\begingroup$ Your formulas do not seem to agree with your descriptions. For instance, if $p$ is a Cauchy density, then $1/p(x)$ is proportional to $1+x^2$, not $1/(1+x^2)$. $\endgroup$ – whuber Aug 25 '16 at 16:36
  • 1
    $\begingroup$ @whuber well spotted, I have corrected the expressions and the argument. Thanks. $\endgroup$ – Eso Aug 25 '16 at 16:40
  • $\begingroup$ So is the kurtosis playing a role here? Should the pdf sampled from have a higher kurtosis than the one we are trying to normalize? $\endgroup$ – Wintermute Aug 25 '16 at 16:42
  • 2
    $\begingroup$ @Wintermute It has to do with the tails. The variance of this estimator exists when the tails of the denominator are heavier. That's the rule of thumb when constructing importance sampling estimators. $\endgroup$ – Eso Aug 25 '16 at 16:44

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.