6
$\begingroup$

This is an example from notes enter image description here

I don't understand why we should think about the $E(x)$ and $Var(x)$ first?

$\endgroup$
  • 3
    $\begingroup$ is this a homework? please add self-study tag $\endgroup$ – Haitao Du Jul 11 '16 at 14:12
  • $\begingroup$ I like how this book thinks that $\infty-\infty=\infty$ $\endgroup$ – Alex R. Jul 11 '16 at 17:48
  • $\begingroup$ Basically, you should look up the reference for importance sampling which involves some assumptions about the functional. It should be straightforward why you wouldn't be using metropolis methods here. $\endgroup$ – AdamO Jul 11 '16 at 19:15
6
$\begingroup$

Edit: I strongly recommend getting a new book to study from because this book is garbage by the looks of it. First off, there is a gross typo: the integral to estimate should be $\int_{-\infty}^\infty xf(x)dx$ as can plainly be seen in the explanation presented. Second, the expectation is NOT infinite, it simply doesn't exist, just like in the case of a Cauchy random variable.

In its most basic form, MC integration draws random samples $x_i$ such that:

$$\frac{(b-a)}{N}[g(x_1)+\cdots+g(x_N)]\approx \int_{a}^{b} g(x)dx.$$

Here $x_i$ are drawn in $[a,b]$, for example as a uniform distribution. When the interval is infinite, we can truncate it to a finite interval and rely on the tail integrals being small. The last sentence translates to $\int_{-\infty}^ag(x)dx+\int_b^\infty g(x)dx<\infty$.

Notice that the above expression is precisely $(b-a)E[g(X)]$ where $X$ is drawn from $[a,b]$. Thus you need the expectation to be finite.

When the expectation exists, but the variance doesn't, there is no good way of putting error bars on the MC estimate of the integral.

So in your case, $g(x)=xf(x)$, and the above methodology fails as the tail integrals of $xf(x)$ are NOT finite.

$\endgroup$
  • 2
    $\begingroup$ @AlexR. but how does MC integration estimate unbounded integrals? How does one sample uniformly across all reals? Over any interval $[a, b]$ if the density is truncated, you have a finite mean and variance. Even if the case that $f$ had finite moments, like a Gaussian curve, how would we integrate that? $\endgroup$ – AdamO Jul 11 '16 at 17:09
  • $\begingroup$ What expectation are you referring to? The only expectation that actually appears explicitly in this answer is "$E[f(X)]$" (relative to a uniform variable on $[a,b]$), which is finite regardless of the values of $a$ or $b$. $\endgroup$ – whuber Jul 11 '16 at 17:49
  • $\begingroup$ @AdamO: Your comment made me realize the original question is deeply flawed. So if you have an infinite interval, yes you can truncate to a finite large interval, as for example in the case of a Gaussian. However, you still need the tail integrals to be finite. So of example if you have a Cauchy density, the tail integrals are undefined, hence the methodology won't work. $\endgroup$ – Alex R. Jul 11 '16 at 18:02
  • $\begingroup$ The OP clearly states this is from the "notes" and not a book. It could be an honest typo. Fair point about the expectation being undefined. $\endgroup$ – Greenparker Jul 11 '16 at 18:06
  • 1
    $\begingroup$ @AlexR. don't be so quick to judge! For metropolis type algorithms you're right, but that's not the only way to use MC methods for integration. There are importance based sampling methods. You're right though, in saying it's a somewhat large omission on the part of the original question. $\endgroup$ – AdamO Jul 11 '16 at 18:26
2
$\begingroup$

There might be the need for a caveat for the question, in term of what is it that you call Monte Carlo integration.

You want to use Monte Carlo to estimate $\int_{-\infty}^{\infty} f(x) dx$, a quantity we know to be 1. One way to to do this is to find a pdf $g(x)$ such that you can sample from $g$, and then

$$\int_{-\infty}^{\infty} f(x) = \int_{-\infty}^{\infty} \dfrac{f(x)}{g(x)}g(x)dx = E_g \left[\dfrac{f(x)}{g(x)} \right].$$

So you can get $N$ numbers from $g$, $x_1, \dots, x_N$, and calculate

$$\dfrac{1}{N}\sum_{i=1}^{N}\dfrac{f(x_i)}{g(x_i)} \approx \int_{-\infty}^{\infty} f(x) .$$

Thus, your choice of $g$ will lead to different estimators. @Alex R. in their answer used the uniform distribution pdf $$g(x) = \dfrac{1}{b-a} I_{a < x<b}. $$

However, in that case you are sampling from a Uniform distribution from the whole real line, and such a distribution does not exist, thus $g$ cannot be the pdf of a uniform distribution.

I believe there might a typo in the notes, since the following R code seems to suggest that a Monte Carlo estimate of $\int_{-\infty}^{\infty} f(x)$ can be obtained. Here I use $g$ to be the pdf of a $N(0,100)$, where $100$ is the variance.

set.seed(1)
N <- 1e4
f <- function(x)
{
  y <- ifelse(abs(x) > 4, 1/x^2, 1/16)
  return(y)
}
# Draws from g
prop <- rnorm(N, 0, 10)
#Evaluate g(x)
g_prop <- dnorm(prop, 0, 10)
#Evaluate f(x)
f_prop <- f(prop)

mc_est <- sum(f_prop/g_prop)/N
mc_est
[1] 0.9502994

I can probably use a better $g$ to get a better estimate, but it looks like Monte Carlo integration is possible. Maybe the question wanted to say "... estimate the value of $\int_{-\infty}^{\infty} x f(x) dx$". In which case Monte Carlo is not possible since the quantity you want to estimate does not exist.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.