2
$\begingroup$

I came across a paper, where (section 3.2) importance sampling is used to estimate an integral. I think I understand what importance sampling is but I don't understand how they got the solution.

The integral they try to estimate:

integral (1)

The estimation:

estimation of the integral (2)

where q is a proposal distribution they sample from.

For me the traditional definition of importance sampling is:

importance sampling (3).

In the paper there is y instead of x, so let's continue with that. In section 3.1 They say the integrand from (1) is not normalized, this becomes clear from equation 3 in the paper.

I thought a lot about it, but I do not understand how the solution they provided gets computed from this.

It seems like that from (3), p(y) or f(y) is 1, and the other is the integrand from (1). That is the only way both (2) and (3) are true.

But if p(y) is 1, then it is not a PDF. If f(y) is 1, then p(y) is the integrand from (1), so again not a PDF. (I think it should be a PDF in (3), maybe that's where I'm wrong.)

I would be very grateful if someone could explain how their solution comes from the definition of importance sampling I provided in (3) specifically.

Thanks in advance!

$\endgroup$
3
  • 2
    $\begingroup$ I think the original distribution is just uniform [0,1] , f(x) = (1), for the equation to be valid $\endgroup$ Oct 2, 2023 at 13:57
  • $\begingroup$ @PrestonLui I edited my question, now it involves my thought process more. I also cover why f(x) cannot be 1 in my opinion. $\endgroup$ Oct 3, 2023 at 0:31
  • 1
    $\begingroup$ I think in general, for a uniform [-K/2,K/2] RV, to estimate the integral you need to multiply the $E(f(x))$ by K. When taking K to infinity, this equation hold $\endgroup$ Oct 8, 2023 at 0:41

1 Answer 1

5
$\begingroup$

Importance sampling means using a substitute density $q(\cdot)$ when integrating an arbitrary integrable function $H(\cdot)$ $$\int H(y)\,\text dy$$ since $$\int H(y)\,\text dy=\int \frac{H(y)}{q(y)}q(y)\,\text dy=\mathbb E_q\left[\frac{H(Y)}{q(Y)}\right]$$ Therefore, the integrand $H(\cdot)$ need not be decomposed as $p(\cdot)f(\cdot)$ with $f$ a probability density function for importance sampling to apply. (I always point out in class that the decomposition $H(\cdot)=p(\cdot)f(\cdot)$ is anything but unique.)

$\endgroup$
2
  • $\begingroup$ I understand your answer. This raises another question for me. I suppose they use Monte-Carlo because computing the integral (for example with rectangles) requires too much resource. But what's the benefit of this importance-sampling integration to "normal" Monte-Carlo integration? $\endgroup$ Oct 3, 2023 at 11:40
  • $\begingroup$ In my mind "normal" was when I multiply the sample mean by the length of the integration interval. But now I think I understand. The "normal" means the specific case when q(y) (from your answer) is uniform distribution. Importance sampling lets us pick q(y). In this way we can tell more specifically where to sample from with a bigger probability. I think it's the best to use distributions that have their mass near to where the integrand has it. Am I correct? $\endgroup$ Oct 3, 2023 at 14:36

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.