# Condition for Law of Large Numbers, Monte Carlo

In some lecture notes I am reading, there is the following;

Consider $X_{1},...,X_{n}$, each with pdf $g$ (the instrumental distribution). Our aim is to estimate $E_{f}[h(X)]$ where $h(X)$ is some measurable function and $f$ a pdf (the target distribution).

Then:

$\large\frac{1}{n}\displaystyle\sum\limits_{i=1}^n\frac{f(X_{i})h(X_{i})}{g(X_{i})} \rightarrow E_{g}\left[\frac{f(X_{i})h(X_{i})}{g(X_{i})}\right] = E_{f}[h(X)]$ as $n\rightarrow \infty$,

provided

$E_{g}\left[\left|\large\frac{f(X_{i})h(X_{i})}{g(X_{i})}\right|\right] < \infty$ and supp($g$) $\supset$ supp($f.h$).

Now, it is the condition on the last line I am not happy with. I understand that the expectation given must be finite to be able to apply the law of large numbers. However, I really do not understand how the last statement comes into play.

Is it to ensure that the expectation is finite? Is it to ensure that the series above converges?

It makes sense to me that we should want supp($g$) $\supset$ supp($f$), since $g$ is the pdf we are using to aquire information from the target distribution $f$.

Thanks for your insight and thoughts.

Our target is $$E_f[h(X)] = \int h(x) f(x) \ dx.$$ We want to use $\frac 1 N \sum_{i = 1} ^ N \frac{h(X_i)f(X_i)}{g(X_i)}$ to estimate this and appeal to the SLLN, sampling the $X_i$ from $g(x)$. If the expectation of $\frac{h(X_i)f(X_i)}{g(X_i)}$ exists then this will converge to its expected value (infinite is okay, as long as the expected value exists i.e. either the positive or negative part of $\frac{h(x)f(x)}{g(x)}$ is integrable wrt $g$).

But what is this expected value? Let $\mathcal F$ be the support of $f$ and $\mathcal G$ the support of $g$. It is $$E_g\left[\frac{h(X)f(X)}{g(X)}\right] = \int \frac{h(x)f(x)}{g(x)} g(x) \ dx = \int_{\mathcal G} \frac{h(x)f(x)}{g(x)} g(x) \ dx = \int_{\mathcal G} h(x) f(x) \ dx.$$ But if $\mathcal F \subset \mathcal G$ fails, this is not nessecarily $\int h(x) f(x) \ dx$. The key thing to notice is that $g(x)$ cancels with $\frac 1 {g(x)}$ only when $g(x) \ne 0$. In measure theory, the "right" thing to take $g(x) \cdot \frac 1 {g(x)}$ to be when $g(x) = 0$ turns out to be $0$, since if $X$ cannot land outside the support of $g$ the term $\frac 1 {g(x)}$ effectively contributes nothing to the expected value.

We typically just take things like $g(x) \cdot \frac 1 {g(x)} = 0$ as definitions when $g(x) = 0$ when we are using measure theory, e.g. it is common to define $0 \cdot \infty = 0$ when working with set $[0, \infty]$ (this is a little different than when $0 \cdot \infty$ is regarded as an indeterminate form for studying limits in calculus). Formally, $$\int_{\mathcal G^C} \frac{h(x)f(x)}{g(x)} g(x) \ dx = \int_{\mathcal G^C} \frac{h(x)f(x)}{g(x)} \cdot 0 \ dx = \int_{\mathcal G^C} 0 \ dx = 0 \ne \int_{\mathcal G^C} h(x) f(x) \ dx.$$

As you say, it makes intuitive sense that we should want supp($f$) $\subset$ supp($g$).

However, while this is a sufficient condition, it is not necessary.

All that is required is supp($fh$) $\subset$ supp($g$), which is a less demanding condition because supp($fh$) $\subset$ supp($f$).

• This does not answer my question. Why is the less demanding condition needed? – Delvesy Apr 21 '13 at 14:49
• So what you're asking is why the support of the sampling function needs to encompass the support of the target distribution? If so, the answer it because otherwise there's no way that part of the target distribution will be included in the averaging. – Patrick Apr 21 '13 at 20:16
• No, my question is why the support of g needs to encompass the support of f.h only. i.e. I could choose a g such that it emcompasses f.h but does not emcompass f. Whereas intuitively, as your comment puts it, it feels natural to choose a g that emcompasses f only. – Delvesy Apr 21 '13 at 20:36
• Because if a sample, x, is drawn from g that is outside the support of h, then f(x)h(x) is zero, even if f(x) might not be zero. So that part of the support of f doesn't necessarily have to be in the support of g, hence the less stringent requirement. – Patrick Apr 21 '13 at 22:03
• Well, did you get it? @Delvesy – Patrick Apr 24 '13 at 16:05