The (Strong) Law of large numbers states that

$ \frac{1}{N}\sum_{k=1}^N h(X_k) \rightarrow \mathbb{E}\left[h(X)\right]$

a.s in $\mu$ as $N\rightarrow \infty$.

but I can't find any conditions on $h(\cdot)$ other than it needs to be $\mu$-integrable.

I am not overly comfortable with measure and integration theory so I might miss the point here but what conditions are set on $h$? The proofs I have found on LLN that doesn't involve measure theory, doesn't have any function $h$ and I don't really know what is included in being $\mu$-integrable.

I need this in my thesis for explaining why my Sequential Monte Carlo estimators are consistent. Any reference to a book that has this proof might be really helpful but understanding is my biggest priority here.


1 Answer 1


For iid data, the only condition is that $\mathbb E[h(X)]$ exists. If $\mathbb E[h(X)] = +\infty$ or $-\infty$ the SLLN still holds; the only issue comes when $\mathbb E[h(X)]$ is undefined, i.e. the positive and negative parts of $h(X)$ integrate to $\infty$. The integrability condition (which is overly strong for iid data) is essentially saying that $\int |h(X)| \ d\mu < \infty$ where $(\Omega, \mathcal F, \mu)$ is some underlying probability space; usually one would check that $\int |h(x)| f(x) < \infty$ if $X$ is continuous or that $\sum |h(x)| f(x) < \infty$ if $X$ is discrete where $f$ is the pdf/pmf of $X$.

This is all assuming iid data; for independent data, or for Markov chains, things are more complicated. With Markov chains, some measure theory is probably inescapable.

  • $\begingroup$ Thanks, I am working with hidden Markov models atm so I probably need to read up a bit more on measure theory then. This really answered my questions though so thank you! $\endgroup$
    – while
    Dec 12, 2012 at 7:24

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.

Not the answer you're looking for? Browse other questions tagged or ask your own question.