0
$\begingroup$

Law of large numbers states that:

If $X_1, \dots X_n \sim p(x)$ are IID, then $ \frac{1}{n} \sum_{i=1}^{n} X_i \rightarrow \mathbb{E}_{p(x)}\{X\}= \mu$, where $X \sim p(x)$.

Below is what I'm having trouble connecting to this, as this is new notation for me from an online lecture:

If we have a function $f(X)$ of some random variable $X$ and a pdf $p(x)$, then by LLN $\frac{1}{n} \sum_{i=1}^{n} f(X_i) \rightarrow \mathbb{E}_{p(x)}\{f(X)\}$, where $X_1, \dots , X_n \sim p(x)$

Is $p(x)$ referring to the transformed PDF of $X$ under $f(\cdot)$ or a different distribution? Or is this notation too ambiguous to tell?

So in the first case, $X_1, \dots X_n \sim g(x)$, therefore $f(X_1), \dots, f(X_n)$ are random variables with pdf $p(x)$, where $p(x)$ can be computed using the formula for transformation of a random variable. Hence, we can use LLN to state that the summation will eventually converge to $\mathbb{E}\{f(X)\}$ where $X \sim p(x)$, which can also be written as $\mathbb{E}_{p(x)}\{f(X)\}$.

Improve this question
$\endgroup$
7
  • 2
    $\begingroup$ $p(x)$ is simply the density of $X$. This remains true both in the original LLN statement at the start of your question and in the generalized version with $f(X_i)$ replacing $X_i$. There is no need in the theorem to transform the pdf. $\endgroup$
    – Gordon Smyth
    Commented Jul 26, 2022 at 7:38
  • 1
    $\begingroup$ The notation is awful but it's relatively clear. (One has to make assumptions about how "$X_i,$" "$X,$" and "$x$" might be related and the specific meaning of $p$ is not entirely evident). However, in the statement you are asking about, the only random variable in sight is $X,$ whence "$\mathbb{E}_{p(x)}$" must refer to expectation relative to $X.$ $\endgroup$
    – whuber ♦
    Commented Jul 26, 2022 at 13:17
  • $\begingroup$ @whuber I see, so the expectation being considered in the case with $f(\cdot)$ is not the expectation of $f(X)$ w.r.t its own pdf but rather the expectation of $f(X)$ under $p(x)$. If I had omitted the $p(x)$ subscript and just had $\mathbb{E}\{f(X)\}$, would this imply that I would have an expectation under the transformed PDF of $X$ that I talk about in the question? $\endgroup$
    – Aditya Mehrotra
    Commented Jul 26, 2022 at 14:47
  • $\begingroup$ As @GordonSmyth pointed out, this makes little sense. $X$ is the random variable. All expectations are taken with respect to its distribution. In fact, even when $X$ has a PDF, $f(X)$ might not have one. As an example, let $f(X)$ be the sign of $X:$ now $f(X)$ is a binary variable. $\endgroup$
    – whuber ♦
    Commented Jul 26, 2022 at 15:50
  • 1
    $\begingroup$ @whuber This clears everything up, thank you! $\endgroup$
    – Aditya Mehrotra
    Commented Jul 26, 2022 at 19:55

0

Reset to default

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Browse other questions tagged or ask your own question.