1
$\begingroup$

I'm having trouble understanding a certain equation in a paper I'm reading.

  • Let $A, B, C$ be random variables and let $\mathcal{B}$ be the range of $B$.
  • Let $\mathbb{E}$ be the expected value an $\mathbb{P}$ a probability measure.
  • Let $\mathbb{1}_{X=x}$ the indicator function that is $1$ iff $X$ takes on value $x$ and $0$ otherwise.

The claim seems to be that, almost surely (original statement given below) $$ \mathbb{E}[A | B, C] = \sum_{b \in \mathcal{B}} \frac{\mathbb{E}[A \cdot \mathbb{1}_{B=b} | C]}{\mathbb{P}[B=b | C]} \mathbb{1}_{B=b} $$

I'm having trouble seeing why this statement is true, let alone proving it. To begin with, I am unsure what we even mean with conditioning an expectation on a random variable, i.e. writing $\mathbb{E}[A|B]$. I've seen conditioning on a concrete value of a random variable, i.e. $\mathbb{E}[A|B=b]$; or conditioning on an event, i.e. $\mathbb{E}[A|\{B=b\}]$ (which is probably the same thing.)

Further, what does "almost surely" mean here? Does this mean that the "probability" of this equivalence is 1? What would that mean?

I've tried applying definitions of conditional probability, expectation and the fact that $\mathbb{E}[\mathbb{1}_{B=b}] = \mathbb{P}[B=b]$ but I'm not really getting anywhere. Would appreciate if anyone could help me unwrap this.


Full statement from Scornet2015 - Consistency of Random Forests: Let $Z_{i,j} = (Z_{i}, Z_{j})$ be another random variable.

... Thus, almost surely, $$ \begin{aligned} \mathbb{E}\left[Y_i\right. & \left.-m\left(\mathbf{X}_i\right) \mid Z_{i, j}, \mathbf{X}_i, \mathbf{X}_j, Y_j\right] \\ & =\sum_{\ell_1, \ell_2=1}^2 \frac{\mathbb{E}\left[\left(Y_i-m\left(\mathbf{X}_i\right)\right) \mathbb{1}_{Z_{i, j}=\left(\ell_1, \ell_2\right)} \mid \mathbf{X}_i, \mathbf{X}_j, Y_j\right]}{\mathbb{P}\left[Z_{i, j}=\left(\ell_1, \ell_2\right) \mid \mathbf{X}_i, \mathbf{X}_j, Y_j\right]} \mathbb{1}_{Z_{i, j}=\left(\ell_1, \ell_2\right)} \end{aligned} $$

$\endgroup$

2 Answers 2

1
$\begingroup$

Let $(\Omega, F, P)$ be a probability space with $F$ a sigma-algebra on $\Omega$. Then for a set $S \in F$ such that $P(S) \in (0,1)$, the conditional probability given $S$ is the following probability measure:

$$ P(\cdot|S) : F \rightarrow [0,1], \ U \rightarrow P(U|S) := \frac{P(U\cap S)}{P(S)}. $$

For a random variable $X$, we can define the expected value of $X$ given $A$ as

$$ E(X|S) := \int X(\omega)P(d\omega|S). $$ The previous expression is equivalent to $$ E(X|S) := \int X(\omega)P(d\omega \cap S)/P(S) $$

$$ \Leftrightarrow E(X|S) := \int X(\omega)\mathbf{1}_AP(d\omega)/P(S) $$

$$ \Leftrightarrow E(X|S) := \frac{E(X\mathbf{1}_S)}{P(S)}. $$

Now, unformally, for two random variables $X$ and $Y$ defined on $(\Omega, F,P)$ $E(X|Y)$ is not a number but a random variable because $Y$ is a random variable, but the last equation still holds, this means that, for a $y \in \Omega$, and if $Y$ is a discrete random variable, we have:

$$ E(X|Y=y) := \frac{E(X\mathbf{1}_{Y=y})}{P(Y=y)}, $$ and if you add a third random variable $Z$ $$ E(X|Y=y,Z) := \frac{E(X\mathbf{1}_{Y=y}|Z)}{P(Y=y|Z)}, $$

which follows from the first equation. From there it follows $$ E(X|Y,Z) := \sum_{y\in\Omega}\frac{E(X\mathbf{1}_{Y=y}|Z)}{P(Y=y|Z)}. $$ Indded, we have $\mathbf{1}_{Y} = \sum_{Y=y}\mathbf{1}_{Y=y}$.

$\endgroup$
3
  • 1
    $\begingroup$ Thank you. Do you mean "... the expected value of $X$ given $S$" and $\mathbb{1}_S$ instead of $A$? $\endgroup$
    – ngmir
    Mar 30 at 14:54
  • 1
    $\begingroup$ I am afraid the "unformally" equation is too "unformal". The notation $P(Y)$ does not make sense when $Y$ is a r.v. The equation given in the question looks suspicious either (as it is obviously wrong if $B$ is not discrete). $\endgroup$
    – Zhanxiong
    Mar 30 at 15:45
  • $\begingroup$ To clarify, let me quote from "All of Statistics (Wasserman2010)": "Whereas $\mathbb{E}(X)$ is a number, $\mathbb{E}(X \mid Y=y)$ is a function of $y$. Before we observe $Y$, we don't know the value of $\mathbb{E}(X \mid Y=y)$, so it is a random variable which we denote $\mathbb{E}(X \mid Y)$. In other words, $\mathbb{E}(X \mid Y)$ is the random variable whose value is $\mathbb{E}(X \mid Y=y)$ when $Y=y$." $\endgroup$
    – ngmir
    Mar 31 at 9:29
1
$\begingroup$

Once again, it is enough to apply the definition of conditional expectation: \begin{gather*} \mathbb E\left[\left(\sum_{b\in\mathcal B}\frac{\mathbb E\big[A\cdot\mathbf 1_{B=b}\mid C\big]}{\mathbb P\big[B=b\mid C\big]}\mathbf 1_{B=b}\right)\cdot\mathbf 1_{B=\beta\,\cap\, C\in\mathcal C}\right]= \mathbb E\left[\frac{\mathbb E\big[A\cdot\mathbf 1_{B=\beta}\mid C\big]}{\mathbb P\big[B=\beta\mid C\big]}\mathbf 1_{B=\beta}\cdot\mathbf 1_{C\in\mathcal C}\right]=\\ \mathbb E\left[\frac{\mathbb E\big[A\cdot\mathbf 1_{B=\beta}\mid C\big]}{\mathbb P\big[B=\beta\mid C\big]}\mathbf 1_{C\in\mathcal C}\cdot\mathbb E\big[\mathbf 1_{B=\beta}\mid C\big]\right]= \mathbb E\Big[\mathbb E\big[A\cdot\mathbf 1_{B=\beta}\mid C\big]\cdot\mathbf 1_{C\in\mathcal C}\Big]=\\ \mathbb E\big[A\cdot\mathbf 1_{B=\beta}\cdot\mathbf 1_{C\in\mathcal C}\big]= \mathbb E\big[A\cdot\mathbf 1_{B=\beta\,\cap\, C\in\mathcal C}\big] \end{gather*}

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.

Not the answer you're looking for? Browse other questions tagged or ask your own question.