11
$\begingroup$

In a question like Intuition for Conditional Expectation of $\sigma$-algebra a concept like $E[X|\sigma(Y)]$ is used and I am puzzled about what sort of variable this actually is.

Say we have a probability space $(\Omega,\mathcal{F},P)$ with sample space, event space and probability measure: $$\begin{array}{rcl} \Omega &=& \lbrace 1,2,3\rbrace\\ \mathcal{F} &=& \lbrace \emptyset, \{1\},\{2\},\{3\},\{1,2\}, \{1,3\},\{2,3\},\{1,2,3\}\rbrace\\ p(\omega) &=& 1/3 \end{array}$$

Say, we consider the following two variables that map each element $\omega$ from $\Omega$ to a real value.

$$\begin{array}{rcl} X(\omega) &=& \omega\\ Y(\omega) &=& \textbf{1}_{\omega > 1}\end{array}$$

Then for $E[X|\sigma(Y)]$ I can imagine two different ways to compute the conditional variables

  1. The conditional value of $X$ for each value of $Y$ like $$\begin{array}{} E[X|Y = 0] &=& 1\\ E[X|Y = 1] &=& 2.5\\ \end{array}$$

    This case seems very intuitive to me. Now the expectation can be written as a function of $\omega$, $$E[X|\sigma(Y)](\omega) = \begin{cases} 1 &\quad \omega \in \{1\} \\ 2.5 &\quad \omega \in \{2,3\} \end{cases}$$ but it is not conditional on all of the events in the sigma algebra. E.g what about $E[X|Y = 1 \lor Y = 0] $?

  2. The conditional value of $X$ given each possible event in $\sigma(Y)$ like $$\begin{array}{} E[X|Y = 0] &=& 1\\ E[X|Y = 1] &=& 2.5\\ E[X|Y = 1 \lor Y = 0] &=& 2\\ E[X|\emptyset] &=& \dots\\ \end{array}$$ But in this case it makes that $E[X|\sigma(Y)]$ is not a variable in terms of a function of the samples $\omega$.

Which of these two is intended when people speak about $E[X|\sigma(Y)]$?

$\endgroup$
1

3 Answers 3

11
$\begingroup$

From a probabilist's perspective, neither (regarding the very first version of the post).

To clearly express what $E[X|Y]$ is ("$E[X|Y]$" is a common shorthand for "$E[X|\sigma(Y)]$") for your toy/illustrative example (NB, perhaps in 99% of cases, $E[X|Y]$ does not have a closed-form expression), it is natural to write down what $\sigma(Y)$ exactly is in the first place. Mathematically, $\sigma(Y)$ is the smallest sub-$\sigma$-field of $\mathscr{F}$ to which $Y$ is measurable. Thanks to the $Y$ in this example is discrete with range $\{0, 1\}$, it can be shown that (it will be a good exercise to check your own understanding of the definition $\sigma(Y)$): \begin{align*} \sigma(Y) = \sigma(\{\{Y = 0\}, \{Y = 1\}\}) = \{\varnothing, \{1\}, \{2, 3\}, \Omega\}. \end{align*}

To clarify, in measure theory, the "$\sigma$" in the notation "$\sigma(Y)$" (when the parameter is a random variable) and that in the notation "$\sigma(\{B_1, B_2\})$" (when the parameter is a collection of $\mathscr{F}$-sets) bear different meanings. I tend to interpret the first $\sigma$ as a rule that is based on the second $\sigma$, which simply means "the smallest $\sigma$-field in $\mathscr{F}$ containing "$\{B_1, B_2\}$". In other words, $\sigma(Y) := \sigma(\{Y^{-1}(B): B \in \mathscr{R}^1\})$.

It is clear from the above expression that $\sigma(Y)$ is a sub-$\sigma$-field generated by a partition $\{B_1, B_2\}$ of $\Omega$, where $B_1 = \{1\}, B_2 = \{2, 3\}$. In general, if $\{B_k: k = 1, 2, \ldots\}$ is a finite or countable partition of $\Omega$, and $\mathscr{G} = \sigma(\{B_k: k = 1, 2, \ldots\})$, then one can show that by verifying the two defining properties of the conditional expectation (this also has an intuitive "observer-information" explanation, see the The Discrete Case subsection in Section 33 of Probability and Measure (3rd edition) by Patrick Billingsley):

\begin{align*} E[X|\mathscr{G}](\omega) = \sum_{k = 1}^\infty E[X|B_k]I_{B_k}(\omega). \tag{1}\label{1} \end{align*}

When $P(B_k) > 0$, then $E[X|B_k] = \frac{E[XI_{B_k}]}{P(B_k)}$, an identity that is widely known in introductory-level probability. That both $E[X|\mathscr{G}]$ (where the conditioning object is a sub-$\sigma$-field) and $E[X|B_k]$ (where the conditioning object is an event), or even sometimes $E[X|Y = 1]$ (see the last paragraph), are referred by the same term "conditional expectation" is an unfortunate abuse of nomenclature.

Now it is readily to apply $\eqref{1}$ to your example: since \begin{align*} & E[X|B_1] = \frac{E[XI_{B_1}]}{P(B_1)} = \frac{\int_{B_1}\omega dP}{\frac{1}{3}} = \frac{1 \times \frac{1}{3}}{\frac{1}{3}} = 1, \\ & E[X|B_2] = \frac{E[XI_{B_2}]}{P(B_2)} = \frac{\int_{B_2}\omega dP}{\frac{2}{3}} = \frac{2 \times \frac{1}{3} + 3 \times \frac{1}{3}}{\frac{2}{3}} = 2.5, \end{align*} it follows that \begin{align*} E[X|Y](\omega) = I_{\{1\}}(\omega) + 2.5I_{\{2, 3\}}(\omega). \tag{2}\label{2} \end{align*}

Unlike versions you proposed in the OP, $\eqref{2}$ is more compact and intuitive: it basically says, when you have observed a realized value of $Y$, you immediately obtain some partial information about where $\omega$ comes from (i.e., which $\sigma(Y)$-measurable set $B_k$ that $\omega$ belongs to), you then simply average $X$ over this set to determine $E[X|Y](\omega)$ (the average is the the constant $E[X|B_k]$ appeared in front of the indicator function). For example, if $Y(\omega) = 1$, then $\omega$ must come from the event $\{1\}$, and clearly $E[X|\{1\}] = 1$, hence $E[X| Y](\omega) = 1$ for all $\omega \in \{Y = 1\}$.

Although many people used the notation $E[X|Y = x]$ such as $E[X|Y = 1]$ in your post, it is not equivalent to the formal conditional expectation whose conditioning object $\mathscr{G}$ is a sub-$\sigma$-field and the outcome is a $\mathscr{G}$-measurable function of $\omega$. To this end, you may check this answer for more details.


To OP's follow-up confusion:

When we condition on the sigma algebra, do we mean that we describe all the conditional values, including conditional on the unions?

This is a clear "NO". To further clarify it, let's go back to the most general form of a "conditional expectation", namely, "$E[X|\mathscr{G}]$", where $\mathscr{G}$ is a sub-$\sigma$-field of $\mathscr{F}$. There are several quite evident reasons why $E[X|\mathscr{G}]$ is not a collection of values $\{E[X|A]: A \in \mathscr{G}\}$, or equivalently, not a function defined by \begin{align*} E[X|\mathscr{G}](A) = E[X|A], \; A \in \mathscr{G}. \tag{3}\label{3} \end{align*}

  1. By definition, $E[X|\mathscr{G}]$ (suppose we fixed any version of it) is a $\mathscr{G}$-measurable function from $\Omega$ to $\mathbb{R}$. So the domain of $E[X|\mathscr{G}]$ is $\Omega$, not $\mathscr{G}$. In other words, $E[X|\mathscr{G}]$ is a point function, not a set function.

  2. For $A \in \mathscr{G}$ with zero probability, what is the meaning of "$E[X|A]$"? Obviously, we no longer can use the definition $E[X|A] = \frac{E[XI_A]}{P(A)}$ because the denominator is $0$.

  3. One easy way to handle the above challenge is, define $E[X|A] \equiv 0$ (or any arbitrary constant $c$) for all $A$ such that $P(A) = 0$. While this might solve the "value error" problem, it would immediately conflict with the intuition. For example, if $X$ is a continuous random variable, so that $P(X = x) = 0$ for all $x$. By intuition, we know that $E[X|X](\{X = x\})$ should give value $x$, but this would then be not consistent with the attempt above. In contrast, the classical conditional expectation doesn't have this issue: as we know $E[X|X] = X$, $E[X|X](\omega) = x$ for any $\omega \in \{X = x\}$. In fact, when brilliantly unrolled the concept of "conditional probability" (which is essentially the same as our topic "conditional expectation" here), Billingsley wrote (p. 432 of the aforementioned reference): "the whole point of this section is the systematic development of a notion of conditional probability that covers conditioning with respect to events of probability $0$".

  4. You might propose other ideas for resolving difficulty (2). But they are likely to fail. This is because (as I mentioned before), most of the time "$E[X|\mathscr{G}]$" lacks of a closed-form expression in terms of the law of $X$ and the structure of $\mathscr{G}$ whereas attempts like $\eqref{3}$ (together with any plan to save zero-probability conditioning events) in effect assigned an explicit form to $E[X|\mathscr{G}]$, which kind of violates the existing knowledge, i.e., $E[X|\mathscr{G}]$ has to be defined implicitly through functional equations $\int_G E[X|\mathscr{G}]dP = \int_G X dP$ for all $G \in \mathscr{G}$.

  5. Last but not least, it is in general impossible to express every set in a $\sigma$-field $\mathscr{G}$ explicitly. For example, in the same reference (pp. 31 - 33), given a family of sets $\mathscr{A} \subset \mathscr{F}$, Billingsley showed that it is impossible to exhaust $G \in \sigma(\mathscr{A})$ by repeated finite and countable set-theoretic operations staring with sets in $\mathscr{A}$. This makes the attempt of defining $E[X|\mathscr{G}]$ explicitly in a way of $\eqref{3}$ infeasible.

As a concluding remark, I have to say that for the discrete case (at least for the discrete case such as your toy example), the "conditional expectation" $\eqref{3}$ proposed by you is not entirely wrong -- it actually highlighted an important property of the classical conditional expectation, i.e., for $\omega$'s in the same $A \in \mathscr{G}$, their conditional expectations shared the identical value. This is actually what "$E[X|\mathscr{G}]$ must be $\mathscr{G}$-measurable" (the first defining property of the conditional expectation) meant by. It is also completely valid to evaluate objects like "$E[X|B_1]$, $E[X|B_2]$, $E[X|B_1 \cup B_2] = E[X|\Omega]$" (on the other hand, "$E[X|\varnothing]$" is not well-defined unless you specify it, see bullet point 2 above), but they are just not the same thing as $E[X|Y]$, and information provided by them is essentially all contained in $E[X|Y]$'s expression $\eqref{2}$. For example, \begin{align*} E[X|B_2] &= \frac{E[XI_{B_2}]}{P(B_2)} = \frac{E[E[XI_{B_2}|Y]]}{P(B_2)} \\ &= \frac{E[I_{B_2}E[X|Y]]}{P(B_2)} = \frac{E[I_{B_2}(I_{B_1} + 2.5I_{B_2})]}{P(B_2)} = 2.5. \end{align*} So, given that we know the expression of $E[X|Y]$, we then already have the key of retrieving $E[X|A]$ for every $A \in \sigma(Y)$. It is thus unnecessary to report every $E[X|A]$ one-by-one.

$\endgroup$
15
  • $\begingroup$ My main problem is $E[Y|\Omega]$ why doesn't this or get included? $\endgroup$ Commented May 22 at 18:56
  • $\begingroup$ @SextusEmpiricus "$I$" means indicator function defined on $\Omega$. Specifically $I_A(\omega) = \begin{cases} 1 & \omega \in A \\ 0 & \omega \in A^c\end{cases}$. $E[X|Y](1) = 1, E[X|Y](2) = E[X|Y](3) = 2.5$. $\endgroup$
    – Zhanxiong
    Commented May 22 at 18:56
  • $\begingroup$ So that's just like the first situation that I described? We do get a variable that as in a function of $\omega$. And $E[Y|\Omega]$ is not part of it. $\endgroup$ Commented May 22 at 18:58
  • 1
    $\begingroup$ First, you are asking what $E[X|\sigma(Y)] = E[X|Y]$ is, not something else (e.g., either $E[X|B_1]$ or $E[X|B_1 \cup B_2]$). The definition of $E[X|Y]$ is unique and unambiguous -- in particular, it is not (at least in writing) a collection of $E[X|A]$, where $A$ ranges over $\sigma(Y)$. $\endgroup$
    – Zhanxiong
    Commented May 22 at 20:08
  • 1
    $\begingroup$ "When we condition on the sigma algebra, do we mean that we describe all the conditional values, including conditional on the unions?" Again, this is not how rigorous conditional expectation is defined. Because in the general case, you simply cannot list every element in a sigma algebra $\mathscr{G}$ (although you can in this toy example), this is something very basic in measure theory. $\endgroup$
    – Zhanxiong
    Commented May 22 at 20:12
6
$\begingroup$

The expectation conditional on a sigma algebra, $E[X|\sigma(Y)]$, does not mean that we compute the values $E[X|A]$ for all $A \in \sigma(Y)$.

Instead, this expectation value is a $\sigma(Y)$ measurable function of the samples $E(\omega) = E[X|\sigma(Y)](\omega)$ such that the integral or sum is equal to $X$, for all events $A$ in $\sigma(\omega)$.

$$\sum_{\omega \in A} E(\omega) p(\omega) = \sum_{\omega \in A} X(\omega) p(\omega)$$

With the example we have four elements $A$ in the sigma algebra $\emptyset$, $Y=0$, $Y=1$ and $Y=0 \lor Y=1$, or in terms of the samples $\{\emptyset\}, \{1\}, \{2,3\}, \{1,2,3\}$ Then the left hand side is

$$ \begin{array}{rcl} \sum_{\emptyset} E(\omega) p(\omega) &=& &&&& 0 \\ \sum_{\omega \in \{1\}} E(\omega) p(\omega) &=& E(1) \cdot p(1) &=& 1\cdot \frac{1}{3} &=& \frac{1}{3} \\ \sum_{\omega \in \{2,3\}}E(\omega) p(\omega) &=& E(2) \cdot p(2) + E(3) \cdot p(3) &=& 2.5 \cdot \frac{1}{3} + 2.5 \cdot \frac{1}{3} &=& \frac{5}{3} \\ \sum_{\omega \in \{1,2,3\}} E(\omega) p(\omega) &=& E(1) \cdot p(1) + E(2) \cdot p(2) + E(3) \cdot p(3) &=& 1 \cdot \frac{1}{3}+ 2.5 \cdot \frac{1}{3} + 2.5 \cdot \frac{1}{3} &=& 2 \end{array}$$ And the right hand side is

$$ \begin{array}{rcl} \sum_{\omega \in \emptyset} X(\omega) p(\omega) &=& &&&& 0 \\ \sum_{\omega \in \{1\}} X(\omega) p(\omega) &=& X(1) \cdot p(1) &=& 1\cdot \frac{1}{3} &=& \frac{1}{3} \\ \sum_{\omega \in \{2,3\}} X(\omega) p(\omega) &=& X(2) \cdot p(2) + X(3) \cdot p(3) &=& 2 \cdot \frac{1}{3} + 3 \cdot \frac{1}{3} &=& \frac{5}{3} \\ \sum_{\omega \in \{1,2,3\}} X(\omega) p(\omega) &=& X(1) \cdot p(1) + X(2) \cdot p(2) + X(3) \cdot p(3) &=& 1 \cdot \frac{1}{3}+ 2 \cdot \frac{1}{3} + 3 \cdot \frac{1}{3} &=& 2 \end{array}$$

$\endgroup$
0
2
$\begingroup$

Apart from the extensive discussion in Zhanxiong's post, one can rely on this general, simple and yet useful result (which has been applied in the said answer) formally stated below:

Result $1.$ Let $(\Omega,\boldsymbol{\mathfrak A},\mathbb P) $ be a probability space. Let ${\mathscr G}$ be a sub–$\sigma$– algebra of $\boldsymbol{\mathfrak A}.$ If $B$ is an atom of $\mathscr G$ relative to $\mathbb P, $ then a.s. on $B,$

$$\mathbf E(X\mid \mathscr G) =\frac{1}{\mathbb P(B) }\int_\Omega Y(\omega)\mathbf 1_B(\omega)~\mathbb P(\mathrm d\omega).$$

One can use this result when $\mathscr G$ is generated by a collection of disjoint $B_i$s whose union is $\Omega.$

The result itself is based on the fact that any measurable function is constant almost surely on an atom. Since, $\mathbf E(X\mid \mathscr G) $ is a $\mathscr G$–measurable function on $\Omega, $ it is a constant $ k, $ say, and this would mean by definition of conditional expectation $\int_B X(\omega) ~\mathbb P(\mathrm d\omega) =k\cdot\mathbb P(B). $

$\blacksquare$

--

Reference:

$\rm[I]$ Real Analysis and Probability, Robert B. Ash, Academic Press, $1972, $ Theorem $6.5.9.$

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.