19
$\begingroup$

Let $(\Omega,\mathscr{F},\mu)$ be a probability space, given a random variable $\xi:\Omega \to \mathbb{R}$ and a $\sigma$-algebra $\mathscr{G}\subseteq \mathscr{F}$ we can construct a new random variable $E[\xi|\mathscr{G}]$, which is the conditional expectation.


What exactly is the intuition for thinking about $E[\xi|\mathscr{G}]$? I understand the intuition for the following:

(i) $E[\xi|A]$ where $A$ is an event (with positive probability).

(ii) $E[\xi|\eta]$ where $\eta$ is a discrete random variable.

But I cannot visualize $E[\xi|\mathscr{G}]$. I understand the mathematics of it, and I understand that it is defined in such a way to generalize the simpler cases that we can visualize. But nonetheless I do not find this way of thinking to be useful. It remains a mysterious object to me.


For example, let $A$ be an event with $\mu(A)>0$. Form the $\sigma$-algebra $\mathscr{G} = \{ \emptyset, A, A^c, \Omega\}$, the one generated by $A$. Then $E[\xi|\mathscr{G}](\omega)$ would be equal to $\frac{1}{\mu(A)} \int_A \xi$ if $\omega \in A$, and equal to $\frac{1}{\mu(A^c)} \int_{A^c} \xi$ if $\omega \not \in A$. In other words, $E[\xi|\mathscr{G}](\omega) = E[\xi|A]$ if $\omega\in A$, and $E[\xi|\mathscr{G}](\omega) = E[\xi|A^c]$ if $\omega \in A^c$.

The part which is confusing is that $\omega \in \Omega$, so why do we not just write $E[\xi|\mathscr{G}](\omega) = E[\xi|\Omega] = E[\xi]$? Why do we replace $E[\xi|\mathscr{G}]$ by $E[\xi| A\text{ or } A^c]$ depending on whether or not $\omega\in A$, but not permitted to replace $E[\xi|\mathscr{G}]$ by $E[\xi]$?


Note. In responding to this question do not explain this by using the rigorous definition of conditional expectation. I understand that. What I want to understand is what the conditional expectation is supposed to be calculating and why we reject one in place of another.

$\endgroup$
16
+500
$\begingroup$

One way to think about conditional representation is as a projection onto the $\sigma$-algebra $\mathscr{G}$.

enter image description here (from Wikimedia commons)

This is actually rigorously true when talking about square-integrable random variables; in this case $\mathbb{E}[\xi|\mathscr{G}]$ is actually the orthogonal projection of the random variable $\xi$ onto the subspace of $L^2(\Omega)$ consisting of random variables measurable with respect to $\mathscr{G}$. And in fact this even turns out to be true in some sense for $L^1$ random variables via approximation by $L^2$ random variables.

(See the comments for references.)

If one considers $\sigma-$algebras as representing how much information we have available (an interpretation which is de rigueur in the theory of stochastic processes), then larger $\sigma-$algebras mean more possible events and thus more information about possible outcomes, while smaller $\sigma-$algebras mean fewer possible events and thus less information about possible outcomes.

Therefore, projecting the $\mathscr{F}$-measurable random variable $\xi$ onto the smaller $\sigma-$algebra $\mathscr{G}$ means taking our best guess for the value of $\xi$ given the more limited information available from $\mathscr{G}$.

In other words, given only the information from $\mathscr{G}$, and not the whole of information from $\mathscr{F}$, $\mathbb{E}[\xi|\mathscr{G}]$ is in a rigorous sense our best possible guess for what the random variable $\xi$ is.


With regards to your example, I think you might be confusing random variables and their values. A random variable $X$ is a function whose domain is the event space; it is not a number. In other words, $X: \Omega \to \mathbb{R}$, $X \in \{f\ |\ f: \Omega \to \mathbb{R} \}$ whereas for an $\omega \in \Omega$, $X(\omega)\in\mathbb{R}$.

The notation for conditional expectation, in my opinion, is really bad, because it is a random variable itself, i.e. also a function. In contrast, the (regular) expectation of a random variable is a number. The conditional expectation of a random variable is an entirely different quantity from the expectation of the same random variable, i.e., $\mathbb{E}[\xi|\mathscr{G}]$ doesn't even "type-check" with $\mathbb{E}[\xi]$.

In other words, using the symbol $\mathbb{E}$ to denote both regular and conditional expectation is a very big abuse of notation, which leads to much unnecessary confusion.

All of that being said, note that $\mathbb{E}[\xi|\mathscr{G}](\omega)$ is a number (the value of the random variable $\mathbb{E}[\xi|\mathscr{G}]$ evaluated at the value $\omega$), but $\mathbb{E}[\xi|\Omega]$ is a random variable, but it turns out to be a constant random variable (i.e. trivial degenerate), because the $\sigma$-algebra generated by $\Omega$, $\{ \emptyset, \Omega\}$ is trivial/degenerate, and then technically speaking the constant value of this constant random variable, is $\mathbb{E}[\xi]$, where here $\mathbb{E}$ denotes regular expectation and thus a number, not conditional expectation and thus not a random variable.

Also you seem to be confused about what the notation $\mathbb{E}[\xi|A]$ means; technically speaking it is only possible to condition on $\sigma-$algebras, not on individual events, since probability measures are only defined on complete $\sigma-$algebras, not on individual events. Thus, $\mathbb{E}[\xi|A]$ is just (lazy) shorthand for $\mathbb{E}[\xi|\sigma(A)]$, where $\sigma(A)$ stands for the $\sigma-$algebra generated by the event $A$, which is $\{ \emptyset, A, A^c, \Omega\}$. Note that $\sigma(A) = \mathscr{G} = \sigma(A^c)$; in other words, $\mathbb{E}[\xi|A]$, $\mathbb{E}[\xi|\mathscr{G}]$, and $\mathbb{E}[\xi|A^c]$ are all different ways to denote the exact same object.

Finally I just want to add that the intuitive explanation I gave above explains why the constant value of the random variable $\mathbb{E}[\xi|\Omega]=\mathbb{E}[\xi|\sigma(\Omega)]= \mathbb{E}[\xi| \{ \emptyset, \Omega\}]$ is just the number $\mathbb{E}[\xi]$ -- the $\sigma-$algebra $\{ \emptyset, \Omega\}$ represents the least possible amount of information we could have, in fact essentially no information, so under this extreme circumstance the best possible guess we could have for which random variable $\xi$ is is the constant random variable whose constant value is $\mathbb{E}[\xi]$.

Note that all constant random variables are $L^2$ random variables, and they are all measurable with respect to the trivial $\sigma$-algebra $\{\emptyset, \Omega\}$, so indeed we do have that the constant random $\mathbb{E}[\xi]$ is the orthogonal projection of $\xi$ onto the subspace of $L^2(\Omega)$ consisting of random variables measurable with respect to $\{\emptyset, \Omega\}$, as was claimed.

$\endgroup$
  • 2
    $\begingroup$ @William I disagree with you about the use of $E[\xi|A]$ as a ran var. Many books define $E[\xi|A]$ to be a number, not a ran var. It is the best possible estimate of $\xi|_A$. This is a useful notion and highly intuitive. Disregarding it completely, just because you have a generalized notion of cond exp as a ran var is wrong from a pedagogical point-of-view. I am not confused about what a r.v. is, nor do I see how anything I wrote would lead you to thinking like that. $\endgroup$ – Nicolas Bourbaki Aug 18 '16 at 22:57
  • 1
    $\begingroup$ @William Thinking of cond expe as an estimate to the ran var with $\mathscr{G}$ representing information, is something I have seen said before but I never gave it that much thought and tried to find a different way of visualizing cond expec. Using your suggestion, I am going to write up a simple example, and post it as an answer, for myself, and for other people. Perhaps, some people can then elaborate on my example and give a more exotic one. $\endgroup$ – Nicolas Bourbaki Aug 18 '16 at 22:59
  • 1
    $\begingroup$ @NicolasBourbaki I recommend that you look at p.221 of the 4th edition of Durrett's Probability - Theory and Examples. I can refer you to other sources discussing this as well. In any case, it is not really a matter of opinion -- in the most general case, a conditional expectation is a random variable, and conditioning is only done with respect to $\sigma-$algebras; conditioning with respect to an event is conditioning with respect to the $\sigma-$algebra generated by the event, and conditioning with respect to a random variable is conditioning w.r.t. the $\sigma$-algebra generated by the RV $\endgroup$ – Chill2Macht Aug 18 '16 at 23:17
  • 3
    $\begingroup$ @William And I can refer you to sources which do define the cond. exep. of an event to be a real number. I do not know why you are so stuck on this point. One can define it any way, as long as the notions are not mixed up. For pedagogical reasons, teaching a class on prob. theory, and instantly jumping into the most general def., is not illuminating. In either case, it really does not matter in this discussion, and your complaint is about notation/semantics. $\endgroup$ – Nicolas Bourbaki Aug 18 '16 at 23:55
  • 1
    $\begingroup$ @NicolasBourbaki Chapter 5 of Whittle's Probability via Expectation gives a very good account (in my opinion) of both characterizations of conditional expectation, and explains well how each definition relates to and is motivated by the other definition. You are right that the distinction is one more of semantics. My enthusiasm for the more general definition stems (I think) from reading this chapter (5 of Whittle's Probability via Expectation), which made (I believe) good arguments about how the more general definition is in some ways easier to understand. $\endgroup$ – Chill2Macht Mar 23 '17 at 15:30
3
$\begingroup$

I am going to try to elaborate what William suggested.

Let $\Omega$ be the sample space of tossing a coin twice. Define the ran. var. $\xi$ to be the num. of heads that occur in the experiment. Clearly, $E[\xi] = 1$. One way of thinking of what $1$, as an expec. value, represents is as the best possible estimate for $\xi$. If we had to take a guess for what value $\xi$ would take, we would guess $1$. This is because $E[(\xi - 1)^2] \leq E[(\xi - a)^2]$ for any real number $a$.

Denote by $A = \{ HT, HH \}$ to be the event that the first outcome is a head. Let $\mathscr{G} = \{ \emptyset, A, A^c, \Omega\}$ be the $\sigma$-alg. gen. by $A$. We think of $\mathscr{G}$ as representing what we know after the first toss. After the first toss, either heads occured, or heads did not occur. Hence, we are either in the event $A$ or $A^c$ after the first toss.

If we are in the event $A$, then the best possible estimate for $\xi$ would be $E[\xi|A] = 1.5$, and if we are in the event $A^c$, then the best possible estimate for $\xi$ would be $E[\xi|A^c] = 0.5$.

Now define the ran. var. $\eta(\omega)$ to be either $1.5$ or $0.5$ depending on whether or not $\omega\in A$. This ran. var. $\eta$, is a better approximation than $1 = E[\xi]$ since $E[(\xi - \eta)^2] \leq E[(\xi -1)^2]$.

What $\eta$ is doing is providing the answer to the question: what is the best estimate of $\xi$ after the first toss? Since we do not know the information after the first toss, $\eta$ will depend on $A$. Once the event $\mathscr{G}$ is revealed to us, after the first toss, the value of $\eta$ is determined and provides the best possible estimate for $\xi$.

The problem with using $\xi$ as its own estimate, i.e. $0=E[(\xi - \xi)^2] \leq E[(\xi - \eta)^2]$ is as follows. $\xi$ is not well-defined after the first toss. Say the outcome of the experiment is $\omega$ with first outcome being heads, we are in the event $A$, but what is $\xi(\omega)=?$ We do not know from just the first toss, that value is ambiguous to us, and so $\xi$ is not well-defined. More formally, we say that $\xi$ is not $\mathscr{G}$-measurable i.e. its value is not well-defined after the first toss. Thus, $\eta$ is the best possible estimate of $\xi$ after the first toss.

Perhaps, somebody here can come up with a more sophisticated example using the sample space $[0,1]$, with $\xi (\omega) = \omega$, and $\mathscr{G}$ some non-trivial $\sigma$-algebra.

$\endgroup$
1
$\begingroup$

Although you request not to use the formal definition, I think that the formal definition is probably the best way of explaining it.

Wikipedia - conditional expectation:

Then a conditional expectation of X given $\displaystyle \scriptstyle {\mathcal {H}}$, denoted as $\displaystyle \scriptstyle \operatorname {E} (X\mid {\mathcal {H}})$, is any $\displaystyle \scriptstyle {\mathcal {H}}$-measurable function ( $\displaystyle \scriptstyle \Omega \to \mathbb {R} ^{n}$) which satisfies:

$\displaystyle \int _{H}\operatorname {E} (X\mid {\mathcal {H}})\;dP=\int _{H}X\;dP\qquad {\text{for each}}\quad H\in {\mathcal {H}}$

Firstly, it is a $\displaystyle \scriptstyle {\mathcal {H}}$-measurable function. Secondly it has to match the expectation over every measurable (sub)set in $\displaystyle \scriptstyle {\mathcal {H}}$. So for an event,A, the sigma algebra is $ \{A,A^C,\emptyset, \Omega\}$, so clearly it is set as you specified in your question for $\omega \in A/A^c$. Similarly for any discrete random variable ( and combinations of them), we list out all primitive events and assign the expectation given that primitive event.

Now consider tossing a coin an infinite number of times, where at each toss i, you get $1/2^i$, if your coin is tails then your total winnings are $X=\sum _{i=1}^\infty \frac{1}{2^i}c_i$ where $c_i$ = 1 for tails and 0 for heads. Then X is a real random variable on $[0,1]$. After n coin tosses, you know the value of X to precision $1/2^n$, eg after 2 coin tosses it is in [0,1/4], [1/4,1/2], [1/2,3/4] or [3/4,1] - after every coin toss, your associated sigma algebra is getting finer and finer, and similarly the conditional expectation of X is getting more and more precise.

Hopefully this example of a real valued random variable with a sequence of sigma algebras getting finer and finer (Filtration) gets you away from the purely event based intuition you are used to, and clarifies its purpose.

$\endgroup$
  • $\begingroup$ I apologize, but I downvoted this question. It does not answer what I originally asked. Nor does it provide any new information that I did not know before. $\endgroup$ – Nicolas Bourbaki Sep 3 '16 at 19:48
  • $\begingroup$ What I am trying to suggest to you is you do not understand the formal definition as well as you think you do (as the other answer also suggested), so unless you work through what is unintuitive with the formal definition you will not progress. $\endgroup$ – seanv507 Sep 3 '16 at 21:04
  • $\begingroup$ I understand the formal definition just fine. The questions that I asked, I know how to answer them when working from the formal definitions. The 'other answer', was trying to explain my question without using the definition of con. exp. $\endgroup$ – Nicolas Bourbaki Sep 3 '16 at 22:16

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.