3
$\begingroup$

This board has many questions on how to understand conditional expectations w.r.t a $\sigma$-algebra; it's clearly a topic that confuses many. I am one of them. I read all the other questions and answers and I still don't understand the intuition behind it.

I have an example here that leads to the wrong answer, and I hope that by figuring out where I'm going wrong I can figure out more about how these conditional expectations work, and also why $E(\xi \mid \mathcal{H}) = \xi (\omega)$ when $\xi$ is $\mathcal{H}$-measurable.


Here I will follow the definition on wikipedia. Say we have:

  • a probability space $(\Omega, \mathcal{F}, \mathbb{P})$
  • a random variable $\xi : (\Omega, \mathcal{F}) \mapsto (X, \mathcal{B})$
  • a sub-$\sigma$-algebra $\mathcal{H} \subseteq \mathcal{F}$, where $\xi$ is not necessarily $\mathcal{H}$-measurable (i.e. we cannot say that $\sigma (\xi) \subseteq \mathcal{H}$, where $\sigma(\xi)$ is the $\sigma$-algebra generated by $\xi$).

The conditional expectation of $\xi$ w.r.t $\mathcal{H}$, or $E(\xi \mid \mathcal{H})$, is not a constant number but a function $E(\xi \mid \mathcal{H}) : \Omega \mapsto X$. It is defined as any $\mathcal{H}$-measurable function $E(\xi \mid \mathcal{H}) : \Omega \mapsto X$ that satisfies:

\begin{equation} \int _H E(\xi \mid \mathcal{H}) (\omega) d\mathbb{P} (\omega) = \int _H \xi (\omega) d\mathbb{P} (\omega) \tag{1}\label{1} \end{equation}

for any $H \in \mathcal{H}$.

It can then be shown that it is unique and equal to what I think is a Radon-Nikodyn derivative.

Finally, if $\xi$ is $\mathcal{H}$-measurable, we have that $E(\xi \mid \mathcal{H}) = \xi (\omega)$.


Now, if we then consider the following example: $\Omega = \mathbb{R}$, $X = \mathbb{R}$, $\mathcal{B} = \{\emptyset, X\}$, $\mathcal{H} = \{ \emptyset, \Omega\}$, and $\xi$ being any nice continuous function from the reals to the reals.

In this example, it seems obvious to me that $\sigma(\xi) = \{ \emptyset, \Omega\}$ and thus that $\xi$ is $\mathcal{H}$-measurable. Therefore $E(\xi \mid \mathcal{H}) = \xi (\omega)$.

But could I not also satisfy Equation \ref{1} by letting the conditional expectation be a constant? Something like:

\begin{eqnarray} E(\xi \mid \mathcal{H})(\omega) & = \begin{cases} \frac{\int _\Omega \xi (\omega') d\mathbb{P} (\omega')} {\int _\Omega d\mathbb{P} (\omega')} & \omega \in \Omega \tag{2}\label{2} \\ \text{anything} & \omega \in \emptyset \end{cases} \end{eqnarray}

So does this not mean that $E(\xi \mid \mathcal{H})(\omega)$ is not unique? Equation \ref{2} seems to be at odds with $E(\xi \mid \mathcal{H}) = \xi (\omega)$. So what did I do wrong here?

For example: does the standard definition for conditional expectations assume that the $\sigma$-algebra of the random variable is a Borel $\sigma$-algebra? I can see why the above would then be wrong.

$\endgroup$
5
  • $\begingroup$ In the formulation given in Wikipedia, the random variable X maps Omega to Rn, presumably with the usual Borel Sets as the sigma algebra. In that case the only H-measurable function would be a constant and your solution (2) would be the unique solution. The weird thing in the setup here is that the sigma algebra for R has only the two minimal elements. Are you sure all the usual conditional expectation properties hold in that case? $\endgroup$ – abstrusiosity Nov 8 '20 at 17:46
  • $\begingroup$ I am not aware of any situation where the conditional expectation is claimed always to be unique: as always -- with any random variable -- its values may be altered on any set of measure zero without changing its defining properties. In this particular circumstance, though, there appears to be no problem at all, because the condition "$\omega\in\emptyset$" is superfluous, being never true. $\endgroup$ – whuber Nov 8 '20 at 17:49
  • $\begingroup$ Right, I'd assumed that the $\sigma$-algebra on the output of the random variable was general, but if it's assumed to be the Borel one then that would make more sense. I used the two minimal elements for simplicity, so I could understand what was going on, but I guess that might be the problem. Does this mean I can't define conditional expectations if the random variable is not real with a Borel-$\sigma$-algebra? Also another question: can Radon-Nikodym derivatives even be defined if the $\sigma$-algebra is not the Borel one? Because I also found that confusing. $\endgroup$ – Marses Nov 8 '20 at 18:10
  • $\begingroup$ @whuber But in the example, the point is that the conditional expectation could be equal to a constant for all $\omega$, or to $\xi (\omega)$ for all $\omega$. Perhaps conditional expectations don't need to be unique (actually now I check, it doesn't state that it's unique on wikipedia). However, what does that mean then if it's non-unique? How do you pick one then? That means there are conditional expectations for which rules like $E(\xi \mid \mathcal{H}) = \xi (\omega)$ hold, and others for which they don't. $\endgroup$ – Marses Nov 8 '20 at 18:13
  • $\begingroup$ Radon-Nikodym derivatives don't require Borel sets. The PMF of a discrete distribution, for example, is a Radon-Nikodym derivative wrt counting measure. On the other hand, the definition of conditional expectation requires measurability of the function and measurability depends on the $\sigma$-algebra on R as well as the on the sub $\sigma$-algebra H. I'm not absolutely certain, but it looks like the properties you're talking about are derived in the context of Lebesgue integration over the Borel sets on the real line. $\endgroup$ – abstrusiosity Nov 8 '20 at 18:55
4
$\begingroup$

You're not doing anything wrong in your example. However, you're operating outside of the 'usual' framework of conditions: most of the theory of expectations and conditional expectations assumes (not always explicitly) that the 'output' $\sigma$-algebra is the Borel one. (Incidentally, the notation $\mathcal{B}$ in your example is a bit confusing, as $\mathcal{B}$ is usually reserved for the Borel $\sigma$-algebra on the reals.)

In the context of your example, one problem is that any indicator function $1_A$ is measurable regardless of $A$, while normally when defining Lebesgue integrals we have $1_A$ being measurable iff $A\in\mathcal{F}$, which comes from assuming that the 'output' $\sigma$-algebra is the Borel one. That is to say, the definition of the integral assumes that the 'output' $\sigma$-algebra is the Borel one. So, I guess you can still define the expectation $\int _\Omega \xi (\omega) d\mathbb{P} (\omega)$ but be aware that this is implicitly assuming that the 'output' $\sigma$-algebra is Borel.

You can then define the conditional expectation, but the proof of uniqueness of the conditional expectation (up to sets of measure zero) assumes that the 'output' $\sigma$-algebra is Borel e.g. if you look at Section 9.5 in the book by Williams, Probability with Martingales, the uniqueness proof starts by considering two 'versions' of the conditional expectation $Y$ and $\tilde{Y}$ before considering the set $\{Y-\tilde{Y}>n^{-1}\}$ which is a member of the sub-$\sigma$-algebra in the context of the book, but does not necessarily belong to your $\mathcal{H}$. In your example, the conditional expectation is not unique: it can be any random variable that integrates to the same as $\xi$.

To get more intuition I'd suggest looking at the case where a random variable takes only finitely many values, which is covered in Section 9.1 of the same book.

$\endgroup$
4
  • $\begingroup$ Ok, thank you I think that helped a lot actually. In courses I've been reading on stochastic processes and probability, I think there often seem to be results that assume that the $\sigma$-algebra of a random variable is the Borel-$\sigma$-algebra of its output (at least the results and intuition only makes sense to me if that's the case). So I should assume that, unless stated otherwise, the $\sigma$-algebras on random variables are defined to be the Borel ones right? $\endgroup$ – Marses Nov 9 '20 at 12:19
  • $\begingroup$ Or perhaps more accurately: I get the feeling that it is sometimes implied that the $\sigma$-algebra on the random variable's output defines how precisely we can know the random variable's output (i.e. the smallest events in the $\sigma$-algebra are the most precise information we can have about the output). So if I have some sort of discrete $\sigma$-algebra like the one in my question, then it's as though I'm dealing with a discrete random variable where it makes sense to treat the average value of $\xi$ within an event as the most information we can have about $\xi$. $\endgroup$ – Marses Nov 9 '20 at 12:27
  • $\begingroup$ @Marses yes, I would assume that the 'output' $\sigma$-algebra is Borel unless stated otherwise. Your choice of 'output' $\sigma$-algebra in your example seems simple (perhaps because it only has 2 elements?) but it's probably not a good example for developing your intuition because it falls outside of the 'usual' framework. $\endgroup$ – S. Catterall Nov 9 '20 at 12:29
  • 1
    $\begingroup$ @Marses re your 2nd comment, not really. It's the 'input' $\sigma$-algebra that's important in determining how much we can know about the value of the random variable. The output $\sigma$-algebra represents the set of questions we can ask about the rv $X$ e.g. if $(-\infty,2]$ belongs to the output $\sigma$-algebra then we can ask about $\{X<2\}$. If $\{X<2\}$ belongs to $\mathcal{F}$ then we 'know' this event and can compute its probability. The smaller $\mathcal{F}$ is, the less likely we are to know about $\{X<2\}$. $\endgroup$ – S. Catterall Nov 9 '20 at 12:41

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.