I feel a bit uneasy with how I've mentally dealt with Borel's paradox and other associated "paradoxes" dealing with conditional probability. For those who are reading this that aren't familiar with it, see this link. My mental response up to this point has been mostly to ignore it because no one seems to talk about it, but I feel I should rectify this.

We know that this paradox exists, and yet it seems like in practice (as an extreme example, Bayesian analysis) we are perfectly fine with conditioning on events of measure $0$; if $X$ is my data, we condition on $X = x$ all the time, even though this is an event of measure $0$ when $X$ is continuous. And we certainly make no effort to construct a sequence of events converging to the event we observed to resolve the paradox, at least not explicitly.

I think this is okay because we have essentially fixed the random variable $X$ (in principle) before the experiment, and so we are conditioning on $\sigma(X)$. That is, $\sigma(X)$ is the natural $\sigma$-algebra to condition on because the information $X = x$ is coming to use through $X$ - if it had come to us in some other fashion, we would condition on a different $\sigma$-algebra. Borel's paradox arises because (I guess) it isn't obvious what the appropriate $\sigma$-algebra to condition on, but the Bayesian has specified $\sigma(X)$. Because we are specifying a priori that the information $X = x$ came to us by means of measuring $X$ we are in the clear. Once we have specified the $\sigma$-algebra, everything is fine; we construct our conditional expectation using Radon-Nikodym and everything is unique up-to null sets.

Is this essentially right, or am I way off? If I'm way off, what is the justification for behaving as we do? [Given the Q&A nature of this site, regard this as my question.] When I took my measure-theoretic probability we, for some reason I don't understand, never even touched conditional expectation. As a result, I'm worried that my ideas are very confused.

  • 2
    $\begingroup$ When I took my measure-theoretic probability we, for some reason I don't understand, never even touched conditional expectation. Whoa. I'm interested in this little snippet. What text did you use? How did you take a course with such a name and never look at martingales, Markov chains, or a number of other "standard" topics? $\endgroup$ – cardinal Mar 2 '13 at 19:41
  • 1
    $\begingroup$ I think the "big picture" behind this answer provides at least a partial response to the present questions. :) $\endgroup$ – cardinal Mar 2 '13 at 19:56
  • 1
    $\begingroup$ @cardinal We did not use a textbook, we used the instructors notes. The instructor spent his entire research career proving laws of large numbers for Banach space valued random elements, and apparently had no need for such things. As a result, he didn't teach them. We learned the topics that he found important for his work. The other professor who taught probability used Billingsley and wasn't as short sighted. I picked up what I know by reading Billingsley in my own time. $\endgroup$ – guy Mar 2 '13 at 19:59
  • 4
    $\begingroup$ Thanks for indulging me and (+1) to your question. By the way, Billingsley is a wonderful reference text, but must have been a bit frustrating as a class test and self-study choice, if for no other reason than the organization. You might be interested in D. Williams' Probability with Martingales if you want a short companion that places a decidedly large emphasis on conditional expectation. Cheers. :-) $\endgroup$ – cardinal Mar 2 '13 at 20:43

As a Bayesian, I would say Borel's paradox has nothing (or very little) to do with Bayesian statistics. Except that Bayesian statistics uses conditional distributions, of course. The fact that there is no paradox in defining a posterior distribution as conditional on a set of measure zero $\{X=x\}$ is that $x$ is not chosen in advance, but as the result of the observation. Thus, if we want to use exotic definitions for the conditional distributions on sets of measure zero, there is zero chance that those sets will contain the $x$ that we will observe in the end. The conditional distribution is defined uniquely almost everywhere and hence almost surely wrt our observation. This is also the meaning of the (great) quote of A. Kolmogorov in the wikipedia entry.

A spot in Bayesian analysis where measure-theoretic subtleties may turn into a paradox is the Savage-Dickey representation of the Bayes factor, since it depends on a specific version of the prior density (as discussed in our paper on the topic...)


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.