19
$\begingroup$

I feel a bit uneasy with how I've mentally dealt with Borel's paradox and other associated "paradoxes" dealing with conditional probability. For those who are reading this that aren't familiar with it, see this link. My mental response up to this point has been mostly to ignore it because no one seems to talk about it, but I feel I should rectify this.

We know that this paradox exists, and yet it seems like in practice (as an extreme example, Bayesian analysis) we are perfectly fine with conditioning on events of measure $0$; if $X$ is my data, we condition on $X = x$ all the time, even though this is an event of measure $0$ when $X$ is continuous. And we certainly make no effort to construct a sequence of events converging to the event we observed to resolve the paradox, at least not explicitly.

I think this is okay because we have essentially fixed the random variable $X$ (in principle) before the experiment, and so we are conditioning on $\sigma(X)$. That is, $\sigma(X)$ is the natural $\sigma$-algebra to condition on because the information $X = x$ is coming to use through $X$ - if it had come to us in some other fashion, we would condition on a different $\sigma$-algebra. Borel's paradox arises because (I guess) it isn't obvious what the appropriate $\sigma$-algebra to condition on, but the Bayesian has specified $\sigma(X)$. Because we are specifying a priori that the information $X = x$ came to us by means of measuring $X$ we are in the clear. Once we have specified the $\sigma$-algebra, everything is fine; we construct our conditional expectation using Radon-Nikodym and everything is unique up-to null sets.

Is this essentially right, or am I way off? If I'm way off, what is the justification for behaving as we do? [Given the Q&A nature of this site, regard this as my question.] When I took my measure-theoretic probability we, for some reason I don't understand, never even touched conditional expectation. As a result, I'm worried that my ideas are very confused.

$\endgroup$
4
  • 3
    $\begingroup$ When I took my measure-theoretic probability we, for some reason I don't understand, never even touched conditional expectation. Whoa. I'm interested in this little snippet. What text did you use? How did you take a course with such a name and never look at martingales, Markov chains, or a number of other "standard" topics? $\endgroup$
    – cardinal
    Mar 2 '13 at 19:41
  • 1
    $\begingroup$ I think the "big picture" behind this answer provides at least a partial response to the present questions. :) $\endgroup$
    – cardinal
    Mar 2 '13 at 19:56
  • 1
    $\begingroup$ @cardinal We did not use a textbook, we used the instructors notes. The instructor spent his entire research career proving laws of large numbers for Banach space valued random elements, and apparently had no need for such things. As a result, he didn't teach them. We learned the topics that he found important for his work. The other professor who taught probability used Billingsley and wasn't as short sighted. I picked up what I know by reading Billingsley in my own time. $\endgroup$
    – guy
    Mar 2 '13 at 19:59
  • 4
    $\begingroup$ Thanks for indulging me and (+1) to your question. By the way, Billingsley is a wonderful reference text, but must have been a bit frustrating as a class test and self-study choice, if for no other reason than the organization. You might be interested in D. Williams' Probability with Martingales if you want a short companion that places a decidedly large emphasis on conditional expectation. Cheers. :-) $\endgroup$
    – cardinal
    Mar 2 '13 at 20:43
10
$\begingroup$

As a Bayesian, I would say Borel's paradox has nothing (or very little) to do with Bayesian statistics. Except that Bayesian statistics uses conditional distributions, of course. The fact that there is no paradox in defining a posterior distribution as conditional on a set of measure zero $\{X=x\}$ is that $x$ is not chosen in advance, but as the result of the observation. Thus, if we want to use exotic definitions for the conditional distributions on sets of measure zero, there is zero chance that those sets will contain the $x$ that we will observe in the end. The conditional distribution is defined uniquely almost everywhere and hence almost surely wrt our observation. This is also the meaning of the (great) quote of A. Kolmogorov in the wikipedia entry.

A spot in Bayesian analysis where measure-theoretic subtleties may turn into a paradox is the Savage-Dickey representation of the Bayes factor, since it depends on a specific version of the prior density (as discussed in our paper on the topic...)

$\endgroup$
2
  • 1
    $\begingroup$ The link to your paper seems broken. $\endgroup$
    – Joe
    Jul 5 at 20:04
  • $\begingroup$ Here is a paper where Borel's paradox actually became relevant in data analysis $\endgroup$ Jul 5 at 20:59
0
$\begingroup$

I'm not sure we ever do condition on events of probability zero in real life. Suppose I measure that a person's mass as 123.45678kg. Going forwards, I'm not conditioning on their mass being exactly 123.45678kg. I'm conditioning on myself having measured their mass as 123.45678kg, something which is consistent with their mass being anywhere in the range [123.456775kg, 123.456785kg] - i.e. it's an event of nonzero probability. So I don't see how the paradox would ever arise.

$\endgroup$
3
  • 1
    $\begingroup$ Well, but we use models which admits of events having zero probability occurring! So we need to use maths that are consistent with the models. As for occuring in practice, see my comment on the other answer by X'ian $\endgroup$ Jul 5 at 21:52
  • $\begingroup$ @kjetilbhalvorsen I guess the argument I'm making is that OP should sleep easier because the paradox doesn't arise when conditioning on real-world events, it is only arises for simplified models of reality. I haven't read that paper you linked - does the author really give an example of a real-world observation with probability 0? $\endgroup$
    – fblundun
    Jul 5 at 22:34
  • 2
    $\begingroup$ No, but an example of Borel paradox arising in discussion of a quite controversial real-world subject (the discussion was in the International Whaling Commission). The content of the Borel Paradox s that one cannot simply define conditional probability on null events by going to a limit! since different ways of construction the limit gives different answers $\endgroup$ Jul 5 at 22:38

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.