Conditional probability of continuous variable

Question

Suppose that random variable $U$ follows a continuous Uniform distribution with parameters 0 and 10 (i.e. $U \sim \rm{U}(0,10)$ )

Now let's denote A the event that $U$ = 5 and B the event that $U$ is equal either to $5$ or 6. According to my understanding, both events have zero probability to occur.

Now, if we consider to compute $P(A|B)$ , we cannot use the conditional law $P\left( {A|B} \right) = \frac{{P\left( {A \cap B} \right)}}{{P\left( B \right)}}$, because $P(B)$ is equal to zero. However, my intuition tells me that $P(A|B) = 1/2$.

Answer 1

"The concept of a conditional probability with regard to an isolated hypothesis whose probability equals 0 is inadmissible." A. Kolmogorov

For continuous random variables, $X$ and $Y$ say, conditional distributions are defined by the property that they recover the original probability measure, that is, for all measurable sets $A\in\mathcal{A}(\mathbf{X})$, $B\in\mathcal{B}(\mathbf{Y})$,$$\mathbb{P}(X\in A,Y\in B)=\int_B \text{d}P_Y(y) \int_B \text{d}P_{X|Y}(x|y)$$This implies that the conditional density is defined arbitrarily on sets of measure zero or, on other words, that the conditional density $p_{X|Y}(x|y)$ is defined almost everywhere. Since the set $\{5,6\}$ is of measure zero against the Lebesgue measure, this means that you can define both $p(5)$ and $p(6)$ in absolutely arbitrary manners and hence that the probability $$\mathbb{P}(U=5|U\in\{5,6\})$$can take any value.

This does not mean you cannot define a conditional density by the ratio formula $$f(y|x)=f(x,y)\big/f(x)$$as in the bivariate normal case but simply that the density is only defined almost everywhere for both $x$ and $y$.

"Many quite futile arguments have raged - between otherwise competent probabilists - over which of these results is 'correct'." E.T. Jaynes

The fact that the limiting argument (when $\epsilon$ goes to zero) in the above answer seems to give a natural and intuitive answer is related with Borel's paradox. The choice of the parametrisation in the limit matters, as shown by the following example I use in my undergrad classes.

---

Take the bivariate normal $$X,Y\stackrel{\text{i.i.d.}}{\sim}\mathcal{N}(0,1)$$ What is the conditional density of $X$ given that $X=Y$?

---

If one starts from the joint density $\varphi(x)\varphi(y)$, the "intuitive" answer is [proportional to] $\varphi(x)^2$. This can be obtained by considering the change of variable $$(x,t)=(x,y-x) \sim \varphi(x)\varphi(t+x)$$ where $T=Y-X$ has the density $\varphi(t/\sqrt{2})/\sqrt{2}$. Hence $$f(x|t)=\dfrac{\varphi(x)\varphi(t+x)}{\varphi(t/\sqrt{2})/\sqrt{2}}$$ and $$f(x|t=0)=\dfrac{\varphi(x)\varphi(x)}{\varphi(0/\sqrt{2})/\sqrt{2}}=\varphi(x)^2\sqrt{2}$$ However, if one considers instead the change of variable $$(x,r)=(x,y/x) \sim \varphi(x)\varphi(rx)|x|$$ the marginal density of $R=Y/X$ is the Cauchy density $\psi(r)=1/\pi\{1+r^2\}$ and the conditional density of $X$ given $R$ is $$f(x|r)=\varphi(x)\varphi(rx)|x| \times \pi \{1+r^2\}$$ Therefore, $$f(x|r=1)= \pi\varphi(x)^2|x|/2\,.$$ And here lies the "paradox": the events $R=1$ and $T=0$ are the same as $X=Y$, but they lead to different conditional densities on $X$.

Answer 2

Here's a controversial answer:

Xi'an is right that you can't condition on events with zero probability. However, Yair is also right that once you decide on a limiting process, you can evaluate a probability. The problem is there are many limiting processes that arrive at the desired condition.

I think the principle of indifference can sometimes resolve such choices. It argues that the result should not be affected by an arbitrary interchange of labels. in your case, say, flipping the interval so that it's uniform on $(1, 11)$ and the points 5 and 6 have been switched. Flipping changes an answer $p$ to $1-p$. So if you had chosen a different limiting process for one than the other, then you have by an arbitrary change of labels (in this case, changing positive infinity for negative infinity) gotten a different result. That should not happen according to the principle of indifference. Therefore, the answer is 0.5 as you guessed.

Note that many statisticians do not accept the principle of indifference. I like it because it reflects my intuitions. Although I'm not always sure how to apply it, maybe in 50 years it will be more mainstream?

Answer 3

Yes we can! You can condition on events of zero probability! The math gets complicated - you need some measure theory but you can do it. In simple cases like this I would seek intuition by defining $A = [5 - \frac{\epsilon}{2} , 5 + \frac{\epsilon}{2}]$ and $B = [5 - \frac{\epsilon}{4} , 5 + \frac{\epsilon}{4}] \cup [6 - \frac{\epsilon}{4} , 6 + \frac{\epsilon}{4}]$. Do everything now as you did before and take $\epsilon \to 0$.

Let me stress again (and again) that the above method is used for intuition. Conditioning on events of zero probability is done very often without much thought. The best example I can think of is if $(X_1, X_2) \sim \mathcal{N}(0, \Sigma)$ is a bivariate gaussian. One often considers the density of $X_1$ given (say) $X_2 = 0$, which is an event of measure zero. This is well grounded in theory, but not at all trivial. Regarding @Xi'an's quote of Kolmogorov - I can only quote Varadhan: "One of our goals is to seek a definition that makes sense when $P(\xi = a) = 0$" (Probability Theory, Courant lecture notes, page 74).

So, yes, you can give meaning to conditioning on events of measure zero.