# Conditional probability of continuous variable

Suppose that random variable $U$ follows a continuous Uniform distribution with parameters 0 and 10 (i.e. $U \sim \rm{U}(0,10)$ )

Now let's denote A the event that $U$ = 5 and B the event that $U$ is equal either to $5$ or 6. According to my understanding, both events have zero probability to occur.

Now, if we consider to compute $P(A|B)$ , we cannot use the conditional law $P\left( {A|B} \right) = \frac{{P\left( {A \cap B} \right)}}{{P\left( B \right)}}$, because $P(B)$ is equal to zero. However, my intuition tells me that $P(A|B) = 1/2$.

• What would your intuition tell you if $U$ had non-uniform density $0.02u, u \in (0,10)$? Commented Feb 6, 2015 at 4:11
• @DilipSarwate My intuition would tell me that the answer is a number slightly lower than 0.5
– Noob
Commented Feb 6, 2015 at 19:52

"The concept of a conditional probability with regard to an isolated hypothesis whose probability equals 0 is inadmissible." A. Kolmogorov

For continuous random variables, $$X$$ and $$Y$$ say, conditional distributions are defined by the property that they recover the original probability measure, that is, for all measurable sets $$A\in\mathcal{A}(\mathbf{X})$$, $$B\in\mathcal{B}(\mathbf{Y})$$,$$\mathbb{P}(X\in A,Y\in B)=\int_B \text{d}P_Y(y) \int_B \text{d}P_{X|Y}(x|y)$$This implies that the conditional density is defined arbitrarily on sets of measure zero or, on other words, that the conditional density $$p_{X|Y}(x|y)$$ is defined almost everywhere. Since the set $$\{5,6\}$$ is of measure zero against the Lebesgue measure, this means that you can define both $$p(5)$$ and $$p(6)$$ in absolutely arbitrary manners and hence that the probability $$\mathbb{P}(U=5|U\in\{5,6\})$$can take any value.

This does not mean you cannot define a conditional density by the ratio formula $$f(y|x)=f(x,y)\big/f(x)$$as in the bivariate normal case but simply that the density is only defined almost everywhere for both $$x$$ and $$y$$.

"Many quite futile arguments have raged - between otherwise competent probabilists - over which of these results is 'correct'." E.T. Jaynes

The fact that the limiting argument (when $$\epsilon$$ goes to zero) in the above answer seems to give a natural and intuitive answer is related with Borel's paradox. The choice of the parametrisation in the limit matters, as shown by the following example I use in my undergrad classes.

Take the bivariate normal $$X,Y\stackrel{\text{i.i.d.}}{\sim}\mathcal{N}(0,1)$$ What is the conditional density of $$X$$ given that $$X=Y$$?

If one starts from the joint density $$\varphi(x)\varphi(y)$$, the "intuitive" answer is [proportional to] $$\varphi(x)^2$$. This can be obtained by considering the change of variable $$(x,t)=(x,y-x) \sim \varphi(x)\varphi(t+x)$$ where $$T=Y-X$$ has the density $$\varphi(t/\sqrt{2})/\sqrt{2}$$. Hence $$f(x|t)=\dfrac{\varphi(x)\varphi(t+x)}{\varphi(t/\sqrt{2})/\sqrt{2}}$$ and $$f(x|t=0)=\dfrac{\varphi(x)\varphi(x)}{\varphi(0/\sqrt{2})/\sqrt{2}}=\varphi(x)^2\sqrt{2}$$ However, if one considers instead the change of variable $$(x,r)=(x,y/x) \sim \varphi(x)\varphi(rx)|x|$$ the marginal density of $$R=Y/X$$ is the Cauchy density $$\psi(r)=1/\pi\{1+r^2\}$$ and the conditional density of $$X$$ given $$R$$ is $$f(x|r)=\varphi(x)\varphi(rx)|x| \times \pi \{1+r^2\}$$ Therefore, $$f(x|r=1)= \pi\varphi(x)^2|x|/2\,.$$ And here lies the "paradox": the events $$R=1$$ and $$T=0$$ are the same as $$X=Y$$, but they lead to different conditional densities on $$X$$.

• This is just plain wrong. If you take a rigorous course in probability theory you will see that conditioning on events of measure zero is possible, and practical. Consider a bitivariate Gaussian. Everyone knows you can condition on the first variable taking the value zero, though this event has zero probability. See wikipedia. en.wikipedia.org/wiki/… Commented Feb 6, 2015 at 14:47
• What does the word "isolated" mean in that Kolmogorov quote? Does it just mean "outside of the context of a limiting argument"? Commented Mar 2, 2021 at 13:32
• @fblundun: I cannot claim any personal knowledge of Kolmogorov's meaning but the way I understood the "isolated" was as "single" in opposition to the collection of all possible conditioning events. Commented Mar 2, 2021 at 14:50

Xi'an is right that you can't condition on events with zero probability. However, Yair is also right that once you decide on a limiting process, you can evaluate a probability. The problem is there are many limiting processes that arrive at the desired condition.

I think the principle of indifference can sometimes resolve such choices. It argues that the result should not be affected by an arbitrary interchange of labels. in your case, say, flipping the interval so that it's uniform on $(1, 11)$ and the points 5 and 6 have been switched. Flipping changes an answer $p$ to $1-p$. So if you had chosen a different limiting process for one than the other, then you have by an arbitrary change of labels (in this case, changing positive infinity for negative infinity) gotten a different result. That should not happen according to the principle of indifference. Therefore, the answer is 0.5 as you guessed.

Note that many statisticians do not accept the principle of indifference. I like it because it reflects my intuitions. Although I'm not always sure how to apply it, maybe in 50 years it will be more mainstream?

• Thank you for a thoughtful post. I, for one, seriously doubt the "principle of indifference" will ever be mainstream, because it is not workable. Your argument falls apart when the underlying values are re-expressed. The uniform distribution on $[0,10]$ might thereby become, say, a Cauchy distribution, $5$ could become $0$, and $6$ become $\sqrt{1-\frac{2}{\sqrt{5}}}$. Your "principle of indifference" now produces a completely different answer. (I used the probability transforms to work out this example.)
– whuber
Commented Mar 27, 2015 at 21:37
• Sure it does: there are plenty of ways to transform one continuous distribution to another that swap two values. Actually, your "flipping" didn't even preserve the original distribution. (It changed its support altogether.) So it would appear that all you're doing is replacing one distribution by another one. There doesn't seem to be any principle operating here at all.
– whuber
Commented Mar 27, 2015 at 21:45
• That is a much more interesting argument. I want to give it some thought, because densities are tricky to work with. For instance, if $5$ and $6$ had been replaced by $0$ and $10$, the answer might be different. And if the uniform density on $[0,10]$ had been replaced by an equal mixture of uniform densities on $[0,5)$ and $(5,10]$, some delicate reasoning would be needed--even though the underlying distribution in both cases is exactly the same.
– whuber
Commented Mar 28, 2015 at 0:20
• @whuber: You're right. I really liked Potato's answer to one of my questions. I personally think that if there is discrepancy between theory and intuition, that we should seek new, more complete theories. Maybe the "principle of indifference" isn't quite right, or isn't generally workable, but I have a natural desire for probability theory to answer questions for which we have an intuitive understanding. Maybe Lebesgue had the same kind of angst about Riemann integration when he created his integral? Commented Mar 28, 2015 at 0:52
• When theory and intuition diverge, one cure is to change the theory. But first you should try to understand the discrepancy. All that "theory" amounts to, in the end, is drawing ineluctable conclusions from a set of axioms that are taken to be "intuitively obvious" or incontrovertibly demonstrated. When one of those conclusions seems intuitively false, then we should seriously consider that our intuition is inconsistent. Indeed, that's the normal state of affairs for human beings. Thus our default reaction should first be to recalibrate our intuition and not be hasty to abandon the theory.
– whuber
Commented Mar 28, 2015 at 13:51

Yes we can! You can condition on events of zero probability! The math gets complicated - you need some measure theory but you can do it. In simple cases like this I would seek intuition by defining $A = [5 - \frac{\epsilon}{2} , 5 + \frac{\epsilon}{2}]$ and $B = [5 - \frac{\epsilon}{4} , 5 + \frac{\epsilon}{4}] \cup [6 - \frac{\epsilon}{4} , 6 + \frac{\epsilon}{4}]$. Do everything now as you did before and take $\epsilon \to 0$.

Let me stress again (and again) that the above method is used for intuition. Conditioning on events of zero probability is done very often without much thought. The best example I can think of is if $(X_1, X_2) \sim \mathcal{N}(0, \Sigma)$ is a bivariate gaussian. One often considers the density of $X_1$ given (say) $X_2 = 0$, which is an event of measure zero. This is well grounded in theory, but not at all trivial. Regarding @Xi'an's quote of Kolmogorov - I can only quote Varadhan: "One of our goals is to seek a definition that makes sense when $P(\xi = a) = 0$" (Probability Theory, Courant lecture notes, page 74).

So, yes, you can give meaning to conditioning on events of measure zero.

• Suppose $U\sim \text{U}[0,10]$: that is, both $0$ and $10$ are possible. How would you deal with the situation when $A=\{0\}$ and $B=\{0,6\}$? Would $P(A|B)=1/2$ (which "intuitively" is the right answer because all numbers in $[0,10]$ have the same densities) or perhaps $1/3$ (which a simple change of $5$ to $0$ in your formula would give) or even $0$?
– whuber
Commented Feb 6, 2015 at 17:41
• @YairDaon Thank you for you answer! If I understood well, you mean to do the following: for small $\varepsilon$, we have: $P\left( {A|B} \right) = \frac{{P\left( {A \cap B} \right)}}{{P\left( B \right)}} = \frac{{\int\limits_{5 - \frac{\varepsilon }{4}}^{5 + \frac{\varepsilon }{4}} {f\left( u \right)du} }}{{\int\limits_{5 - \frac{\varepsilon }{4}}^{5 + \frac{\varepsilon }{4}} {f\left( u \right)du} + \int\limits_{6 - \frac{\varepsilon }{4}}^{6 + \frac{\varepsilon }{4}} {f\left( u \right)du} }} = \frac{{\frac{\varepsilon }{2}}}{{\frac{\varepsilon }{2} + \frac{\varepsilon }{2}}} = 0.5$
– Noob
Commented Feb 6, 2015 at 20:08
• @YairDaon But I think that the result is not invariant if originally we had defined A as $\left[ {5 - \frac{\varepsilon }{8},5 + \frac{\varepsilon }{8}} \right]$ (and B the same as before). In such a case the result would be ${\frac{1}{8}}$
– Noob
Commented Feb 6, 2015 at 20:10
• It is excellent for the intuition by showing there is no unique answer: that is the basis for Kolmogorov's statement quoted by @Xi'an. The fact you had to change your procedure to make things come out as you thought they should ought to alert you to the problems with this approach.
– whuber
Commented Feb 7, 2015 at 0:14
• The density of $X_2$ given $X_1$ is well-defined, contrary to the density of $X_2$ given $X_1=0$. Commented Feb 7, 2015 at 9:05