# Conditional probability density from probabilities

I am trying to understand conditional probablility densities in relation to the conditional probablilities. From the Measure-theoretic definition on Wikipedia, if $$X$$ and $$Y$$ are non-degenerate and jointly continuous random variables with density $$f_{X,Y}(x, y)$$ then, if $$B$$ has positive measure,

$$P(X\in A\mid Y\in B)={\frac {\int _{y\in B}\int _{x\in A}f_{X,Y}(x,y)\,dx\,dy}{\int _{y\in B}\int _{x\in \mathbb {R} }f_{X,Y}(x,y)\,dx\,dy}}.} \tag{1}$$

If we allow $$X \in (x,x+\epsilon]$$ and $$Y \in (y,y+\epsilon]$$ for some $$\epsilon>0$$, then $$(1)$$ becomes

$$\begin{equation} \begin{array}{ll} P(X\in (x, \ x+\epsilon] \mid Y\in (y, \ y+\epsilon]) &= {\frac {P(X\in (x, \ x+\epsilon] \ \cap \ Y\in (y, \ y+\epsilon])}{P(Y\in (y, \ y+\epsilon])}} \end{array} \end{equation} \tag{2}$$

$${F_{X \mid Y}(x+\epsilon \mid \ y+\epsilon) \ - \ F_{X \mid Y}(x \mid \ y) = {\frac {(F_{X,Y}(x+\epsilon, \ y+\epsilon) \ - \ F_{X,Y}(x, \ y))/\epsilon}{(F_Y(y+\epsilon) - F_Y( y))/\epsilon} \xrightarrow[]{\lim_{\epsilon \rightarrow 0}} \frac{f_{X,Y}(x,y)}{f_{Y}(y)}} } \tag{3}$$

where, $$F_{X,Y}$$ denotes the joint CDF of $$X$$ and $$Y$$, $$F_{X \mid Y}$$ is the conditional CDF, and $$F_{Y}$$ is the marginal CDF on $$Y$$. I seem to understand the limit based approach on the right hand side of equation $$(3)$$, but what about the left hand side?. How does one get to the following form

$$f_{X \mid Y}(x \mid y) = \frac{f_{X,Y}(x,y)}{f_{Y}(y)} \tag{4}$$

• Sorry, what do you mean by understanding the LHS but not the RHS? Apologies, but could you be a bit more specific.. I don't quite get what you're asking Jun 8 '20 at 2:29
• Are you trying to understand this from a pure measure-theoretic point of view, or just general probability theory? (i.e. do you know what a conditional probability is?) Doing it from the measure-theoretic approach might be a bit too much if you're just trying to get a basic understanding. Jun 8 '20 at 4:03
• @tchainzzz Yes, I understand what conditional probability is if I utilize a frequentists interpretation. However, I am trying to understand, form the measure-theoretic approach, the connection between the probabilities and densities. A meaningful derivation of some form.
– jsid
Jun 8 '20 at 10:07
• (1) How do you obtain the first equality in the expression after "leading to"? (2) This approach doesn't work in general. You need the Radon-Nikodym theorem to make this limiting argument rigorously. (3) This is purely mathematical and therefore has nothing to do with any interpretations of probability.
– whuber
Jun 8 '20 at 15:38
• Yes, I realise that now. Then should the numerator be $\frac{F(x+\epsilon, y+\epsilon) + F(x, y) - F(x+\epsilon, y) - F(x, y+\epsilon)}{\epsilon}$ ?. Regardless of that, can one loosly show $(4)$ from $(2)$ via such limiting argument?.
– jsid
Jun 8 '20 at 18:43

We ought to be a little concerned about using "$$\epsilon$$" for two separate purposes. So, let's back up a little and generalize your idea.

Fix numbers $$x,y.$$ Looking at your original quotient,

suppose for all $$\delta \gt 0,$$ $$\Pr(Y \in (y, y+\delta])$$ is nonzero.

This permits us to use an elementary definition of conditional probability as

$${\Pr}_{X\mid Y}(X \in \mathcal{A} \mid Y \in (y, y+\delta]) = \frac{{\Pr}_{X,Y}(X\in \mathcal{A}, Y \in (y, y+\delta])}{{\Pr}_{X,Y}(Y \in (y, y+\delta])}\tag{*}$$

where $$\mathcal A \times \mathbb{R}$$ is any measurable set.

By definition, the CDF of $$(X,Y)$$ is

$$F(a,b) = F_{X,Y}(a,b) = \Pr(X \le a, Y \le b)$$

for any real numbers $$a,b.$$ Apply this in the case $$\mathcal{A} = (x, x+\epsilon]$$ to re-express the fraction in $$(*)$$ as

\eqalign{ {\Pr}_{X\mid Y}(\mathcal{A} \mid (y, y+\delta]) &= \frac{{\Pr}_{X,Y}(X\in (x,x+\epsilon], Y \in (y, y+\delta])}{{\Pr}_{X,Y}(Y \in (y, y+\delta])} \\ &= \frac{F(x+\epsilon,y+\delta) - F(x,y+\delta) - (F(x+\epsilon, y) - F(x,y))}{F_Y(y+\delta) - F_Y(y)}. }

This is perfectly fine because we have assumed the denominator is nonzero. But we want to take the limit as $$\delta$$ shrinks to zero and this might be undefined. To proceed,

assume $$F$$ is continuously differentiable in its second argument with derivative $$D_2F.$$

This permits us to apply L'Hopital's Rule to the $$\delta$$ limit, allowing us to set

\eqalign{{\Pr}_{X\mid Y}(\mathcal{A} \mid Y=y) & := \lim_{\delta\to 0^+} {\Pr}_{X\mid Y}((x,x+\epsilon] \mid (y, y+\delta]) \\ &=\lim_{\delta\to 0^+} \frac{F(x+\epsilon,y+\delta) - F(x,y+\delta) - (F(x+\epsilon, y) - F(x,y))}{F_Y(y+\delta) - F_Y(y)} \\ &= \lim_{\delta\to 0^+} \frac{D_2F(x+\epsilon,y) - D_2F(x,y) - (0 - 0)}{D_2F_Y(y) - 0} \\ &= \frac{D_2F(x+\epsilon,y) - D_2F(x,y)}{D_2F_Y(y)}. }

This may look strange, but let's proceed further and

suppose the function $$x \to D_2F(x,y)$$ is continuously differentiable at $$x.$$

This permits us to differentiate the preceding with respect to $$x$$ by taking the limit of the difference quotient thus:

\eqalign { f_{X\mid Y}(x,y) & := \lim_{\epsilon\to 0^+} \frac{{\Pr}_{X\mid Y}((-\infty,x+\epsilon] \mid Y=y) - {\Pr}_{X\mid Y}((-\infty,x] \mid Y=y)}{\epsilon} \\ &=\lim_{\epsilon\to 0^+} \frac{{\Pr}_{X\mid Y}((x,x+\epsilon] \mid Y=y)}{\epsilon} \\ &= \lim_{\epsilon\to 0^+}\frac{1}{\epsilon} \left(\frac{D_2F(x+\epsilon,y) - D_2F(x,y)}{D_2F_Y(y)}\right) \\ &= \lim_{\epsilon\to 0^+}\frac{1}{D_2F_Y(y)} \left(\frac{D_2F(x+\epsilon,y) - D_2F(x,y)}{\epsilon}\right) \\ &= \frac{D_1 D_2 F(x,y)}{D_2F(x,y)} \\ &= \frac{f_{X,Y}(x,y)}{f_Y(y)} }

in terms of the (conventional) shorthand $$f_{X\mid Y}(x,y)$$ for the conditional density of $$X \mid Y=y,$$ $$f_{X,Y} = D_1 D_2 F_{X,Y},$$ and $$f_Y(y)=\mathrm{d}F_Y(y)/\mathrm{d}y:$$ QED.

(The only reason to take limits through positive values of $$\delta$$ and $$\epsilon$$ and to assume continuous differentiability was due to the interval notation; the limits can be evaluated through negative values of $$\delta$$ and $$\epsilon$$ using the same method and comparable assumptions.)

• Why do $D_2F(x+\epsilon,y)$, $D_2F(x,y)$, and $D_2F_Y(y)$ turn $0$ in the L'Hopital's part of your answer?. Did you use the limit of the difference quotient on the following: $\lim_{\delta\to 0^+} \frac{\left( F(x+\epsilon,y+\delta) - F(x+\epsilon, y)\right )/\delta - \left(F(x,y+\delta) - F(x,y)\right)/\delta}{\left(F_Y(y+\delta) - F_Y(y)\right)/\delta}$ part?.
– jsid
Jun 8 '20 at 23:09
• L'Hopital's rule differentiates numerator and denominator (separately) with respect to the limiting variable $\delta.$ Any quantity that does not vary with $\delta$ has a zero derivative.
– whuber
Jun 9 '20 at 12:52