Conditional probability density from probabilities

Question

I am trying to understand conditional probablility densities in relation to the conditional probablilities. From the Measure-theoretic definition on Wikipedia, if $X$ and $Y$ are non-degenerate and jointly continuous random variables with density $f_{X,Y}(x, y)$ then, if $B$ has positive measure,

$${\displaystyle P(X\in A\mid Y\in B)={\frac {\int _{y\in B}\int _{x\in A}f_{X,Y}(x,y)\,dx\,dy}{\int _{y\in B}\int _{x\in \mathbb {R} }f_{X,Y}(x,y)\,dx\,dy}}.} \tag{1}$$

If we allow $X \in (x,x+\epsilon]$ and $Y \in (y,y+\epsilon]$ for some $\epsilon>0$, then $(1)$ becomes

$$\begin{equation} \begin{array}{ll} P(X\in (x, \ x+\epsilon] \mid Y\in (y, \ y+\epsilon]) &= {\frac {P(X\in (x, \ x+\epsilon] \ \cap \ Y\in (y, \ y+\epsilon])}{P(Y\in (y, \ y+\epsilon])}} \end{array} \end{equation} \tag{2}$$

leading to

$$ {F_{X \mid Y}(x+\epsilon \mid \ y+\epsilon) \ - \ F_{X \mid Y}(x \mid \ y) = {\frac {(F_{X,Y}(x+\epsilon, \ y+\epsilon) \ - \ F_{X,Y}(x, \ y))/\epsilon}{(F_Y(y+\epsilon) - F_Y( y))/\epsilon} \xrightarrow[]{\lim_{\epsilon \rightarrow 0}} \frac{f_{X,Y}(x,y)}{f_{Y}(y)}} } \tag{3}$$

where, $F_{X,Y}$ denotes the joint CDF of $X$ and $Y$, $F_{X \mid Y}$ is the conditional CDF, and $F_{Y}$ is the marginal CDF on $Y$. I seem to understand the limit based approach on the right hand side of equation $(3)$, but what about the left hand side?. How does one get to the following form

$$ f_{X \mid Y}(x \mid y) = \frac{f_{X,Y}(x,y)}{f_{Y}(y)} \tag{4}$$

Answer 1

We ought to be a little concerned about using "$\epsilon$" for two separate purposes. So, let's back up a little and generalize your idea.

Fix numbers $x,y.$ Looking at your original quotient,

suppose for all $\delta \gt 0,$ $\Pr(Y \in (y, y+\delta])$ is nonzero.

This permits us to use an elementary definition of conditional probability as

$${\Pr}_{X\mid Y}(X \in \mathcal{A} \mid Y \in (y, y+\delta]) = \frac{{\Pr}_{X,Y}(X\in \mathcal{A}, Y \in (y, y+\delta])}{{\Pr}_{X,Y}(Y \in (y, y+\delta])}\tag{*}$$

where $\mathcal A \times \mathbb{R}$ is any measurable set.

By definition, the CDF of $(X,Y)$ is

$$F(a,b) = F_{X,Y}(a,b) = \Pr(X \le a, Y \le b)$$

for any real numbers $a,b.$ Apply this in the case $\mathcal{A} = (x, x+\epsilon]$ to re-express the fraction in $(*)$ as

$$\eqalign{ {\Pr}_{X\mid Y}(\mathcal{A} \mid (y, y+\delta]) &= \frac{{\Pr}_{X,Y}(X\in (x,x+\epsilon], Y \in (y, y+\delta])}{{\Pr}_{X,Y}(Y \in (y, y+\delta])} \\ &= \frac{F(x+\epsilon,y+\delta) - F(x,y+\delta) - (F(x+\epsilon, y) - F(x,y))}{F_Y(y+\delta) - F_Y(y)}. }$$

This is perfectly fine because we have assumed the denominator is nonzero. But we want to take the limit as $\delta$ shrinks to zero and this might be undefined. To proceed,

assume $F$ is continuously differentiable in its second argument with derivative $D_2F.$

This permits us to apply L'Hopital's Rule to the $\delta$ limit, allowing us to set

$$\eqalign{{\Pr}_{X\mid Y}(\mathcal{A} \mid Y=y) & := \lim_{\delta\to 0^+} {\Pr}_{X\mid Y}((x,x+\epsilon] \mid (y, y+\delta]) \\ &=\lim_{\delta\to 0^+} \frac{F(x+\epsilon,y+\delta) - F(x,y+\delta) - (F(x+\epsilon, y) - F(x,y))}{F_Y(y+\delta) - F_Y(y)} \\ &= \lim_{\delta\to 0^+} \frac{D_2F(x+\epsilon,y) - D_2F(x,y) - (0 - 0)}{D_2F_Y(y) - 0} \\ &= \frac{D_2F(x+\epsilon,y) - D_2F(x,y)}{D_2F_Y(y)}. }$$

This may look strange, but let's proceed further and

suppose the function $x \to D_2F(x,y)$ is continuously differentiable at $x.$

This permits us to differentiate the preceding with respect to $x$ by taking the limit of the difference quotient thus:

$$\eqalign { f_{X\mid Y}(x,y) & := \lim_{\epsilon\to 0^+} \frac{{\Pr}_{X\mid Y}((-\infty,x+\epsilon] \mid Y=y) - {\Pr}_{X\mid Y}((-\infty,x] \mid Y=y)}{\epsilon} \\ &=\lim_{\epsilon\to 0^+} \frac{{\Pr}_{X\mid Y}((x,x+\epsilon] \mid Y=y)}{\epsilon} \\ &= \lim_{\epsilon\to 0^+}\frac{1}{\epsilon} \left(\frac{D_2F(x+\epsilon,y) - D_2F(x,y)}{D_2F_Y(y)}\right) \\ &= \lim_{\epsilon\to 0^+}\frac{1}{D_2F_Y(y)} \left(\frac{D_2F(x+\epsilon,y) - D_2F(x,y)}{\epsilon}\right) \\ &= \frac{D_1 D_2 F(x,y)}{D_2F(x,y)} \\ &= \frac{f_{X,Y}(x,y)}{f_Y(y)} }$$

in terms of the (conventional) shorthand $f_{X\mid Y}(x,y)$ for the conditional density of $X \mid Y=y,$ $f_{X,Y} = D_1 D_2 F_{X,Y},$ and $f_Y(y)=\mathrm{d}F_Y(y)/\mathrm{d}y:$ QED.

---

(The only reason to take limits through positive values of $\delta$ and $\epsilon$ and to assume continuous differentiability was due to the interval notation; the limits can be evaluated through negative values of $\delta$ and $\epsilon$ using the same method and comparable assumptions.)