I am trying to understand conditional probablility densities in relation to the conditional probablilities. From the Measure-theoretic definition on Wikipedia, if $X$ and $Y$ are non-degenerate and jointly continuous random variables with density $f_{X,Y}(x, y)$ then, if $B$ has positive measure,

$${\displaystyle P(X\in A\mid Y\in B)={\frac {\int _{y\in B}\int _{x\in A}f_{X,Y}(x,y)\,dx\,dy}{\int _{y\in B}\int _{x\in \mathbb {R} }f_{X,Y}(x,y)\,dx\,dy}}.} \tag{1}$$

If we allow $X \in (x,x+\epsilon]$ and $Y \in (y,y+\epsilon]$ for some $\epsilon>0$, then $(1)$ becomes

$$\begin{equation} \begin{array}{ll} P(X\in (x, \ x+\epsilon] \mid Y\in (y, \ y+\epsilon]) &= {\frac {P(X\in (x, \ x+\epsilon] \ \cap \ Y\in (y, \ y+\epsilon])}{P(Y\in (y, \ y+\epsilon])}} \end{array} \end{equation} \tag{2}$$

leading to

$$ {F_{X \mid Y}(x+\epsilon \mid \ y+\epsilon) \ - \ F_{X \mid Y}(x \mid \ y) = {\frac {(F_{X,Y}(x+\epsilon, \ y+\epsilon) \ - \ F_{X,Y}(x, \ y))/\epsilon}{(F_Y(y+\epsilon) - F_Y( y))/\epsilon} \xrightarrow[]{\lim_{\epsilon \rightarrow 0}} \frac{f_{X,Y}(x,y)}{f_{Y}(y)}} } \tag{3}$$

where, $F_{X,Y}$ denotes the joint CDF of $X$ and $Y$, $F_{X \mid Y}$ is the conditional CDF, and $F_{Y}$ is the marginal CDF on $Y$. I seem to understand the limit based approach on the right hand side of equation $(3)$, but what about the left hand side?. How does one get to the following form

$$ f_{X \mid Y}(x \mid y) = \frac{f_{X,Y}(x,y)}{f_{Y}(y)} \tag{4}$$

  • $\begingroup$ Sorry, what do you mean by understanding the LHS but not the RHS? Apologies, but could you be a bit more specific.. I don't quite get what you're asking $\endgroup$
    – doubled
    Jun 8 '20 at 2:29
  • $\begingroup$ Are you trying to understand this from a pure measure-theoretic point of view, or just general probability theory? (i.e. do you know what a conditional probability is?) Doing it from the measure-theoretic approach might be a bit too much if you're just trying to get a basic understanding. $\endgroup$
    – tchainzzz
    Jun 8 '20 at 4:03
  • $\begingroup$ @tchainzzz Yes, I understand what conditional probability is if I utilize a frequentists interpretation. However, I am trying to understand, form the measure-theoretic approach, the connection between the probabilities and densities. A meaningful derivation of some form. $\endgroup$
    – jsid
    Jun 8 '20 at 10:07
  • $\begingroup$ (1) How do you obtain the first equality in the expression after "leading to"? (2) This approach doesn't work in general. You need the Radon-Nikodym theorem to make this limiting argument rigorously. (3) This is purely mathematical and therefore has nothing to do with any interpretations of probability. $\endgroup$
    – whuber
    Jun 8 '20 at 15:38
  • 1
    $\begingroup$ Yes, I realise that now. Then should the numerator be $\frac{F(x+\epsilon, y+\epsilon) + F(x, y) - F(x+\epsilon, y) - F(x, y+\epsilon)}{\epsilon}$ ?. Regardless of that, can one loosly show $(4)$ from $(2)$ via such limiting argument?. $\endgroup$
    – jsid
    Jun 8 '20 at 18:43

We ought to be a little concerned about using "$\epsilon$" for two separate purposes. So, let's back up a little and generalize your idea.

Fix numbers $x,y.$ Looking at your original quotient,

suppose for all $\delta \gt 0,$ $\Pr(Y \in (y, y+\delta])$ is nonzero.

This permits us to use an elementary definition of conditional probability as

$${\Pr}_{X\mid Y}(X \in \mathcal{A} \mid Y \in (y, y+\delta]) = \frac{{\Pr}_{X,Y}(X\in \mathcal{A}, Y \in (y, y+\delta])}{{\Pr}_{X,Y}(Y \in (y, y+\delta])}\tag{*}$$

where $\mathcal A \times \mathbb{R}$ is any measurable set.

By definition, the CDF of $(X,Y)$ is

$$F(a,b) = F_{X,Y}(a,b) = \Pr(X \le a, Y \le b)$$

for any real numbers $a,b.$ Apply this in the case $\mathcal{A} = (x, x+\epsilon]$ to re-express the fraction in $(*)$ as

$$\eqalign{ {\Pr}_{X\mid Y}(\mathcal{A} \mid (y, y+\delta]) &= \frac{{\Pr}_{X,Y}(X\in (x,x+\epsilon], Y \in (y, y+\delta])}{{\Pr}_{X,Y}(Y \in (y, y+\delta])} \\ &= \frac{F(x+\epsilon,y+\delta) - F(x,y+\delta) - (F(x+\epsilon, y) - F(x,y))}{F_Y(y+\delta) - F_Y(y)}. }$$

This is perfectly fine because we have assumed the denominator is nonzero. But we want to take the limit as $\delta$ shrinks to zero and this might be undefined. To proceed,

assume $F$ is continuously differentiable in its second argument with derivative $D_2F.$

This permits us to apply L'Hopital's Rule to the $\delta$ limit, allowing us to set

$$\eqalign{{\Pr}_{X\mid Y}(\mathcal{A} \mid Y=y) & := \lim_{\delta\to 0^+} {\Pr}_{X\mid Y}((x,x+\epsilon] \mid (y, y+\delta]) \\ &=\lim_{\delta\to 0^+} \frac{F(x+\epsilon,y+\delta) - F(x,y+\delta) - (F(x+\epsilon, y) - F(x,y))}{F_Y(y+\delta) - F_Y(y)} \\ &= \lim_{\delta\to 0^+} \frac{D_2F(x+\epsilon,y) - D_2F(x,y) - (0 - 0)}{D_2F_Y(y) - 0} \\ &= \frac{D_2F(x+\epsilon,y) - D_2F(x,y)}{D_2F_Y(y)}. }$$

This may look strange, but let's proceed further and

suppose the function $x \to D_2F(x,y)$ is continuously differentiable at $x.$

This permits us to differentiate the preceding with respect to $x$ by taking the limit of the difference quotient thus:

$$\eqalign { f_{X\mid Y}(x,y) & := \lim_{\epsilon\to 0^+} \frac{{\Pr}_{X\mid Y}((-\infty,x+\epsilon] \mid Y=y) - {\Pr}_{X\mid Y}((-\infty,x] \mid Y=y)}{\epsilon} \\ &=\lim_{\epsilon\to 0^+} \frac{{\Pr}_{X\mid Y}((x,x+\epsilon] \mid Y=y)}{\epsilon} \\ &= \lim_{\epsilon\to 0^+}\frac{1}{\epsilon} \left(\frac{D_2F(x+\epsilon,y) - D_2F(x,y)}{D_2F_Y(y)}\right) \\ &= \lim_{\epsilon\to 0^+}\frac{1}{D_2F_Y(y)} \left(\frac{D_2F(x+\epsilon,y) - D_2F(x,y)}{\epsilon}\right) \\ &= \frac{D_1 D_2 F(x,y)}{D_2F(x,y)} \\ &= \frac{f_{X,Y}(x,y)}{f_Y(y)} }$$

in terms of the (conventional) shorthand $f_{X\mid Y}(x,y)$ for the conditional density of $X \mid Y=y,$ $f_{X,Y} = D_1 D_2 F_{X,Y},$ and $f_Y(y)=\mathrm{d}F_Y(y)/\mathrm{d}y:$ QED.

(The only reason to take limits through positive values of $\delta$ and $\epsilon$ and to assume continuous differentiability was due to the interval notation; the limits can be evaluated through negative values of $\delta$ and $\epsilon$ using the same method and comparable assumptions.)

  • $\begingroup$ Why do $D_2F(x+\epsilon,y)$, $D_2F(x,y)$, and $D_2F_Y(y)$ turn $0$ in the L'Hopital's part of your answer?. Did you use the limit of the difference quotient on the following: $\lim_{\delta\to 0^+} \frac{\left( F(x+\epsilon,y+\delta) - F(x+\epsilon, y)\right )/\delta - \left(F(x,y+\delta) - F(x,y)\right)/\delta}{\left(F_Y(y+\delta) - F_Y(y)\right)/\delta}$ part?. $\endgroup$
    – jsid
    Jun 8 '20 at 23:09
  • $\begingroup$ L'Hopital's rule differentiates numerator and denominator (separately) with respect to the limiting variable $\delta.$ Any quantity that does not vary with $\delta$ has a zero derivative. $\endgroup$
    – whuber
    Jun 9 '20 at 12:52

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.