2
$\begingroup$

could someone help me understand how to find the probability $\Pr(F=f_1 | X=x)$ by using the inputs below, where $X$ is a continuous random variable?

Note: I know that probabilities of specific values of continuous random variables (i.e. not intervals) is 0, however I think I heard somewhere that there is a variation of the Bayes theorem by which densities can be used in place of probabilities, and I hope to use this to answer the question that is detailed below.

Question Details:

Let:

  • $\mathcal{X} = [a, b]$ be the universal set of samples.
  • $\mathcal{F} = \{f_1, f_2, \ldots, f_n\}$ be the set of processes that generate samples in $\mathcal{X}$.

  • $X$ be a random variable that takes values in $\mathcal{X}$.

  • $F$ be a random variable that takes values in $\mathcal{F}$.
  • For any $1 \le i \le n$, $X_i$ be a random variable that takes values in $\mathcal{X}$ as generated by process $f_i \in \mathcal{F}$.
  • $\Pr(X)$ be the PDF of r.v. $X$.
  • $\Pr(F)$ be the PMF of r.v. $F$.
  • For any $1 \le i \le n$, $\Pr(X_i)$ be the PDF of r.v. $X_i$.

Suppose that you are given these as input:

  • Some $x$ where $x \in \mathcal{X}$.
  • $\Pr(X)$.
  • $\Pr(F)$.
  • For any $1 \le i \le n$, $\Pr(X_i)$.

Then the question is: what is $\Pr(F=f_1 | X=x)$?


A first guess on a solution:

Let:

  • $PDFX(x)$ be the value of the PDF $\Pr(X)$ at point $x$.
  • For any $1 \le i \le n$, $PDFX_i(x)$ be the value of the PDF $\Pr(X_i)$ at point $x$.

\begin{equation} \begin{split} \Pr(F=f_1 | X=x) &= \frac{\Pr(X=x|F=f_1) \Pr(F=f_1)}{\Pr(X=x)}\\\ &= \frac{PDFX_1(x) \Pr(F=f_1)}{PDFX(x)} \end{split} \end{equation}

Any thoughts?

$\endgroup$
5
  • $\begingroup$ With only the inputs $Pr(x)$ and $Pr(F)$ [terrible notations by the way!], you cannot find the joint distribution of $(F,X)$ hence cannot apply Bayes' theorem. $\endgroup$
    – Xi'an
    Commented Jan 12, 2016 at 20:59
  • $\begingroup$ @Xi'an could you please point out where in the equation (last equation) there is a need for the joint distribution $(F,X)$ that is not in the input? Additionally, you you also please point out why the notation is terrible, and how to fix it? :) -- thank you! $\endgroup$
    – caveman
    Commented Jan 12, 2016 at 21:22
  • $\begingroup$ This is why our elders and betters left us measure theory. $\endgroup$ Commented Jan 13, 2016 at 22:57
  • $\begingroup$ Xi'an is correct, you haven't allowed yourself $Pr(X=x | F=f_1)$. Recall $p(a|b)p(b)=p(a,b)$. $\endgroup$ Commented Jan 13, 2016 at 23:05
  • $\begingroup$ @conjectures, but I have given myself $f_{x|F=f_1}(x)$, which in my terrible notation is $PDFX_1$ as I personally think that $PDFX_1 = f_{x|F=f_1}(x)$, do you agree? $\endgroup$
    – caveman
    Commented Jan 14, 2016 at 16:02

1 Answer 1

0
$\begingroup$

Your result looks correct to me. One way to justify this mixture of continuous and discrete random variables is by means of generalized functions, namely, Dirac delta function.

Discrete random variables don't have a density because their cumulative distribution function is not differentiable at the points of interest. Generalized functions can help with that – just as any polynomial has a root on a complex plane, every function has a (weak) derivative.

So for discrete r.v. $F$ taking values $f_1, \dots, f_n$ with probabilities $\alpha_1, \dots, \alpha_n$ corresponding pdf would be

$$ p_F(f) = \sum_{i=1}^n \alpha_i \delta(f - f_i) $$

Now we can apply Bayes' theorem:

$$ p(F = f | X = x) = \frac{p_{X|F}(x|F = f) p_F(f)}{p_X(x)} = \frac{p_{X|F}(x|F = f) \sum_{i=1}^n \alpha_i \delta(f - f_i)}{p_X(x)}\\ = \sum_{i=1}^n \frac{\alpha_i p_{X|F}(x|F = f)}{p_X(x)} \delta(f - f_i) $$

Note all quantities involved were densities. It's easy to see that resulting density corresponds to a discrete r.v. with PMF (note that each summand is non-zero only if $f - f_i = 0$).

$$ \text{Pr}(F = f_i | X = x) = \frac{\alpha_i p_{X|F}(x|F = f_i)}{p_X(x)} $$

$\endgroup$
4
  • $\begingroup$ I don't get how is this related, can you explain please? What I see is that you are trying to explain how to get a PDF out of a discrete random variable. But the question is about how to get probabilities that are conditioned on continuous random variables. $\endgroup$
    – caveman
    Commented Jan 13, 2016 at 3:22
  • $\begingroup$ PDF of a discrete r.v. is the first part of my answer. I then proceed to calculating conditional pdf, and transforming it into a PMF. $\endgroup$ Commented Jan 13, 2016 at 4:22
  • $\begingroup$ PDF of discrete r.v.? I thought these have only PMFs instead, no? $\endgroup$
    – caveman
    Commented Jan 13, 2016 at 12:58
  • $\begingroup$ Excuse me, but did you read my answer? I explicitly said that for discrete r.v. we need to use generalized functions to define a density function. $\endgroup$ Commented Jan 13, 2016 at 14:31

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.