3
$\begingroup$

If we have one continuous r.v. $x$ and a discrete r.v. $y$ which takes one of the two values $y_1$ and $y_2$. Let's say we know the prior probabilities $P(y_1)$ and $P(y_2)$.

From Bayes theorem we know

$p(y_i,x) = P(y_i|x)\cdot p(x)$

$P$ corresponds to prob. mass function, $p$ corresponds to prob. density function.

On LHS we have a probability value coming from a continuous distribution. On RHS we have product of two probabilities one coming from a discrete distribution, another coming from a continuous distribution.

My question is - Why is first term on RHS $P(y_i|x)$ and not $p(y_i|x)$ (Since it's possible that, given $x$, the probability that $y$ will take a value of $y_i$, might smoothly vary from $0$ to $1$)? Are there different ways of writing Bayes theorem that will have $p(y_i|x)$ instead of $P(y_i|x)$?

$\endgroup$
3
  • $\begingroup$ I think you're confused. The values that $y_i$ can take don't change from when you condition on $x$ to when you don't. If $y$ is a discrete r.v., it remains discrete. The probabilities it takes those discrete values can change as $x$ changes and that can happen in a smooth way. $\endgroup$
    – Glen_b
    Commented Jul 2, 2014 at 5:56
  • $\begingroup$ @Glen_b forget what values $y_i$ can take, think of what values $P(y_i|x)$ can take. It's not discrete set of probabilities. If we plot $P(y_i|x)$ and it might look continuous. edit. i think i got what u r saying. $\endgroup$
    – user13107
    Commented Jul 2, 2014 at 5:59
  • $\begingroup$ $P(y_i|x)$ considered as a function of $x$ varies continuously with $x$, but $P$ is the pmf for $y$, and as such is defined over the discrete values $y$ takes. $\endgroup$
    – Glen_b
    Commented Jul 2, 2014 at 6:09

2 Answers 2

2
$\begingroup$

(Since it's possible that, given x, the probability that y will take a value of yi, might smoothly vary from 0 to 1)

I believe you are misunderstanding what the distinction between a pdf and a pmf is based on. A pdf corresponds to the case when the argument of the probability function (for lack of a better alternative term) takes values on a continuum. Similarly, a pmf corresponds to the case when the argument of the probability function takes discrete values.

The argument of the probability function of $y|x$ is $y$, not $x$. You should think of $x$ as a constant (arbitrary, but fixed) in the probability function of $y|x$. Since $y$, the argument is discrete, the probability function of $y|x$ is a pmf, and indeed the notation $P(y_i|x)$ is correct.

Note: All my comments are based on the assumption that pdfs/pmfs exist when they need to. I am not talking about pathological cases here.

$\endgroup$
2
  • $\begingroup$ consider this view - a pmf has only a discrete set of values ALLOWED for probability, whereas pdf has infinite number of values allowed for probability. If we consider this, then P(y|x) is capable of taking infinite values and not discrete values. $\endgroup$
    – user13107
    Commented Jul 2, 2014 at 5:25
  • $\begingroup$ i have understood it based on glen's comment. your way of explaining was a bit convoluted for me. anyway upvoted for help. $\endgroup$
    – user13107
    Commented Jul 2, 2014 at 6:08
0
$\begingroup$

I think the way to understand this is to plot $P(y_i|x)$ Vs $y_i$. It's discrete distribution because X axis can only take discrete values. Thanks to @Glen_b for comment.

$\endgroup$
9
  • $\begingroup$ I think a better way to see the difference is actually to plot it in terms of both $y$ and $x$. You'll see a pair of side-by-side smooth densities. If you fix $y=y_i$ it's a continuous function in $x$ (but not a density in general). If you fix $x$ - which is what the conditioning symbol says to do - then it's a slice across those curves - a pair of delta functions, describing the probability that $y=y_i$. $\endgroup$
    – Glen_b
    Commented Jul 2, 2014 at 6:13
  • $\begingroup$ @Glen_b can we write $p(y_i,x)$ as a function of $p(y_i|x)$ and $p(x)$ instead of $P(y_i|x)$ and $p(x)$? Can we write any equation that uses $p(y_i|x)$ in some form or is it a useless concept? $\endgroup$
    – user13107
    Commented Jul 2, 2014 at 6:16
  • $\begingroup$ $y$ is discrete. A probability function for $y$ - no matter what it conditions on - will be a pmf. I am baffled by your difficulty with that point. $\endgroup$
    – Glen_b
    Commented Jul 2, 2014 at 6:18
  • $\begingroup$ @Glen_b I know $y$ is discrete. By $p(y|x)$ i mean when X axis is $x$ instead of $y$. Is this a non-standard concept $p_x(y|x)$? $\endgroup$
    – user13107
    Commented Jul 2, 2014 at 6:19
  • $\begingroup$ @Glen_b consider a plot of $p_x(y|x)$ instead of $P_y(y|x)$. Is the first way of notation non-standard? If not, can it be used in any way in bayes equation? $\endgroup$
    – user13107
    Commented Jul 2, 2014 at 6:23

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.