Reading through Bishop, I stumbled upon this picture on p. 42 top left under the topic of Bayesian classification, but I am unclear on how this can be two posterior distributions, as they seemingly do not integrate to 1.

First of all, we obtain the posterior by multiplying the class conditional likelihood by the prior probability distribution. We do not use the class conditional probabilities to actually classify new data points, but instead we choose $p(C_i | x)$ if $P(C_i | x) > P(C_j | x)$ where $i \neq j$. We can define some some decision boundary, $g_i(x) = log(p(C_i|x)) = log(P(x|c_i) + log(P(c_i)$, but will this decision boundary be knitted to the class conditionals or the posteriors?

enter image description here

  • $\begingroup$ Why should these curves integrate to unity? They're not probability density functions. $\endgroup$ – whuber Jan 13 '20 at 23:02
  • $\begingroup$ @whuber but a posterior probability should be a probability density function, right? $\endgroup$ – Joachim Jan 13 '20 at 23:07
  • $\begingroup$ Not necessarily, because not all distributions even have densities. $\endgroup$ – whuber Jan 13 '20 at 23:25
  • 2
    $\begingroup$ $p(C_1\mid x)$ and $p(C_2\mid x)$ are a (conditional) probability distribution: the values of the (conditional) probability mass function (discrete density function) of a discrete random variable that takes on value $i, i=1,2$ with probability $p(C_i\mid x)$, all this being conditioned on the value of the observation $x$. Notice, for example, that $p(C_1\mid x)+p(C_2\mid x) =1$ for all choices of $x$. $\endgroup$ – Dilip Sarwate Jan 14 '20 at 3:41

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.