I have a problem on calculating the likelihood of observing a data point x given the predicted lable. My application is on text classification where I have to detect Spam and No Spam documents.

I use a Logistic Regression classifier that gives me the $P(Y=Spam|X=document)$.

Now I want to calculate:

$P(X=x|Y=Spam)$. The probability that I have a document $x$ given that the label is Spam.

This is simplified as follows: $\frac{P(Y=Spam|X=x)*P(X=x)}{P(Y=Spam)}$.

The $P(X=x)$ is known from dataset. The $P(Y=Spam)$ is also know from the dataset as prior.

Is this approach correct for calculating the Likelihood?


1 Answer 1


Your question is a little bit general, so I'll give a general answer.

$P(A|B) = \frac{P(A, B)}{P(B)} = \frac{P(B|A)P(A)}{P(B)} = \frac{P(B|A)P(A)}{\int{P(B|A)P(A)dA}}$

So $P(B)$ is an integral over all possible data values ($B$), given all possible priors ($A$).

Maybe you'll find the diagram below helpful. It shows in a graphical way Bayes theorem by parts. It is a simplified example, where we have only two variables, of two dimensions each. As you can see, $P(A)$ is "all cases where $A$ is true" ($w$ and $x$) and $P(B)$ is "all cases where $B$ is true" ($w$ and $y$), and $P(A|B)$ is "both $A$ and $B$ are true". However, as you can also see from the diagram, in $P(A|B) = P(B|A)P(A)/P(B)$, the $P(B)$ part stands also for "all the possible values" (i.e. $w, x, y, z$ on the diagram). It is a little bit ambiguous with Bayes theorem, but should be remembered. I hope this clarifies things a little bit.

Wikipedia graph on Bayes theorem

You could also find helpful introductory chapters of books by Gelman and Krushke.

  • $\begingroup$ It would be helpful to explain how this generic picture actually applies to the question. $\endgroup$
    – whuber
    Oct 13, 2014 at 17:36

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.

Not the answer you're looking for? Browse other questions tagged or ask your own question.