Probability to Likelihood

Question

I have a problem on calculating the likelihood of observing a data point x given the predicted lable. My application is on text classification where I have to detect Spam and No Spam documents.

I use a Logistic Regression classifier that gives me the $P(Y=Spam|X=document)$.

Now I want to calculate:

$P(X=x|Y=Spam)$. The probability that I have a document $x$ given that the label is Spam.

This is simplified as follows: $\frac{P(Y=Spam|X=x)*P(X=x)}{P(Y=Spam)}$.

The $P(X=x)$ is known from dataset. The $P(Y=Spam)$ is also know from the dataset as prior.

Is this approach correct for calculating the Likelihood?

Answer 1

Your question is a little bit general, so I'll give a general answer.

$P(A|B) = \frac{P(A, B)}{P(B)} = \frac{P(B|A)P(A)}{P(B)} = \frac{P(B|A)P(A)}{\int{P(B|A)P(A)dA}}$

So $P(B)$ is an integral over all possible data values ($B$), given all possible priors ($A$).

Maybe you'll find the diagram below helpful. It shows in a graphical way Bayes theorem by parts. It is a simplified example, where we have only two variables, of two dimensions each. As you can see, $P(A)$ is "all cases where $A$ is true" ($w$ and $x$) and $P(B)$ is "all cases where $B$ is true" ($w$ and $y$), and $P(A|B)$ is "both $A$ and $B$ are true". However, as you can also see from the diagram, in $P(A|B) = P(B|A)P(A)/P(B)$, the $P(B)$ part stands also for "all the possible values" (i.e. $w, x, y, z$ on the diagram). It is a little bit ambiguous with Bayes theorem, but should be remembered. I hope this clarifies things a little bit.

You could also find helpful introductory chapters of books by Gelman and Krushke.