I was looking at the Naive Bayes classifier models (Binomial, Multinomial and Gaussian) and trying to understand the theory behind them a bit better, but am unsure if I understand the MLE approach correctly and use the correct terminology.
Let's use C for Class and W for word.
Then the Bayes Theorem looks like this: $p(C|W) = \frac{p(W|C)*p(C)}{p(W)}$.
We ignore the denominator and take the log of everything:
$ln(p(C|W)) = ln(p(W|C)) + ln(p(C))$
So, this part $p(W|C)$ is called the likelihood and $ln(p(W|C))$ is then the log-likelihood. This $ln(p(C))$ is the log class prior.
The entire expression is the log posterior.
And even though only one part of the formula is called likelihood, we use the Maximum Likelihood approach both for the likelihood and the prior, right?
In a two-class Naive Bayes, the prior is a Binomial, so we just do an MLE for that one parameter. In a multi-class-problem, it's a Multinomial.
And if the class-conditional is, say, a Gaussian, we do the MLE for the two parameters of that Gaussian.
What I am getting at: It's a bit confusing that only one part of the formula is called likelihood but we actually do two separate ML estimations, one for the actual likelihood and one for the prior.
Is there anything wrong with the above?
One reason I am asking this is because I was looking at this blog post https://towardsdatascience.com/quadratic-discriminant-analysis-ae55d8a8148a and under "Derivation and training", I think the writer uses the term "log-likelihood" for what I called "log posterior distribution".
Help and feedback are appreciated!