From what I understand, Naive Bayes classifies by doing:
$$ y \leftarrow argmax_{y_k}P(Y=y_k)\prod_{i}P(X_i|Y=y_k) $$
There are two things there we need to know: $P(Y=y_k)$ and all the $P(X_i|Y=y_k)$
Now, for estimating the parameters of a probability distribution
- MLE maximizes $P(D|\theta)$
- MAP maximizes $P(\Theta|D) = \frac{P(D|\Theta)P(\Theta}{P(D)}$
The problem for me is bringing everything together.
From what I've read so far, both $P(Y=y_k)$ and all the $P(X_i|Y=y_k)$ are the parameters of the Naive Bayes model.
So these things is what we want to estimate using either MLE or MAP.
So far so good. But now I get confused:
What exactly is $P(D|\Theta)$ in this case? Is it $P(Y=y_k)\prod_{i}P(X_i|Y=y_k)$?
What confuses me is that MLE kinda looks like $P(X_i|Y=y_k)$. But then I read that $P(Y=y_k)$ can also be estimated using MLE.
But the formula for MLE contains a conditional probability. How does $P(Y=y_k)$ fit into the picture?
For MAP, it kinda makes more sense to me, because we assume a prior for $P(Y=y_k)$.