Parameters in Naive Bayes This is from https://scikit-learn.org/stable/modules/naive_bayes.html
In the last line it says

and we can use Maximum A Posteriori (MAP) estimation to estimate
$P(y)$ and $P(x_i|y)$; the former is then the relative frequency of
class $y$ in the training set.

Why would we estimate $P(y)$ and $P(x_i|y)$ through MAP? Shouldn't we calculate $P(y)$ and $P(x_i|y)$ through training data?

 A: There's no such a thing as estimating parameters "through training data". To get point estimates, you can train a probabilistic model by maximizing the likelihood function (MLE) alone, or maximizing the posterior probabilities (MAP). The above description says that it uses MAP, so it considers not only the data but also a prior. In naive Bayes, the common choice is to use Laplace smoothing (uniform prior) to prevent probabilities of zeroes for the unobserved cases, which would zero-out everything in the calculations.
You can just use maximum likelihood estimator, i.e. calculate probability as number of occurrences / sample size, but there are two potential problems:

*

*For every $(x_i, y)$ pair you would need to observe enough data to be able to calculate the $P(x_i|y)$ probability reliably. If $x_i$ is rare or is rare for the particular subgroup $y=c$, you would need to collect a lot of data to calculate it reliably. This is why we often use procedures as Laplace smoothing, or assume other priors so that we can correct the empirical estimates. With Laplace smoothing, we can make the empirical probabilities smoother (more uniform-ish) or more extreme. If you have reasonable prior information to peak an informative prior, you can use it to "correct" the information contained with data based on other information or prior results.


*If for some $(x_i, y)$ pair the number of occurrences of $x_i|y$ would be equal to zero, then the estimated probability $P(x_i|y)$ would be also equal to zero 0 / sample size = 0. In such case
$$
   P(y) \prod_{i=1}^n P(x_i | y)
   $$
would also zero-out no matter what the other probabilities are. Laplace smoothing fixes this problem by forcing the probabilities to be non-zero.
Using the Bayesian approach (MAP) solves the two problems at the cost of making the estimates of the individual probabilities slightly biased.
