# Why Bernoulli Naive Bayes explicitly penalizes the non-occurrence of a feature while Multinominal NB simply ignoring it?

In the naive bayes page in Scikit-Lean, one of differences between Bernoulli Naive Bayes and Multinomial Naive Bayes is that

it (bernoulli) explicitly penalizes the non-occurrence of a feature i that is an indicator for class y, where the multinomial variant would simply ignore a non-occurring feature.

I wanna ask the the reason of this difference.
IMHO, Bernoulli Naive Bayes should also ignore the non-occurrence features like what Multinomial Naive Bayes does.

I found the reason of this difference. The answer is the difference of feature distribution between these two models:

1. In Bernoulli Naive Bayes, the features are viewed as independent binary variables. According to Naive Bayes' wikipedia,

If $x_{i}$ is a boolean expressing the occurrence or absence of the i'th term from the vocabulary, then the likelihood of a document given a class $C_{k}$ is given by $p(\mathbf {x} \mid C_{k})=\prod _{i=1}^{n}p_{ki}^{x_{i}}(1-p_{ki})^{(1-x_{i})}$ where $p_{{ki}}$ is the probability of class $C_{k}$ generating the term $w_{i}$

Therefore, every feature in the feature set should be calculated even when the feature is non-occurring.

2. In Multinomial Naive Bayes, the features obeys a multinomial distribution. According to the wikipedia,

A feature vector ${\mathbf {x}}=(x_{1},\dots ,x_{n})$ is then a histogram, with $x_{i}$ counting the number of times event i was observed in a particular instance. This is the event model typically used for document classification, with events representing the occurrence of a word in a single document (see bag of words assumption). The likelihood of observing a histogram x is given by ${p(\mathbf {x} \mid C_{k})={\frac {(\sum _{i}x_{i})!}{\prod _{i}x_{i}!}}\prod _{i}{p_{ki}}^{x_{i}}}$.

Then, for a non-occurring feature i, the $x_i$ should be zero. It means that $x_i!$ = 1, and ${p_{ki}}^{x_i} = 1$. The non-occurring featurs does not contribute to $p(\mathbf {x} \mid C_{k})$.

To sum up, the different distribution between Bernoulli and Multinomial is the reason.