Suppose I am thinking about a classification problem and I have my features $X = (X_1,...,X_n)$ and my classification $C$ (taking values in some finite set of classes $c_1,...,c_k$). The naive Bayes classifier assumes that the different individual features $X_1,...,X_n$ are independent when conditioned on $C$. We then construct a classifier by taking the Baysian classifier $$ c(x) = \arg\max_{c_i} P(C=c_i | X = x) $$ and by Bayes rule together with our independence assumption, we have $$ P(C = c_i | X = x) = \frac{P(C = c_i) \prod_j{P(X_j=x_j | C = c_i)}}{P(X = x)} $$

To get a classifier out of this, we need to be able to compute these probabilities. One way to do this is to make distributional assumptions about the conditional distributions of the $X_i$ (for example assuming that they are Bernoulli or Gaussian and then using the data to estimate the parameters of these distributions as well as the marginal distribution of $C$).

On the wikipedia page for Naive Bayes it brings up what is called multinomial naive Bayes classification where we assume that the features $X$ follow multinomial distributions when conditioned on $C$. To get a classifier from this, the parameters of these multinomial distributions are estimated from the data (where we actually do not need the "number of rolls" parameter $n$ as it comes out in the wash when taking the argmax) and then used to compute $P(X = x | C = c_i)$ (without splitting it up).

One thing to note with a random vector that follows a multinomial distribution is that the different components are not independent (e.g., if you roll a 2 on a dice, you know you do not roll a 5). So the multinomial "naive Bayes" classifier does not satisfy the "naive" assumption that the components of $X$ are independent when conditioned on $C$.

Am I understanding this correctly? Why multinomial naive Bayes a naive Bayes classifier (my guess, assuming I am understanding everything else correctly: it is naive (making the distributional assumption) and it uses Bayes' rule).

New contributor
Sprotte is a new contributor to this site. Take care in asking for clarification, commenting, and answering. Check out our Code of Conduct.


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Browse other questions tagged or ask your own question.