I have created a Multinomial Naive Bayes classifier. The dataset looks like this:

Category   Features
Coca-cola  tasty fresh happiness modern ...
Coca-cola  materialist pollution globalization ... 
Pepsi      Sugar energetic ...
Coca-cola  ...
...        ...

As you can see there are a set of features associated with 2 categorical vairables (coca-cola or pepsi). The classifier has an aceptable accuracy in identifying a category after a given set of features.

What I want to know now is if there is a way to understand what the classifier is doing to predict the categories. I mean: what features or patterns is the classifier taking into account to make the prediction?. I supose the classifier is looking for certain patterns that makes it possible to predict categories.

How can I know this patterns? Is there a methodology or test to get this information?

Thanks in advance.


1 Answer 1


Typically, Multinomial Naive Bayes (MNB) classifier uses Bag of Words model, where each word is a feature, and you have a dictionary of all words, pre-defined or constructed/extended with data. For each word, it calculates a class conditional probability, e.g. $P(W=w_i|C=\text{Pepsi})$, which is the probability of occurrence of word $w_i$ in sentences belonging to class Pepsi. These probabilities are estimated from data. Then, a total likelihood of the sentence is calculated by multiplying the probability of appearances of each word, e.g. let $S$ be our sentence: $$P(S|C=\text{Pepsi})=P(W=\text{tasty}|C=\text{Pepsi})P(W=\text{modern}|C=\text{Pepsi})\dots$$

This likelihood is multiplied with the prior probabiliy of the class, i.e. $P(C=\text{Pepsi})$, which is also estimated from the data. Then, we get $P(S|C=\text{Pepsi})P(C=\text{Pepsi})$, which is calculated for each class and Bayes chooses the one with the maximum value, since it is proportional to the posterior probability.

  • $\begingroup$ Thanks for your help and the clarity of your exposition. So I guess the condictional probability of a certain word (feature), for example "awesome" could mean 73% chance of being in "Pepsi" and 34% in "Coca-cola". By studying this I can get what words or combinations of words are more important. Isnt't it? $\endgroup$
    – j4n7
    Jan 23, 2020 at 12:01
  • $\begingroup$ $P(W=w|C=c)$ means the probability of occurrence of word $w$ in class $c$, considering all the words, so yes, it's similar and probabilities will be smaller than your example. $\endgroup$
    – gunes
    Jan 23, 2020 at 12:08

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.