# Is there a way to know what features a Multinomial Naive Bayes classifier is taking into account to predict categories?

I have created a Multinomial Naive Bayes classifier. The dataset looks like this:

Category   Features
--------------------
Coca-cola  tasty fresh happiness modern ...
Coca-cola  materialist pollution globalization ...
Pepsi      Sugar energetic ...
Coca-cola  ...
...        ...


As you can see there are a set of features associated with 2 categorical vairables (coca-cola or pepsi). The classifier has an aceptable accuracy in identifying a category after a given set of features.

What I want to know now is if there is a way to understand what the classifier is doing to predict the categories. I mean: what features or patterns is the classifier taking into account to make the prediction?. I supose the classifier is looking for certain patterns that makes it possible to predict categories.

How can I know this patterns? Is there a methodology or test to get this information?

Typically, Multinomial Naive Bayes (MNB) classifier uses Bag of Words model, where each word is a feature, and you have a dictionary of all words, pre-defined or constructed/extended with data. For each word, it calculates a class conditional probability, e.g. $$P(W=w_i|C=\text{Pepsi})$$, which is the probability of occurrence of word $$w_i$$ in sentences belonging to class Pepsi. These probabilities are estimated from data. Then, a total likelihood of the sentence is calculated by multiplying the probability of appearances of each word, e.g. let $$S$$ be our sentence: $$P(S|C=\text{Pepsi})=P(W=\text{tasty}|C=\text{Pepsi})P(W=\text{modern}|C=\text{Pepsi})\dots$$
This likelihood is multiplied with the prior probabiliy of the class, i.e. $$P(C=\text{Pepsi})$$, which is also estimated from the data. Then, we get $$P(S|C=\text{Pepsi})P(C=\text{Pepsi})$$, which is calculated for each class and Bayes chooses the one with the maximum value, since it is proportional to the posterior probability.
• $P(W=w|C=c)$ means the probability of occurrence of word $w$ in class $c$, considering all the words, so yes, it's similar and probabilities will be smaller than your example. Jan 23, 2020 at 12:08