Is there a way to know what features a Multinomial Naive Bayes classifier is taking into account to predict categories?

Question

I have created a Multinomial Naive Bayes classifier. The dataset looks like this:

Category   Features
--------------------
Coca-cola  tasty fresh happiness modern ...
Coca-cola  materialist pollution globalization ... 
Pepsi      Sugar energetic ...
Coca-cola  ...
...        ...

As you can see there are a set of features associated with 2 categorical vairables (coca-cola or pepsi). The classifier has an aceptable accuracy in identifying a category after a given set of features.

What I want to know now is if there is a way to understand what the classifier is doing to predict the categories. I mean: what features or patterns is the classifier taking into account to make the prediction?. I supose the classifier is looking for certain patterns that makes it possible to predict categories.

How can I know this patterns? Is there a methodology or test to get this information?

Thanks in advance.

Answer 1

Typically, Multinomial Naive Bayes (MNB) classifier uses Bag of Words model, where each word is a feature, and you have a dictionary of all words, pre-defined or constructed/extended with data. For each word, it calculates a class conditional probability, e.g. $P(W=w_i|C=\text{Pepsi})$, which is the probability of occurrence of word $w_i$ in sentences belonging to class Pepsi. These probabilities are estimated from data. Then, a total likelihood of the sentence is calculated by multiplying the probability of appearances of each word, e.g. let $S$ be our sentence: $$P(S|C=\text{Pepsi})=P(W=\text{tasty}|C=\text{Pepsi})P(W=\text{modern}|C=\text{Pepsi})\dots$$

This likelihood is multiplied with the prior probabiliy of the class, i.e. $P(C=\text{Pepsi})$, which is also estimated from the data. Then, we get $P(S|C=\text{Pepsi})P(C=\text{Pepsi})$, which is calculated for each class and Bayes chooses the one with the maximum value, since it is proportional to the posterior probability.