# Is the Naive Bayes family of classifiers linear?

There are a lot of places where you'll see the proof that Naive Bayes classifiers are linear, like this and this. But they always assume a special case of the family of Naive Bayes classifiers which more often than not happens to be Multinomial Naive Bayes.

$$P(X_1 = x_1, X_2 = x_2, \cdots , X_k = x_k) = \frac{n!}{x_1!x_2!\cdots x_k!} p_1^{x_1} p_2^{x_2} \cdots p_k^{x_k}$$

Now, Multinomial Naive Bayes can be proved to be a linear classifier as follows:

Considering a binary classification example with $y \in \{0, 1\}$, the decision boundary would be:

$$$$log(\frac{P(Y = 1|X = x)}{P(Y = 0|X = x)}) = 0$$$$

where $X = (X_1, X_2, \cdots ,X_k)$ and $x = (x_1, x_2, \cdots ,x_k)$

Using Bayes' rule and multinomial distribution, the decision boundary could be written as:

$$\sum_{i=1}^{k} x_i log(\frac{p_i}{p_i^{'}}) + log(\frac{P(Y=1)}{P(Y=0)}) = 0$$

where $p_i$ and $p_i^{'}$ are probabilities of the occurrence of ith feature in class $y = 1$ and $y = 0$ respectively.

So, we've got a linear form ($w^Tx + b = 0$) for our decision boundary.

But is this true for the entire family of Naive Bayes classifiers?

I suspect it would depend on the the nature of the probability distributions you choose for the input features.

• I was thinking about this recently. I wrote a related answer earlier (on naive bayes and discriminant analysis methods) -- stats.stackexchange.com/a/297877/1704 -- but I didn't look into whether all the "Discriminant Analysis" methods were linear. I know that QDA is not linear. And, as you mentioned, the standard discrete/multinomial NB is linear). Need to think about the others. A quick experiement had GNB (gaussian NB) non-linear, but I'm not convinced I didn't make a mistake. Commented Mar 27, 2018 at 12:28