Bayes and Naive Bayes code implementations

Question

I know that Bayes classifier assigns the new data point $\pmb{x}$ to the class $\omega_j, \ j=1,\dots,M$, when

$p(\omega_j \mid \pmb{x}) = \max_{q=1,\dots,M}p(\omega_q \mid \pmb{x})$,

where

$p(\omega_j\mid \pmb{x}) = \frac{p(\pmb{x}\mid \omega_j)p(\omega_j)}{p(\pmb{x})} = \frac{p(\pmb{x}\mid \omega_j)p(\omega_j)}{\sum_j p(\pmb{x} \mid \omega_j)p(\omega_j)}$.

The difference from the Naive Bayes classifier is that Naive Bayes assumes statistical independent features,

$p(\pmb{x}|\omega_j) = \prod_{k=1}^{l}p_k(x_k|\omega_j), \ \ \ \ \ j=1,\dots,M$

where $l$ is the number of features.

Why there are only Naive Bayes classifier implementations and there are not the full Bayes ones?

Answer 1

Specifically, Naive Bayes assumes conditional independence of features (conditioned on classes), not statistical independence. And, informally speaking, despite strong assumptions, it works pretty ok in practice. An obvious reason of using naive assumption is that the number of parameters needed in the model is linear with increasing number of features. Imagine that $p(\mathbf{x}|w_i)$ is Gaussian and you need the covariance matrix to characterize the Bayes classifier, and $\mathbf{x}$ is high dimensional, like $R^{10000}$, like in spam classification problem, e.g. the size of the vocabulary set. Then, the covariance matrix would have $10^8$ entries to be learnt. This isn't scalable. Or using non-parametric methods (like KDE) for joint PDF estimation would need immense amount of data.