Cross Validated is a question and answer site for people interested in statistics, machine learning, data analysis, data mining, and data visualization. It only takes a minute to sign up.
Sign up to join this community
According to this article
Because of the class independence assumption, naive Bayes classifiers can quickly learn to use high dimensional features with limited training data compared to more sophisticated methods. This can be useful in situations where the dataset is small compared to the number of features, such as images or texts.
Why does Naive Bayes work well when the number of features >> sample size compared to more sophisticated ML algorithms?
What the author is getting at is that Naive Bayes implicitly treats all features as being independent of one another, and therefore the sorts of curse-of-dimensionality problems which typically rear their head when dealing with high-dimensional data do not apply.
If your data has $k$ dimensions, then a fully general ML algorithm which attempts to learn all possible correlations between these features has to deal with $2^k$ possible feature interactions, and therefore needs on the order of $2^k$ many data points to be performant. However because Naive Bayes assumes independence between features, it only needs on the order of $k$ many data points, exponentially fewer.
However this comes at the cost of only being able to capture much simpler mappings between the input variables and the output class, and as such Naive Bayes could never compete with something like a large neural network trained on a large dataset when it comes to tasks like image recognition, although it might perform better on very small datasets.
