# Is the Laplace/Lidstone smoothing parameter (talking about Multinomial/Bernoulli Naive Bayes) related to the particular structure of the dataset?

I'm working with Multinomial and Bernoulli Naive Bayes implementation of scikit-learn (python) for text classification. I'm using the 20_newsgroups dataset. From the scikit documentation we have:

class sklearn.naive_bayes.MultinomialNB(alpha=1.0, fit_prior=True, class_prior=None)


and

class sklearn.naive_bayes.BernoulliNB(alpha=1.0, binarize=0.0, fit_prior=True, class_prior=None)


so we need to give a float value to alpha, which represents the smoothing parameter (as scikit says: "setting alpha = 1 is called Laplace smoothing, while alpha < 1 is called Lidstone smoothing").

Now, I noticed the Multinomial version works pretty well with a Laplace smoothing (alpha=1.0), while the Bernoulli one seems pretty bad with such a value. I tried to give different values to the Bernoulli alpha and noticed Bernoulli had acceptable accuracy if alpha was something like "0.01", "0.03", "0.001"...ecc. So I thought Bernoulli Naive Bayes "prefers" a Lidstone smoothing. Now, my question is: is it always like that (alpha=1.0 with Multinomial and alpha=0.01 with Bernoulli) or is the value for the smoothing parameter related to the particular structure of the dataset we're using?

• Clearly the smoothing parameter is a hyper-parameter. And, you tune it to its best value for the task at hand. So, yes it is related to the dataset Apr 3 '19 at 18:34

P("positive" | F=1)  =  P(F=1 | "positive") * P("positive") / P(F=1)