Logic of Sklearn Bernoulli Naive Bayes Classifier when the the predictors are not even binary?

Question

I know the mathematics behind the Naive Baye's Bernoulli Classifier Algorithm and it is used to calculate the probabilistic results. As we know the Bernoulli Naive Bayes Classifier uses binary predictors(Features). The thing I am not getting is sklearn.naive_bayes.BernoulliNB is giving the result if the predictors are not even binary. The example is taken from the sklearn.naive_bayes.BernoulliNBdocumenatation.

• Part(1) :
  I am using binary predictors and I can derive these results by solving the things manually on paper. So know doubt in it.
• Part(2):
  Predictors are no more binary and the example is given on sklearn.naive_bayes.BernoulliNB documenatation. How the Naive Bayes Bernoulli Classifier is implented on it and whats the hidden logic ?

Part 1

import pandas as pd
import numpy as np
from sklearn.naive_bayes import BernoulliNB

# binary predictors(Features) for Bernoulli Naive Bayes Classifier
binary_predictors = np.random.randint(2, size = (6,5))
labels = np.random.choice(a=[False, True], size=(6))

# dataframe for pretty look
df = pd.DataFrame(binary_predictors, columns=["X1", "X2", "X3", "X4", "X5"])
df["Labels"] = labels

Output:

X1  X2  X3  X4  X5  Labels
1   0   1   0   0   True
1   0   0   0   1   True
0   0   0   1   1   False
0   1   1   0   1   False
0   0   1   1   0   False
0   0   1   1   1   True

# modeling
model = BernoulliNB()
model.fit(boolean_predictors, labels)
print(f"The prediction for [0,0,1,1,1] is : {model.predict(np.array([[0,0,1,1,1]]))}")

Output:

The prediction for [0,0,1,1,1] is : [ True]

Part 2

import numpy as np
rng = np.random.RandomState(1)
X = rng.randint(5, size=(6, 100))
Y = np.array([1, 2, 3, 4, 4, 5])
from sklearn.naive_bayes import BernoulliNB
clf = BernoulliNB()
clf.fit(X, Y)

print(clf.predict(X[2:3]))

Output:

array([3])

Just putting the first 10 features and we can see the features are not binary here .

0   1   2   3   4   5   6   7   8   9   10
3   4   0   1   3   0   0   1   4   4   1
1   0   2   4   4   0   4   1   4   1   0
2   4   4   0   3   3   0   3   1   0   2
2   2   3   1   4   0   0   3   2   4   1
0   4   0   3   2   4   3   2   4   2   4
3   3   3   3   0   2   3   1   3   2   3

Could someone help me to understand the logic how is it calcuated even though the predictors are not binary. I can calculate the result by using different things: Multimonial Nave Bayes and Categorical Nave Bayes Classifier. The only thing is how it has impleneted Bernoulli here?

Answer 1

Under the hood, BernoulliNB binarizes the features based on a numeric threshold (default 0.0) given by self.binarize (passed into the constructor), so the non-zero features would be treated as identical. So, as you mention, BernoulliNB is indeed using binary predictors even if your data is not binarized initially. The binarization operation is performed near the beginning of the .fit() method.

The code in question: https://github.com/scikit-learn/scikit-learn/blob/2beed5584/sklearn/naive_bayes.py#L919 -- you can look at the binarize parameter in the class docstring for an explanation.

Ultimately, this turns out to be a code design issue rather than a statistics issue. So your understanding is correct.