So, as I am reading about Bernoulli distribution and text classification, I want to understand how Bernoulli uses TfIdf features? Since TfIdf values are within [0-1) but Multivariate Bernoulli assumes that the features are 0/1. So, how does it work?
I also found this tutorial page on scikit-learn for text classification in which the train and test features are extracted as below:
vectorizer = TfidfVectorizer(sublinear_tf=True, max_df=0.5, stop_words='english') X_train = vectorizer.fit_transform(data_train.data) X_test = vectorizer.transform(data_test.data)
and then Bernoulli distribution is applied:
clf = BernoulliNB(alpha=.01) clf.fit(X_train, y_train) pred = clf.predict(X_test)