Stack Exchange Network

Stack Exchange network consists of 174 Q&A communities including Stack Overflow, the largest, most trusted online community for developers to learn, share their knowledge, and build their careers.

Visit Stack Exchange

A naive Bayes classifier is a simple probabilistic classifier based on applying Bayes' theorem with strong independence assumptions. A more descriptive term for the underlying probability model would be "independent feature model".

6
votes
0answers
1k views

How can I get feature importance for Gaussian Naive Bayes classifier?

I have a dataset consisting of 4 classes and around 200 features. I have implemented a Gaussian Naive Bayes classifier. I want now calculate the importance of each feature for each pair of classes ...
6
votes
0answers
590 views

Computation of log-likelihood in semi-supervised naive bayes

I have the following 2 questions about log-likelihood computation in semi-supervised Naive Bayes. I have read on several documents online that, in every EM iteration of the semi-supervised Naive ...
5
votes
0answers
1k views

Combining multiple classifiers

I am trying to do a binary classification of text articles into {relevant, non-relevant}. The text articles have following features: [[article text, ...
4
votes
0answers
735 views

How to use LDA to predict topic proportion for new document?

I'm interested to learn how I can use a trained LDA (Latent Dirichlet Allocation) model to make predictions on the topic proportion of a new, unseen document using Naive Bayes. Let $z \in \{1, 2, ......
4
votes
0answers
2k views

Is the Naive Bayes family of classifiers linear?

There are a lot of places where you'll see the proof that Naive Bayes classifiers are linear, like this and this. But they always assume a special case of the family of Naive Bayes classifiers which ...
3
votes
0answers
315 views

How to use KL-divergence in naive bayes classifier to weight features?

I have a dataset consisting of 4 classes. I have implemented the Gaussian Naive Classifier (in Matlab). In the training phase I calculate the mean and variance for each feature and each class as well ...
3
votes
0answers
1k views

Naive Bayes and text classification: which probability model and vectorizer combination makes sense?

I am wondering which combinations of Naive models can be paired with different vectorizing methods so that it makes sense. Let's say we have a simple binary spam-classification task. Multinomial ...
3
votes
0answers
286 views

How to model a network analysis problem

I have a weighted graph in which the nodes represent users and weighted undirected edges represent the tie between a pairs of users. For a piece of content $c$, and a node $A$ in the graph, given that ...
3
votes
0answers
2k views

R package for multinomial naive Bayes text classification?

I am looking for a multinomial naive Bayes text classification package in R that accepts a term document matrix (from tm) as input for training and classifies new text based on that. I have about 50....
3
votes
0answers
3k views

Adjusting the classification threshold of Naive Bayes

I've been involved in a machine learning project recently and am now in the process of writing the project up for a paper submission. We used the naive bayes classifier on the project and developed a ...
2
votes
0answers
91 views

Using Naive Bayes classifier for unsupervised learning

I was going through this article to learn about how the EM algorithm can be used to use the Naive Bayes algorithm for unsupervised learning. Suppose we have the following data without labels: 1 0 1 1 ...
2
votes
0answers
22 views

How to improve Naive Bayes?

I am solving a problem that address this question "What are the Actions that lead to high or low score?" I have the following Data that consist of text and score , I want to derive the words or ...
2
votes
0answers
1k views

Classification on highly skewed dataset

I have two classes A and B. 98% of the data belongs to class A and 2% of it belongs to class B. Size of the entire dataset is about 2000. I am interested in correctly classifying all the data points ...
2
votes
0answers
566 views

How to use TFIDF-vectors with Multinomial Naive-Bayes?

Say we have used the TFIDF transform to encode documents into continuous-valued features. How would we now use this as input to a Naive Bayes classifier? Bernoulli naive-bayes is out, because our ...
2
votes
0answers
270 views

Gaussian Naive Bayes sensitive to feature scaling

I'm using a GNB algorithm. As to my knowledge it should be insensitive to feature scaling. However, when I standardize (z-score) or normalize (min-max scaling) those of my features that have a very ...
2
votes
0answers
435 views

Confidence interval for naive bayes estimation

I have some simple model with 2 independent input variables A,B . I want to calculate some probability. $$ P(X|a,b) = P(a,b|X) P(X) / P(a,b) \\ P(X|a,b) = P(a|X) P(b|X) * P(X) / ( P(a) * ...
2
votes
0answers
371 views

Can naive Bayes model this type of (approx. circular) decision boundary?

In a recent exam on machine learning I came across the following question: "Which of the following techniques can model the decision boundary depicted in the figure? (check all that apply)" See my ...
2
votes
0answers
53 views

How to compute the likelihood when uncertain about underlying distribution

In a Bayesian approach, when I have no previous idea about the likelihood of a given event, can I simply gather a lot of data and use that distribution as my likelihood (1) and as I gather more and ...
2
votes
0answers
353 views

Is the dataset Normally distributed?

So based from this link http://www.simafore.com/blog/bid/107702/2-ways-of-using-Naive-Bayes-classification-for-numeric-attributes I began to realize it might be a good idea to compute the pdf of ...
2
votes
0answers
1k views

Is the Laplace/Lidstone smoothing parameter (talking about Multinomial/Bernoulli Naive Bayes) related to the particular structure of the dataset?

I'm working with Multinomial and Bernoulli Naive Bayes implementation of scikit-learn (python) for text classification. I'm using the 20_newsgroups dataset. From the scikit documentation we have: <...
2
votes
0answers
2k views

How to use RFECV for feature selection and cross validation

I am still very new to machine learning and trying to figure things out myself. I am using SciKit learn and have a data set of tweets with around 20,000 features (n_features=20,000). So far I achieved ...
2
votes
0answers
182 views

Naive Bayes Classifier

I've been working with trying to understand and explain how Naive Bayes classifier works with the adjusted (prior and posterior) probabilities, and wanted to show my example to ensure I'm executing it ...
2
votes
0answers
438 views

Naive Bayes - Good for Binary Data?

I have 92 observations with 92 variables. Every observation is a binary outcome (0=no, 1=yes), indicating if that observation co-occurs with a given feature in the feature set. I have 18 classes which ...
2
votes
0answers
109 views

Impact of conjugate priors on mutual information for Naive Bayes

I am currently thinking about the following problem. Suppose you have a simple Naive Bayes model for binary classification based on binary random variables. For example, suppose you want to predict ...
2
votes
0answers
197 views

Word probabilities in a Naive Bayes filter

While implementing a Naive Bayes filter, I stumbled across a problem with the calculation of the conditional probabilities $p(w|c)$ of a word $w \in \mathcal{W}$ given a class $c \in \mathcal{C}$. ...
2
votes
0answers
1k views

Parameter Estimation for Naive Bayes - Maximum a posteriori and Maximum Likelihood

I am wondering if I understand those terms correctly. To summarize my thoughts: In naive Bayes, our decision rule is basically the Maximum a posteriori (MAP) estimate of our hypothesis. We assign an ...
2
votes
0answers
77 views

Boruta test and naive bayes classification

I am currently using Boruta to test which feature is the most important to be used in my model development. For example, I have 3 features(X,Y,Z).Boruta test give the highest importance is Z. However ...
2
votes
0answers
107 views

Obtaining and sampling from the posterior predictive of a naive Bayes classifier

I have trained a naive Bayes classifier with on a dataset with a dichotomous outcome and multinomial attributes (predictors). I managed to get a Maximum a posteriori (MAP) estimate which is good ...
2
votes
0answers
830 views

Naive Bayes Classifier - measure accuracy after training

I have built a prototype Naïve Bayes Classifier in an Excel spreadsheet. My data is a transaction (an order) with 13 parameters. This translates directly to a feature vector of (feature_1, feature_2, …...
2
votes
0answers
1k views

Predicting Naive Bayes model in R on a test data with a single record

I built a naivebayes model using the Housevotes84 data(discrete data) in mlbench package- model <- naiveBayes(Class~., data=HouseVotes84) I took one record ...
2
votes
0answers
222 views

Maximizing incomplete likelihood

Given the conditional distribution $p(x|y)$ and the prior of the hidden variables $p(y|\theta)$ with unknown hyper-parameter $\theta$. Now we have observed i.i.d. samples of $x$. Besides the Bayes ...
2
votes
0answers
208 views

Naive Bayes with invalid independence assumption

I'm trying to understand the effects of adding non-conditionally independent features to a naive Bayes classifier. Let's say I have the features vector $X = [x_1,x_2,x_3,x_4]$ and that for each value ...
2
votes
0answers
46 views

Naive Bayes without model

I have the following scenario: I have two "states". I measure variables $n$ that are affected by the state. A state of 0 is the background state and in this case I expect each variable $n_i$ to be ...
2
votes
0answers
197 views

Computation of Maximization probabilities of the EM algorithm

I have implemented a semi-supervised Naive Bayes that makes use of the EM algorithm to iteratively learn from unlabeled data in a text classification problem, but I am not sure of the processing done ...
2
votes
0answers
231 views

How to choose the distribution and parameters for continuous probability density functions in naive Bayes using maximum likelihood?

Let's assume I want to train a binary naive Bayes classifier, with classes $y_0, y_1$ and $n$-dimensional data. For this one needs to calculate the conditional probabilities $P(x_i | y_j) $ for all ...
2
votes
0answers
727 views

Regarding kernel-based naive Bayesian classifier

Are there any good references for kernel-based Naive Bayesian classifier?
1
vote
0answers
40 views

Exploit grouped features for a classification problem

I've got a dataset of 100.000 labelled vectors, each one with a set of 100 binary features: $$F = \{ f_{1}, f_{2}, ..., f_{100} \}.$$ I am able to build a Bernoulli Naive Bayes classifier for ...
1
vote
0answers
17 views

Review 5-stars Multiclassification Model

So I'm super new to DataScience World. And I'm Trying to do a TextMining Work. My goal is by reading user's reviews to predict their rating to a tech product. Problem? Multiclassification model with ...
1
vote
0answers
74 views

multinomial Naive Bayes: how to calculate likelihood and posterior?

For a multinomial Naive Bayes model for $C$ classes and $D$ features, assuming $\theta \in \mathcal{R}^{C x D}$ is the matrix whose $\theta_{cj} $ element corresponds to $Prob(x_j = 1 | y= c)$, and $\...
1
vote
0answers
16 views

Naive Bayes: Understanding the Entropy equation

I am trying to understand the entropy equation: -p1*log2(p1) - p2*log2(p2) - pn*log2(pn) Specifically why do we multiply each log by the probability? In the ...
1
vote
0answers
118 views

Does smoothing make any sense for Bernoulli Naive Bayes?

I'm trying to understand how the hyperparameters for the Bernoulli Naive Bayes in sklearn work for doing Randomized Search CV. If I use smoothing, and set ...
1
vote
0answers
55 views

use test data set after 10 Cross-validation

I applied 10 Cross-validation but I am a bit confused, I am not sure what is a correct way. 1- Should I apply 10 Cross-validation on all dataset, divide it into 10 folds and sum all the 10 matrices ...
1
vote
0answers
146 views

What attributes to apply laplace smoothing in naive bayes classifier?

I am reading naive Bayes classifier from the book "Data mining practical machine learning tools and techniques". The example of naive Bayes is given using the below dataset. As (Outlook=Overcast | ...
1
vote
0answers
80 views

gaussian naiive bayes with missing values?

I've got some data that I one-hot encode, A is numbers B is categories, C (the thing that I predict) is categories, so... ...
1
vote
0answers
611 views

Multinomial vs Gaussian Naive Bayes Performance in Scikit Learn for Word Embedding Features

I am running some experiments using word embedding features with Multinomial and Gaussian Naive Bayes in Scikit learn. As far as I know, Multinomial Naive Bayes works on features with distribution ...
1
vote
0answers
92 views

Adding features decreases performance of Naive Bayes Classifier

A bit of background. I'm new to predictive modeling but am developing a model for predicting A&E patients who will admit to Hospital. I have available 12 features, 2 continuous, the rest ...
1
vote
0answers
790 views

How do we interpret the output of the Naive Bayes' classifier in e1071 package?

I am executing the code given at Sentiment analysis with machine learning in R. While executing this code, I am trying to examine the contents of the object 'classifier'. The conditional probability ...
1
vote
0answers
258 views

Online learning algorithm not depending on the order of the data?

Are there any online learning algorithm that do not depend on the order of arrival of the data ? I am looking for algorithms that, given a sequence of data $(x_i,y_i)_{i\in[1,n]}$ : Will produce ...
1
vote
0answers
505 views

Naive Bayes vs. logistic regression

I'm working with credit scoring models. Here's what I know: Let Y be the binary outcome variable, $Y \in \{0,1\}$ where $Y = 1$ is the outcome of default and $X = (X_{1},...,X_{m})$ be the random ...
1
vote
0answers
49 views

Can inferencing come from incomplete rule sets?

I have some data for medical diagnosis, consisting of some rules about relationship of diseases and their symptoms, for example disease D1 frequently has symptom S1 ...