Questions tagged [naive-bayes]

A naive Bayes classifier is a simple probabilistic classifier based on applying Bayes' theorem with strong independence assumptions. A more descriptive term for the underlying probability model would be "independent feature model".

Filter by
Sorted by
Tagged with
50
votes
3answers
12k views

Understanding Naive Bayes

From StatSoft, Inc. (2013), Electronic Statistics Textbook, "Naive Bayes Classifier": To demonstrate the concept of Naïve Bayes Classification, consider the example displayed in the ...
41
votes
3answers
88k views

Difference between naive Bayes & multinomial naive Bayes

I've dealt with Naive Bayes classifier before. I've been reading about Multinomial Naive Bayes lately. Also Posterior Probability = (Prior * Likelihood)/(Evidence). The only prime difference (while ...
40
votes
3answers
36k views

Why do naive Bayesian classifiers perform so well?

Naive Bayes classifiers are a popular choice for classification problems. There are many reasons for this, including: "Zeitgeist" - widespread awareness after the success of spam filters about ten ...
35
votes
3answers
34k views

How is Naive Bayes a Linear Classifier?

I've seen the other thread here but I don't think the answer satisfied the actual question. What I have continually read is that Naive Bayes is a linear classifier (ex: here) (such that it draws a ...
33
votes
8answers
93k views

In Naive Bayes, why bother with Laplace smoothing when we have unknown words in the test set?

I was reading over Naive Bayes Classification today. I read, under the heading of Parameter Estimation with add 1 smoothing: Let $c$ refer to a class (such as Positive or Negative), and let $w$ ...
29
votes
3answers
42k views

What algorithms need feature scaling, beside from SVM?

I am working with many algorithms: RandomForest, DecisionTrees, NaiveBayes, SVM (kernel=linear and rbf), KNN, LDA and XGBoost. All of them were pretty fast except for SVM. That is when I got to know ...
22
votes
3answers
32k views

Machine Learning to Predict Class Probabilities

I am looking for classifiers that output probabilties that examples belong to one of two classes. I know of logistic regression and naive Bayes, but can you tell me of others that work in a similar ...
21
votes
2answers
20k views

How does Naive Bayes work with continuous variables?

To my (very basic) understanding, Naive Bayes estimates probabilities based on the class frequencies of each feature in the training data. But how does it calculate the frequency of continuous ...
17
votes
1answer
30k views

When does Naive Bayes perform better than SVM?

In a small text classification problem I was looking at, Naive Bayes has been exhibiting a performance similar to or greater than an SVM and I was very confused. I was wondering what factors decide ...
16
votes
3answers
4k views

In Kneser-Ney smoothing, how are unseen words handled?

From what I have seen, the (second-order) Kneser-Ney smoothing formula is in some way or another given as $ \begin{align} P^2_{KN}(w_n|w_{n-1}) &= \frac{\max \left\{ C\left(w_{n-1}, w_n\right) - ...
15
votes
3answers
17k views

Example of how the log-sum-exp trick works in Naive Bayes

I have read about the log-sum-exp trick in many places (e.g. here, and here) but have never seen an example of how it is applied specifically to the Naive Bayes classifier (e.g. with discrete features ...
15
votes
2answers
32k views

Increasing number of features results in accuracy drop but prec/recall increase

I am new to Machine Learning. At the moment I am using a Naive Bayes (NB) classifier to classify small texts in 3 classes as positive, negative or neutral, using NLTK and python. After conducting ...
15
votes
5answers
4k views

How to do one-class text classification?

I have to deal with a text classification problem. A web crawler crawls webpages of a certain domain and for each webpage I want to find out whether it belongs to only one specific class or not. That ...
15
votes
3answers
4k views

Why does nobody use the Bayesian multinomial Naive Bayes classifier?

So in (unsupervised) text modeling, Latent Dirichlet Allocation (LDA) is a Bayesian version of Probabilistic Latent Semantic Analysis (PLSA). Essentially, LDA = PLSA + Dirichlet prior over its ...
15
votes
1answer
3k views

How can I get feature importance for Gaussian Naive Bayes classifier?

I have a dataset consisting of 4 classes and around 200 features. I have implemented a Gaussian Naive Bayes classifier. I want now calculate the importance of each feature for each pair of classes ...
14
votes
1answer
8k views

Why is the naive bayes classifier optimal for 0-1 loss?

The Naive Bayes classifier is the classifier which assigns items $x$ to a class $C$ based on the maximizing the posterior $P(C|x)$ for class-membership, and assumes that the features of the items are ...
13
votes
4answers
8k views

Can logistic regression's predicted probability be interpreted as the confidence in the classification

Can we interpret posterior probability obtained from a classifier that outputs a predicted class value and a probability (for example, logistic regression or Naive Bayes) as some kind of a confidence ...
13
votes
3answers
4k views

Naive Bayes feature probabilities: should I double count words?

I'm prototyping my own Naive Bayes bag o' words model, and I had a question about calculating the feature probabilities. Let's say I've got two classes, I'll just use spam and not-spam since that's ...
12
votes
4answers
16k views

Calculating the error of Bayes classifier analytically

If two classes $w_1$ and $w_2$ have normal distribution with known parameters ($M_1$, $M_2$ as their means and $\Sigma_1$,$\Sigma_2$ are their covariances) how we can calculate error of the Bayes ...
12
votes
2answers
10k views

Akinator.com and Naive Bayes classifier

Context: I am a programmer with some (half-forgotten) experience in statistics from uni courses. Recently I stumbled upon http://akinator.com and spent some time trying to make it fail. And who wasn't?...
11
votes
2answers
1k views

What kinds of things can I predict with a naive Bayesian classifier?

I'm a beginner to statistics (taken only one college course), but I have a background in programming. I just started playing with a Bayesian classifier library for Ruby and I am looking for ideas ...
11
votes
3answers
10k views

“Good” classifier destroyed my Precision-Recall curve. What happened?

I'm working with imbalanced data, where there are about 40 class=0 cases for every class=1. I can reasonably discriminate between the classes using individual features, and training a naive Bayes and ...
10
votes
2answers
14k views

Naive Bayes on continuous variables

Please allow me to ask a basic question. I understand the mechanics of Naive Bayes for discrete variables, and can redo the calculations "by hand". (code of HouseVotes84 all the way per below). ...
10
votes
2answers
302 views

Confusion about 1- vs 2-tailed tests for feature selection by hypothesis testing

Suppose $x_i\ (i=1,2,...,N)$ be attribute values for $N$ samples from class $W_1$ with mean $\mu_1 $ and $y_i\ (i=1,2,...,N)$ be attribute values for $N$ samples from class $W_2$ with mean $\mu_2 $. ...
9
votes
1answer
27k views

How to use Naive Bayes for multi class problems?

I know how Naive Bayes work for classifying binary problems. I just need to know what are the standard way to apply NB on multi-class classification problems. Any idea please?
9
votes
3answers
14k views

Naive Bayes: Continuous and Categorical Predictors

It's my understanding that most types of common classifiers (Support Vector Machine, for example) can take a mixture of categorical and continuous predictors. However, this doesn't seem to be true ...
9
votes
1answer
2k views

Is Naive Bayes becoming more popular? Why?

This is the google trends result obtained for "Naive Bayes" phrase from Jan 2004-April 2017 (link). According to this figure, the search ratio for "Naive Bayes" in April 2017 is about %25 higher than ...
9
votes
2answers
6k views

Origin of the Naïve Bayes classifier?

I've looking around Google Scholar for the earliest mention of this particular classifier and have not had much luck finding a definitive source. I've seen some sources cite as late as the 1980s and ...
9
votes
1answer
161 views

Algebraic classifiers, more information?

I have read Algebraic classifiers: a generic approach to fast cross-validation, online training, and parallel training and was amazed by the performance of the derived algorithms. However, it seems ...
8
votes
4answers
3k views

Is a Bayesian Classifier a good approach for text with numerical meta-data?

I'm trying to come up with an approach for detecting scam adverts on my website. I think the problem has a lot in common with detecting spam email (for which a naive Bayesian classifier is a common ...
8
votes
3answers
4k views

How are classifications merged in an ensemble classifier?

How does an ensemble classifier merge the predictions of its constituent classifiers? I'm having difficulty finding a clear description. In some code examples I've found, the ensemble just averages ...
8
votes
1answer
12k views

How to use log probabilities for Gaussian Naive Bayes? [duplicate]

I'm currently implementing a Gaussian Naive Bayes classifier. Of course if I'm doing classification by $$ \text{argmax}_{C_i} P(C_i)P(D|C_i), $$ then the probabilities can get very small. So I want ...
8
votes
2answers
4k views

Naive-Bayes classifier for unequal groups

I'm using naive bayes classifier to classify between two groups of data. One group of the data is much larger than the other (above 4 times). I'm using the prior probability of each group in the ...
8
votes
1answer
3k views

Decision threshold for a 3-class Naive Bayes ROC curve

I have some doubts regarding how a ROC curve for a 3-class classifier (Naive Bayes) can be built. Basically, given some test data, the classifier outputs the probabilities for each of the 3 possible ...
8
votes
1answer
3k views

Hidden Markov Model and Naive Bayes similarity

I understand Naive Bayes classifier and already have made a few implementaions. What I dont understand is, considering that I have a training dataset with all the X observations and Y states, what ...
7
votes
1answer
5k views

Naive Bayes classifier gives a probability greater than 1

I'm trying to understand an example regarding how to use a Naive Bayes classifier in spam filtering based on this link. The article picks two "bad" words that they figure are in spam a lot and then ...
7
votes
1answer
2k views

When is a Naive Bayes Model not Bayesian?

I've been using Bayesian inference for a while and as far as I could tell, Naive Bayes was "Bayesian" since it has a prior and a posterior and follows the Bayes rule. I just read a topic on "...
7
votes
3answers
25k views

The difference between the Bayes Classifier and The Naive Bayes Classifier?

I'm trying to find the connection between both classifiers. In NBC we assume that all the features are independent of each other so we can calculate the posterior probability easier. I assume Bayes ...
7
votes
1answer
7k views

Machine learning techniques for spam detection, and in general for text classification

I am going to configure a system for spam detection. What I have is a dataset of labeled (spam/not-spam) strings containing, mostly, sentences. I have a background in machine learning techniques, but ...
7
votes
3answers
2k views

Naive Bayes classification for “That's what she said” problem

I became interested in doing this in C# for my own amusement after reading the following papers: http://www.cs.washington.edu/homes/brun/pubs/pubs/Kiddon11.pdf I also took a look at http://www.cs....
7
votes
2answers
18k views

Naive Bayes with unbalanced classes

As a part of a project for the university is should train a Naive Bayes classifier to classify question and answers in three different categories, the task should be easy since that the 3 classes are ...
7
votes
4answers
3k views

What is the best form (Gaussian, Multinomial) of Naive Bayes to use with categorical (one-hot encoded) features?

I've been asked to use the Naive Bayes classifier to classify a couple of samples. My dataset had categorical features so I had to first encode them using a one-hot encoder, but then I was at a loss ...
6
votes
2answers
4k views

what is the “learning” that takes place in Naive Bayes?

As I recall algorithms like nearest neighbor don't build a model based on training data and then apply that model to test data. It just takes each new instance and compares it to all the data to find ...
6
votes
2answers
982 views

Logistic Regression/Naive Bayes with highly correlated data

Background: We work with data from sports event, more accurately with data about the spectators of sports events: how many people are being violent, what kind of event is this, etc. We have quite a ...
6
votes
2answers
1k views

What if a numerator term is zero in Naive Bayes?

I'm trying to predict the probability that a user will visit a particular website based on several factors (day of the week, time since last visit, etc). My question is what to do if one of the ...
6
votes
2answers
186 views

Can $p(Y|a,b)$ ever be equal to $p(Y|a) \cdot p(Y|b)$?

This strikes me as a simple question, but in re-visiting how the Naive Classifier works I started wondering if there is any probabilistic model that under certain independency assumptions obtains: $p(...
6
votes
2answers
1k views

Overfit by removing misclassified objects?

Actually this question may be simple for you, but I need to learn the correct answer. If I remove misclassified instances from data set with Naive Bayes (it gives minimum FP rate) and then train ...
6
votes
4answers
4k views

Does PCA followed by LDA make sense, when there is more data available for PCA than for LDA?

This is a question about classification. I am a neuroscience student with little experience of classification methods and I'd be grateful for any advice about the best way to implement a linear ...
6
votes
3answers
4k views

Predicting continuous variables from text features

I want to predict a continuous variable from text features. Lets say I have some student essays and I want to predict their quality, as measured by a human grader, using text features (mostly words ...
6
votes
2answers
3k views

PDFs and probability in naive Bayes classification

I have seen a few times the technique of using the Gaussian PDF for continuous features in Naive Bayes. here and here. Illustrated in the first link: How is this possible? I always learnt that the ...

1
2 3 4 5
12