# Questions tagged [naive-bayes]

A naive Bayes classifier is a simple probabilistic classifier based on applying Bayes' theorem with strong independence assumptions. A more descriptive term for the underlying probability model would be "independent feature model".

566 questions
Filter by
Sorted by
Tagged with
1answer
74 views

### Bayesian networks and weird probabilities

I have to solve the following problem: Suppose we have a bayesian net in which we have the following variables: R, PA and PR Let: P(R) = 0.1, P(PA) = 0.5, P(PR|R, PA) = 0.6, P(PR|¬R, PA) = 0.4, P(...
1answer
2k views

### Does Naive Bayes assume normality?

I came across this paper about Naive Bayes that states [Naive Bayes] is based on another common simplifying assumption: the values of numeric attributes are normally distributed within each class. ...
1answer
3k views

### Naive Bayes Derivation

I was going over the derivation of Naive Bayes, and the following 3 lines were given: Suppose $X = \left < X_1, X_2 \right>$ \begin{align} P(X|Y) &= P(X_1, X_2 | Y) \\[2pt] &= ...
1answer
4k views

### Any way of getting vector of probabilities for each response with Naive Bayes in R?

I'm currently using naiveBayes from {e1071}. My response is simply a prediction based on my independent variables. Is there a ...
4answers
4k views

### What is the best form (Gaussian, Multinomial) of Naive Bayes to use with categorical (one-hot encoded) features?

I've been asked to use the Naive Bayes classifier to classify a couple of samples. My dataset had categorical features so I had to first encode them using a one-hot encoder, but then I was at a loss ...
2answers
4k views

### Using the Naive Bayes classifier in R with continuous variables

I am trying to predict a categorical variable (type of job, there are three classes) using a dataset that mainly consists of continuous variables (like years of education, salary,etc), using the Naive ...
1answer
3k views

### Hidden Markov Model and Naive Bayes similarity

I understand Naive Bayes classifier and already have made a few implementaions. What I dont understand is, considering that I have a training dataset with all the X observations and Y states, what ...
1answer
83 views

### Differences between categorical classification algorithms

Given data where the class is categorical (finite and discrete), there are multiple ways to come up with a classifier. One could use multinomial logistic regression, or support vector clustering (...
3answers
27k views

### The difference between the Bayes Classifier and The Naive Bayes Classifier?

I'm trying to find the connection between both classifiers. In NBC we assume that all the features are independent of each other so we can calculate the posterior probability easier. I assume Bayes ...
1answer
3k views

### Normalize non-normal distribution?

I have a query regarding a comment I found, which will surely shed some light. In this article: http://www.analyticsvidhya.com/blog/2015/09/naive-bayes-explained/ I found: If continuous features ...
2answers
3k views

### Is Naive Bayes suitable for large datasets with thousands of features?

I have a data set with 100 million rows and 15,000 categorical variables each with 0/1 values. My target variable is also a 0/1 binary variable. Is Naive Bayes suitable in terms of computational ...
1answer
984 views

### true negative is 0% whereas true positive is 100% correctly classified

I used Naive Bayes from Spark's MlLib to train a model and test it on the data (in the form of an RDD). The results were confusing. the data and results are as follows: The problem is a binary ...
1answer
505 views

### Naive Bayes: Intuition behind the Evidence

I understand how to use NB and have used it often. However, I am trying to understand how the two different ways I use to calculate the evidence (P(E)) result in ...
0answers
55 views

### How to compute probability [duplicate]

I have a dataset consisting of 4000 observation from each 324 continuous features are extracted. Each observation has been labeled a class. Since each feature from that dataset is continuous, have I ...
0answers
443 views

### Is the dataset Normally distributed?

So based from this link http://www.simafore.com/blog/bid/107702/2-ways-of-using-Naive-Bayes-classification-for-numeric-attributes I began to realize it might be a good idea to compute the pdf of ...
0answers
130 views

### Create a model based on the distribution of data for classification purposes

I have data set which is stored as a matrix where each row is an observation (number of observations listed is 4000 ) and each column the feature extracted from that observation (number of features ...
1answer
99 views

### How was this intergral derived from Bayes' Rule in David Heckerman's Bayesian Network paper?

I am trying to follow this paper titled "A Tutorial on Learning With Bayesian Networks" by Microsoft researcher David Heckerman. In it I am unable to figure out how he got to Equation 2 from ...
1answer
2k views

### What is the meaning of laplace, eps and threshold in NaiveBayes package in R e1071 lib?

I am using NaiveBayes for text classification, I am interested on tagging a text (like a blog post). What I am finding is that normally I have results in which a tag has a probability of 0.9999 of ...
1answer
262 views

### Does Naive Bayes( library:klar) in R calculates denominator of conditional probability while giving output?

Generally, when using Naive Bayes for classification, denominator is ignored as probability is directly proportional to the numerator as denominator is same for all the classes. So, I want to know if ...
1answer
89 views

### Choosing number of samples to train a model

(On behalf of a colleague) I have performed some modelling based on a naïve Bayes classifiers model (weighted genomic risk score) and obtained reasonable ROCAUC results (used ROCR, pROC, and SDMtools ...
3answers
10k views

### “Good” classifier destroyed my Precision-Recall curve. What happened?

I'm working with imbalanced data, where there are about 40 class=0 cases for every class=1. I can reasonably discriminate between the classes using individual features, and training a naive Bayes and ...
2answers
792 views

### Hypothesis space of Naive Bayes and kNN

I am confused about the hypothesis space of those two classifiers. In the case of linear regression, it's pretty straightforward ; the possible hypothesis are equations of lines, that is, linear ...
2answers
645 views

### Highly correlated features in text mining despite mutual information criterion

I'm trying to classify documents into two classes using the Bernoulli Naive Bayes algorithm, as described here in chapter 13. I've extracted 500 tokens (out of more than 30,000) from my sample ...
1answer
6k views

### Intuitive example for MLE, MAP and Naive Bayes classifier

I am trying to understand MLE, MAP and naive Bayes classifier, but it's difficult to understand the differences without some numerical example. Can someone give simple intuitive numerical example for ...
2answers
512 views

### Text categorization using Naive Bayes: Why isn't this working?

I'm trying to implement a system for text categorization using Naive Bayes as part of a school project. I have to hand code the algorithm and have been having some issues. To make sure I understand ...
1answer
161 views

### Algebraic classifiers, more information?

I have read Algebraic classifiers: a generic approach to fast cross-validation, online training, and parallel training and was amazed by the performance of the derived algorithms. However, it seems ...
1answer
892 views

### Why does Naive Bayes use gaussian pf rather than Student's t?

Every source in the literature I could find about naive Bayes mentions using a gaussian's probability density function, using the mean and variance estimated from the data itself. This strikes me as ...
1answer
663 views

### Very poor accuracy in Naive Bayes for ancestry/surname classification

Naive Bayes has a very good reputation on the classification of surnames by ancestry (see http://www.ncbi.nlm.nih.gov/pubmed/24944286). I would like to apply a Naive Bayes classifier in R to identify ...
1answer
562 views

### How to apply a fitted Tree-augmented Naive Bayes classifier to new cases

I am running Tree Augmented Naive Bayes algorithm in R and I have got the desired network. However, unlike logistic regression, Bayesian is non-parametric i.e. I do not have any coefficients which I ...
0answers
180 views

### How do I reduce number of features without rebuilding the model?

I'm pretty new to ML/NLP thus my question maybe naive. How do I reduce number of features without rebuilding the model for Naive Bayes Classification? I'm using MALLET to build the model to classify ...
2answers
657 views

### Classifying time-series similarity - what variable should I train on?

I have ~10,000 time series, each with 65 time points. I'm interested in classifying each pair of time series as "similar" or "not similar". Here's an example of two similar (left) and not similar time ...
1answer
3k views

### Naive Bayes error with caret

I want to predict a variable with Naive Bayes. I tried it with another one from the same dataset and it worked perfect but not with the desired. The variable to predict contains values like "OL","DL" ...
1answer
894 views

### SVM classifier - can I average multiple models?

I'm performing SVM classification on a relatively large data set (~1M rows, 4 variables). I want to assign a classification score to each row, not evaluate input parameters, so following the top ...
1answer
4k views

### Is the Laplace/Lidstone smoothing parameter (talking about Multinomial/Bernoulli Naive Bayes) related to the particular structure of the dataset?

I'm working with Multinomial and Bernoulli Naive Bayes implementation of scikit-learn (python) for text classification. I'm using the 20_newsgroups dataset. From the scikit documentation we have: <...
0answers
1k views

### Why is the “training score” I get from the learning curve of Multinomial Naive Bayes so different from the training score of the Bernoulli version?

I'm comparing the learning curves of Bernoulli and Multinomial Naive Bayes using the 20_newsgroups dataset from scikit-learn for text-classification. I considered both the "training score" and the "...
1answer
15k views

### R - Plotting a ROC curve for a Naive Bayes classifier using ROCR. Not sure if I'm plotting it correctly

I have a Naive Bayes classifiers that I'm using to try to predict whether a game is going to win or lose based on historical data. The model has 25 variables in total, all of which are categorical ...
0answers
426 views

### Naïve Bayes with different distributions for each feature

I am looking at how naive Bayes works and I see that it goes over all the classes and finds the probability that maximizes: \$\log(\operatorname{Pr}[Y=y]) + \sum_{i=1}^d \log(\operatorname{Pr}[X_i=x_i|...
1answer
2k views

### Is a one class naive bayes possible?

I have a simple question - I think. I have recently read a paper: https://www.google.co.uk/url?sa=t&source=web&rct=j&url=http://www.cs.columbia.edu/~kewang/paper/DMSEC-camera.pdf&ved=...
2answers
858 views

### Predicting with cross validation

I want to predict labels via naive bayes and cross validation and measure the test accuracy. I do understand the principle of cross validation but not completely how to apply it. My question: Do I ...
1answer
2k views

### Combining multiple classifiers

I am trying to do a binary classification of text articles into {relevant, non-relevant}. The text articles have following features: [[article text, ...
1answer
42 views

### Using Clustering Coefficient to Improve Naive Bayesian Classifier

I am new at statistics and ML. Due to my lack of theoretical background I was wandering if does it make sense to combine NBC and CC. I am participating to the kaggle competition https://www.kaggle.com/...
0answers
62 views

### Is fitting hyperparameters to data in a Machine Learning model appropriate?

I have constructed a machine learning model (it is similar to Naive Bayes) within the Bayesian framework, and as such, have must select priors. In my brief exposure to Bayesian statistics, I was ...
1answer
1k views

### What is the correct spelling and capitalization of “Naive Bayes”?

I wonder which form(s) are correct amongst the following: Naive Bayes (example: Tom Mitchell's chapter on Naive Bayes) naive Bayes (example: the Wikipedia page on naive Bayes) I have also read some ...
0answers
96 views

### Probability of correct classification with optimal Bayes when increasing number of features

Consider the optimal Bayes classifier applied on a problem with N features. Let its probability of correct classification be $$P_N(corr)$$ Assume that we add an extra feature (so now we have N + 1 ...
1answer
715 views

### Document classification sample size

I'm working on a document binary classification problem where I have a decent sized corpus of about 30,000 documents (600-1000 words each). My approach is to select a sample of documents and manually ...
1answer
177 views

### How to combine probabilities of belonging to a category coming from different features?

Let us consider a problem of binary classification based on use of several nominal (categorical variables). For example, we would like to predict if a person has a car based on his/her gender, ...
0answers
1k views

### Accuracy decreased after feature selection

For my machine learning study, I tested different algorithms like SVM, SMO, Naive Bayes, Trees etc. All the algorithms resulted with low accuracy levels. In fact the highest accuracy I obtained was 46%...
1answer
3k views

### Is it wrong if I get training accuracy lower than test accuracy?

I have a dataset with 20000 instances in training, 2300 attributes. I did 10 fold CV and executed on a test set with 9000 instances with naive bayes and J48. The 10 fold CV accuracy is low compared to ...
2answers
605 views

### Naive Bayes for Spam detection

I am studying few examples of simple Naive Bayes for Spam detection. I had a question it, but I am unable to find it in any of the examples. I was wondering, what will happen if a word appears ...
1answer
3k views

### What does it mean “Disadvantage of Naive Bayes Classifier: strong feature independence”?

It is told that the most important disadvantage of Naive Bayes is that it has strong feature independence assumptions. Can someone please explain this more elaborately?