A naive Bayes classifier is a simple probabilistic classifier based on applying Bayes' theorem with strong independence assumptions. A more descriptive term for the underlying probability model would be "independent feature model".

learn more… | top users | synonyms

3
votes
1answer
22 views

Difference between classifier and estimator

Reading documentation of Naive Bayes from sklearn, I read the following: "On the flip side, although naive Bayes is known as a decent classifier, it is known to be a bad estimator, so the probability ...
1
vote
1answer
26 views

Most Informative Features with Naive Bayes

Anyone know how to calculate the most informative features where the attributes are normally distributed using Naive Bayes? My understanding, at least if you have binary attributes, is that you ...
1
vote
0answers
16 views

How to use RFECV for feature selection and cross validation

I am still very new to machine learning and trying to figure things out myself. I am using SciKit learn and have a data set of tweets with around 20,000 features (n_features=20,000). So far I achieved ...
1
vote
1answer
27 views

Naive Bayes Classifier in R with class weights

I'm searching for a Naive Bayes classifier in R where I can add a paramter for class weights. I need this, because my data is highly unbalanced. Eg.: Class1: 1000 examples Class2: 800 examples ...
0
votes
0answers
9 views

Discrete variables: Gaussian Naive Bayes or Bernoulli Naive Bayes?

I have a dataset of which features are: Hour Weekday Day Month 10 7 30 12 12 3 15 1 ... and with binary labels ...
0
votes
0answers
13 views

is it logical to train a binary classifier when one class form most of examples?

I want to train a binary classifier while only few examples in train data are "T" class. However I just used two numeric features but I think It's not logical to use any kind of classifier for this ...
2
votes
0answers
48 views

Naive Bayes Classifier

I've been working with trying to understand and explain how Naive Bayes classifier works with the adjusted (prior and posterior) probabilities, and wanted to show my example to ensure I'm executing it ...
1
vote
1answer
25 views

Fail to improve recall in classification

I have a large data set with over 700,000 examples and I tried to (binary) classify the data set with Naive Bayes and Random Forest. The task was carried out in Python and Scikit-learn data The data ...
0
votes
1answer
33 views

Multinomial Naive Bayes is not Multinomial in text classification

According to Wiki, the Multinomial Naive Bayes's conditional distribution is: $$p(\mathbf{x} \vert C=k) = \text{Multinomial}(n,\mathbf p_k) = \frac{(\sum_d x_d)!}{\prod_d x_d !} \prod_d ...
0
votes
0answers
21 views

Naive Bayes - Good for Binary Data?

I have 92 observations with 92 variables. Every observation is a binary outcome (0=no, 1=yes), indicating if that observation co-occurs with a given feature in the feature set. I have 18 classes which ...
0
votes
0answers
10 views

Getting prob of class using naive bayes

I am trying to classify input with two classes, here is the simple code for the same. dino and crypto are two classes ...
1
vote
2answers
48 views

“Non-naive” bayesian classification algorithms

Based on the problem description in this post: Relating parameters to a measured variable Based on a suggestion, I thought of studying the relationship between the parameters and a measured metric ...
0
votes
1answer
22 views

Iris dataset and a-priori probabilities

I have been playing around with two R packages for naive Bayes classification (e1071 and klaR) using the Iris dataset as an example. During the training phase, the outpur of the apriori probabilities ...
0
votes
0answers
20 views

How to use KL-divergence in naive bayes classifier to weight features?

I have a dataset consisting of 4 classes. I have implemented the Gaussian Naive Classifier (in Matlab). In the training phase I calculate the mean and variance for each feature and each class as well ...
3
votes
1answer
37 views

How to use log probabilities for Gaussian Naive Bayes?

I'm currently implementing a Gaussian Naive Bayes classifier. Of course if I'm doing classification by $$ \text{argmax}_{C_i} P(C_i)P(D|C_i), $$ then the probabilities can get very small. So I want ...
1
vote
0answers
21 views

How can I get feature importance for Gaussian Naive Bayes classifier?

I have a dataset consisting of 4 classes and around 200 features. I have implemented a Gaussian Naive Bayes classifier. I want now calculate the important of each feature for each pair of classes ...
1
vote
1answer
28 views

Does Zero Observations Problem exist for Gaussian Naive Bayes?

I'm currently implementing a Gaussian Naive Bayes classifier. With a Naive Bayes classifier the zero observation or zero probability problem can occur, see e.g. point 11 on ...
0
votes
1answer
67 views

How to use a particle filter for Bayesian inference?

I'm not very well versed in probability theory, so I'm not sure how to assess if my approach is correct. I hope this is the right place to ask. I have implemented a particle filter to get an estimate ...
0
votes
1answer
40 views

Distinguishing objective from subjective text using a Naive Bayes classifier

I am trying to built a classifier for subjective and objective text using imdb data. For objective data point I am using the movie's plot summary as input. For subjective data points I am using ...
0
votes
0answers
22 views

How to use Naive Bayes classifier to predict three different outcomes?

My training data consists of the last season results in the following format (.csv file): ...
0
votes
0answers
22 views

Help Simple Conditional Counts Example

Let's say we have a sequence $S$ \begin{align} t \quad 0 \quad 1 \quad 2 \quad 3 \quad 4 \\ S_t \quad 1 \quad 1 \quad 0 \quad 0 \quad 1 \end{align} And we want to predict $S_{t+1}$ by selecting ...
3
votes
1answer
36 views

Large Increase in Naive Bayes Accuracy results in decrease precision?

I am trying to classify (37 possible classes) a dataset that has 9 features and 900,000 occurrences. I have tried a couple different algorithms but the one that gave the lowest logarithmic loss was ...
0
votes
0answers
69 views

Naive Bayes on a continuous dependent variable?

I read 'Design principles of massive, robust prediction systems' and have a similar requirement, i.e. a system that can evaluate new events extremely quickly (i.e. low single-digit milliseconds) and ...
1
vote
0answers
35 views

Data distribution problem

I am having the transaction dataset which contains 95% or user initiated transactions and 5% fraud transactions. When I am fitting the logistic regression model, it gives me bias prediction - as most ...
0
votes
0answers
23 views

Is it possible to achive low error on MNIST using Random Ferns?

I'm new in machine learning and i want to study how to use random ferns. I read this paper Fast Keypoint Recognition in Ten Lines of Code and implement simple version of algorithm. But then I tried ...
2
votes
0answers
17 views

Impact of conjugate priors on mutual information for Naive Bayes

I am currently thinking about the following problem. Suppose you have a simple Naive Bayes model for binary classification based on binary random variables. For example, suppose you want to predict ...
2
votes
2answers
66 views

How can I improve feature selection for my Naive Bayes Classifier?

I am classifying companies into two classes ( a particular business type, or not that business type ), using a Naive Bayes Classifier. Specifically, I'm using PHP and PHP NLP Tools. I have two ...
1
vote
0answers
25 views

Suspicious Amount of Zeros in Confusion Matrix

I have a data set with about 45000 observations and three features. When I apply machine learning classification algorithms like naive Bayes, kNN and SVM I receive a lot of zeros in the resulting ...
0
votes
0answers
28 views

Why can I use the posterior probability of a classifier as a new classifier?

I have read that, when doing discriminant analysis, you can use the posterior probability you obtain using your classifier as a new fine-tuned classifier. Can anyone talk me through the rationale of ...
1
vote
1answer
68 views

Linear discriminant analysis (Fisher) = Bayes?

I'd like to ask a question, I am reading book right now about mail filtering, both methods: naïve Bayes and Fisher are there very similar in implementation. I am also writing a paper on Bayesian spam ...
0
votes
1answer
51 views

Naive Bayes Binary Classification with Binary Features

I have a dataset with two classes $C_0$ and $C_1$. I have around $10$ to $20$ features that take binary values (either $0$ or $1$). My dataset has around $10000$ instances, with only a hundred of ...
1
vote
1answer
55 views

Can adding an additional feature to a perceptron classifier make the results worse?

I am using perceptron to solve a classification problem. I have a limited amount of features (26) and iterate through all possible combinations of them. A combination of two features [feature_a, ...
1
vote
0answers
19 views

How could these two simple Bayesian algorithms be explained, simply? [closed]

count(this token in class) + 1 / count(all tokens in class) + count( all tokens ) and ...
0
votes
1answer
28 views

Choosing smoothing parameters across multiple Naïve Bayes classifiers with different number of categories

I would like to train multiple Naïve Bayes classifiers with different number of categories, and also have a global threshold for how certain one classifier must be in order for the classification to ...
0
votes
0answers
42 views

K-cross validation and Naive Bayes

I am doing an exercise of machine learning, and I have built a Gaussian Naive Bayes classifier (i.e., I have defined values of mean and standard deviation) using scikit-learn. Now I am supposed to ...
1
vote
0answers
39 views

Is possible classify audio file using naive bayes?

I like use naive bayes classifier to classify audio file (.mp3). I use Key, Mode, Loudness, Energy and Tempo as feature. So, the classifier will classify new file to 2 class, they are POP and ROCK. ...
0
votes
0answers
13 views

Classifier with interchangeable features

I have a situation in which the features used in a classifier are multiple instances of the same kind of measurement, in random (or unknown) order; thus, a sample x1, x2, ... xn -> classA could with ...
1
vote
1answer
414 views

Buiding Ensemble model

I'm new to ensemble model. Suppose I've KNN models like this - (in R) library(class) ...
0
votes
0answers
20 views

Naive Bayes in text classification - Only classifying “0”

I have a data set of 1000 Amazon "art" category reviews. I want to classify Positive +1, Negative -1, Neutral 0 Ratings using user reviews. The final Naive Bayes classifier only predicts 0 for all ...
1
vote
1answer
63 views

How to use Naive Bayes for multi class problems?

I know how Naive Bayes work for classifying binary problems. I just need to know what are the standard way to apply NB on multi-class classification problems. Any idea please?
3
votes
2answers
414 views

How is Naive Bayes a Linear Classifier?

I've seen the other thread here but I don't think the answer satisfied the actual question. What I have continually read is that Naive Bayes is a linear classifier (ex: here) (such that it draws a ...
0
votes
0answers
32 views

Values of PDF in Bayes Classifier

I'm new here and also a beginner in statistics. I'm implementing a Bayes classifier for two classes but get confused with the value of likelihood (pdf). $$P(c|o) = p(o|c)\cdot P(c)/p(o);$$ Here ...
1
vote
0answers
76 views

Bias and variance of a naive bayes classifier and KNN classifier

After reading the paper by J. Friedman, ”On bias, variance, 0/1-loss, and the curse-of-dimensionality,” Data Mining and Knowl- edge Discovery, 1997. I would like to estimate both bias and variance ...
0
votes
0answers
26 views

Fisher's linear discriminant VS naive bayes

i know some basic things about linear classifiers. i prefer two think geometrically about them. and what'll happening if covariance Matrix were different things. Generally; What's the difference of ...
0
votes
0answers
54 views

Accuracy increases using cross-validation and decreases without

I have a question regarding cross validation: I'm using a Naive Bayes classifier to classify blog posts by author. When I validate my dataset without k-fold cross validation I get an accuracy score of ...
1
vote
0answers
33 views

sklearn - Multinomial Naive Bayes (data too big???!!!)

I wanted to really understand the rationale behind the following code as written in python sklearn's manual partial_fit(X, y, classes=None, sample_weight=None) when the data is too big to fit in ...
0
votes
1answer
37 views

From Naive Bayes to “real percentages”

I am working on a problem which is well suited to be solved with a Naive Bayes classifier: I want to know if a certain instance belongs to a certain class. I have three attributes that can help ...
1
vote
0answers
62 views

predict sales using naive bayes and handle sparse data problem

Problem I am trying to use naive bayes for ranking products in a search application. I would like to predict the sales of a given product given the search keyword and the category. the current formula ...
0
votes
0answers
32 views

Is Naive Bayes robust?

We know that according to Naive Bayes assumption input features are assumed to be independent of each others given the target variable $y$. Now, If we intentionally add a duplicate (exact copy of ...
2
votes
2answers
121 views

Naive bayes with duplicated data

In the training set for naive bayes, there are some duplicate samples. Should we train the naive bayes with duplicate samples, or should we eliminate all the duplicates and then train the naive bayes. ...