A naive Bayes classifier is a simple probabilistic classifier based on applying Bayes' theorem with strong independence assumptions. A more descriptive term for the underlying probability model would be "independent feature model".

learn more… | top users | synonyms

0
votes
0answers
17 views

Fisher's linear discriminant VS naive bayes

i know some basic things about linear classifiers. i prefer two think geometrically about them. and what'll happening if covariance Matrix were different things. Generally; What's the difference of ...
0
votes
0answers
24 views

Accuracy increases using cross-validation and decreases without

I have a question regarding cross validation: I'm using a Naive Bayes classifier to classify blog posts by author. When I validate my dataset without k-fold cross validation I get an accuracy score of ...
0
votes
0answers
14 views

sklearn - Multinomial Naive Bayes (data too big???!!!)

I wanted to really understand the rationale behind the following code as written in python sklearn's manual partial_fit(X, y, classes=None, sample_weight=None) when the data is too big to fit in ...
0
votes
1answer
25 views

From Naive Bayes to “real percentages”

I am working on a problem which is well suited to be solved with a Naive Bayes classifier: I want to know if a certain instance belongs to a certain class. I have three attributes that can help ...
0
votes
0answers
22 views

predict sales using naive bayes and handle sparse data problem

Problem I am trying to use naive bayes for ranking products in a search application. I would like to predict the sales of a given product given the search keyword and the category. the current formula ...
0
votes
0answers
20 views

Is Naive Bayes robust?

We know that according to Naive Bayes assumption input features are assumed to be independent of each others given the target variable $y$. Now, If we intentionally add a duplicate (exact copy of ...
2
votes
2answers
83 views

Naive bayes with duplicated data

In the training set for naive bayes, there are some duplicate samples. Should we train the naive bayes with duplicate samples, or should we eliminate all the duplicates and then train the naive bayes. ...
0
votes
0answers
12 views

To interpret SVM's probability output in text analysis

So, my question is about the application of the SVM's (or naive Bayes') probability output (via Platt's scaling). I know the interpretation of the output is the probability of a given observation ...
0
votes
0answers
16 views

How to deal with mixture of continuous and discrete features when using Naive Bayes classifier

My task is to use Naive Bayes classifier for prediction, where I have both continuous and discrete variables as predictor variables. In literature the classifier is written as: $$\hat{y}= ...
1
vote
0answers
38 views

Is the Naive Bayes family of classifiers linear?

There are a lot of places where you'll see the proof that Naive Bayes classifiers are linear, like this and this. But they always assume a special case of the family of Naive Bayes classifiers which ...
1
vote
0answers
24 views

Naive Bayes text classification on different cardinality classes

I have written a Naive Bayes classifier (with Laplace smoothing) and am using it to classify text into a few simple classes. However, I found that the classes are not of the same vocab size -- to be ...
0
votes
0answers
53 views

Can a Naive Bayes Model predict using pattern alone?

Say I have a set of data abc-def-ghi jkl-mno-pqr stu-vwx-yza and lots of other training samples which are catagorized as **names*. The above dataset does not have ...
0
votes
0answers
23 views

Is Laplace smoothing still good when the class sizes are unbalanced?

I'm thinking about text classification problems using naive bayes. Laplace smoothing is often recommended to eliminate zero probabilities. Let's assume that there are only two classes: A and B. ...
0
votes
0answers
66 views

How to validate sentiment classification and compare different algorithms

I need to compare SVM and NB about sentiment classification by evaluating accuracy, precision and recall measures. I have 1500 manually classified documents, and I would know which is the best way to ...
0
votes
0answers
21 views

Normalization of Naive Bayes output

In Scikit-learn documentation it is possible to see that the MultinomialNB estimator has a method called predict-proba in which it has the following description: "Returns the probability of the ...
3
votes
1answer
48 views

Deriving the Bayes Filter Correction Equation

The correction rule for Bayes filters is: $$p\left(x_{k}|D_{k}\right)=\dfrac{p\left(y_{k}|x_{k}\right)\cdot p\left(x_{k}|D_{k-1}\right)}{p\left(y_{k}|D_{k-1}\right)} $$ For: State at time $k$ is ...
1
vote
1answer
73 views

Text Classification using TfIdf and Bernoulli NB

So, as I am reading about Bernoulli distribution and text classification, I want to understand how Bernoulli uses TfIdf features? Since TfIdf values are within [0-1) but Multivariate Bernoulli assumes ...
0
votes
1answer
309 views

Request: Clever Things to do with Naive Bayes

I am trying to drive up the performance of a Naive Bayes classifier, and I haven't been having terribly much luck. I've been working in Weka, but have enough knowledge of R to (possibly) implement ...
0
votes
0answers
33 views

Decision function for BernoulliNB classifier. ( for use in plotting ROC and PR curves )

I would like to plot the PR curve using scikit-learn for the Bernoulli Naive Bayes estimator. However, attempting to do so give an error, ...
3
votes
0answers
60 views

Explanation for large difference in SVM and Naive bayes results

I have a dataset with 389 data evenly distributed into 6 classes. Each data has 1024 features. So my dimension is much larger than my observation data. I have tried to see some common classifiers on ...
0
votes
0answers
84 views

Naive Bayes Nearest Neighbor (NBNN) implementation problems in MATLAB

I'm currently trying to classify the CIFAR-10 image dataset. I cam across a number of papers praising the the results from a non-parametric approach called Naive Bayes Nearest Neighbors. It uses SIFT ...
1
vote
0answers
81 views

Alternatives to Non-Linear Regression

I'm not a professional statistician but I frequently work in the area of data analysis using R and Python, and frequently use linear regression models (OLS) or quantile regression, and tree based ...
1
vote
2answers
62 views

Balanced datasets in Naive Bayes

In a classification model, a desirable situation is to have classification classes evenly represented in the training dataset. Datasets that satisfy this property are called balanced datasets. ...
0
votes
1answer
31 views

Biasing SkLearn Algorithms to Positive Outcomes

I am trying to run multinomial naive bayes on a series of examples in python using sci kit learn. I am consitently getting all examples classified as negative. (The ratio of positives to negatives in ...
1
vote
0answers
21 views

fragmentation problem in decision tree

I am taking a NLP class, in which it says decision tree has the fragmentation problem. It says ...
2
votes
2answers
82 views

How to prepare a dataset for text classification

I would like to compare some algorithms for performing sentiment classification (Naive Bayes, SVM, and ...
0
votes
1answer
50 views

define prior probabilities in naive bayes with unbalanced classes and asymetric cost

I'm trying to apply Naive bayes to the following supervised problem: It's a binary classification problem The classes are unbalanced. The target class represents the 0.004266432 of the total and the ...
3
votes
2answers
109 views

Can $p(Y|a,b)$ ever be equal to $p(Y|a) \cdot p(Y|b)$?

This strikes me as a simple question, but in re-visiting how the Naive Classifier works I started wondering if there is any probabilistic model that under certain independency assumptions obtains: ...
0
votes
0answers
21 views

Naive Bayes Produce Confidence

I am pretty newbie in machine learning. Please forgive and point out anyone incorrect use of terminology. Now I am learning Naive Bayes algorithm. As I have learned Neural Network, when predicting, ...
0
votes
0answers
40 views

Naives Bayes example walk through

My instructor have give me an example that I didn't really understand, can someone help me in understanding it? Given a list of occupation and a list of grocery bought I want to recommend a list of ...
0
votes
0answers
27 views

Regarding Naive Bayes and conditional independence

We all have been talking about how Naive Bayes may, in some cases, not perform well due to the fact that this assumes conditional independence of features and MOSTLY, this is not true for real world ...
0
votes
0answers
60 views

Which naive Bayes?

I am attempting to use a naïve Bayes classifier in python (using scikit-learn), with two examples. The first example has 6 classes and 2 hypotheses, the 2nd example has 2 classes and 6 hypotheses. ...
0
votes
1answer
32 views

Turning categorized output into continuous

I'm using a NaiveBayes algorithm that generates categorized probabilities as output instead of continuous values, which is what I need for this webapp I'm working on. Unfortunately I can't switch ...
-1
votes
1answer
47 views

How to choose the right model after k-fold cross validation is done?

I'm using naive bayes to classify tweet into three classes. and i want to use k-fold cross validation to predict the right model, but i'm confused how to choose the right model after k-fold validation ...
0
votes
1answer
55 views

Independent variables in Naïve Bayes

In the perfect explanation of Bayes' Theorem here as far as I know features of the class should be independent. The question is how to prove statistically that two given features are independent? I ...
0
votes
0answers
42 views

What is the relationship between naive Bayes and Mahalanobis distance

Recently, I found a code project which uses the Mahalanobis distance to compute Bayes value, but I don't know why you can do that. Second, as I know naive Bayes is based on the Bayes rule, and how ...
0
votes
0answers
20 views

Estimating covariance for naive Bayes

I am a beginner in Pattern Recognition and started reading up Bayesian classifiers. I came across the case of naive Bayes with equal covariance in all dimensions. Given sufficient data, how does one ...
1
vote
0answers
88 views

In Kneser-Ney smoothing, how are unseen words handled?

From what I have seen, the (second-order) Kneser-Ney smoothing formula is in some way or another given as $ \begin{align} P^2_{KN}(w_n|w_{n-1}) &= \frac{\max \left\{ C\left(w_{n-1}, w_n\right) - ...
1
vote
0answers
162 views

Naive Bayes and text classification: which probability model and vectorizer combination makes sense?

I am wondering which combinations of Naive models can be paired with different vectorizing methods so that it makes sense. Let's say we have a simple binary spam-classification task. Multinomial ...
1
vote
0answers
38 views

Word probabilities in a Naive Bayes filter

While implementing a Naive Bayes filter, I stumbled across a problem with the calculation of the conditional probabilities $p(w|c)$ of a word $w \in \mathcal{W}$ given a class $c \in \mathcal{C}$. ...
1
vote
0answers
9 views

Smoothing strategies for features assuming values from countably infinite domains

I am in the midst of programming a simple Naive Bayes classifier as an exercise. It is supposed to perform word-sense disambiguation on natural language phrases, e.g. predicting the correct meaning of ...
1
vote
0answers
65 views

Parameter Estimation for Naive Bayes - Maximum a posteriori and Maximum Likelihood

I am wondering if I understand those terms correctly. To summarize my thoughts: In naive Bayes, our decision rule is basically the Maximum a posteriori (MAP) estimate of our hypothesis. We assign an ...
1
vote
1answer
40 views

Classification tips for a begginer

I'm doing a graduation work that involves applying Classification algorithms in a dataset of matches from Dota 2 (a popular MOBA game). Here's an explanation of the problem: Dota 2 matches are played ...
1
vote
0answers
29 views

Document classification problem

Assume we have $L$ labelled documents, and $U$ unlabeled ones, where all the documents from class $k$ were generated from a multinomial or Naive Bayes distribution with parameter $\theta_k$, and ...
0
votes
0answers
24 views

Entropy weighted Naive Bayes performs poorer than regular Naive Bayes?

I have a text classification problem, where there are many different classes, and the text to be classified is very short (about 1 sentence each): ...
0
votes
0answers
45 views

Naive Bayes classifier calculation

I'm trying to use naive Bayes classifier to classify my dataset.My questions are: 1- Usually when we try to calculate the likehood we use the formula: P(c|x)= P(c|x1) * P(c|x2)*...P(c|xn)*P(c) . But ...
0
votes
0answers
30 views

Post-process the output of a Multinomial Naive Bayes text classifier

I have a multinomial text classification application where there are other features than the words in text which can be useful to do the classification e.g, contains email address, contains an URL, ...
3
votes
3answers
525 views

In Naive Bayes, why bother with Laplacian smoothing when we have unknown words in the test set?

I was reading over Naive Bayes Classification today. I read, under the heading of Parameter Estimation with add 1 smoothing: "Let $c$ refer to a class (such as Positive or Negative), and let $w$ ...
1
vote
1answer
110 views

How to handle unseen features in a Naive Bayes classifier?

I am writing a naive bayes classifier for a text classification problem. I have a bunch of words and an associated label: ...
1
vote
1answer
83 views

Greater than 1 Naive Bayes Probabilities?

I am trying to train a Naive Bayes classifier. In addition to getting the most likely class as an output from the Naive Bayes classifier, I would also like to compute the probabilities associated with ...