Tagged Questions

A naive Bayes classifier is a simple probabilistic classifier based on applying Bayes' theorem with strong independence assumptions. A more descriptive term for the underlying probability model would be "independent feature model".

learn more… | top users | synonyms

0
votes
0answers
14 views

Naive Bayes Produce Confidence

I am pretty newbie in machine learning. Please forgive and point out anyone incorrect use of terminology. Now I am learning Naive Bayes algorithm. As I have learned Neural Network, when predicting, ...
0
votes
0answers
31 views

Naives Bayes example walk through

My instructor have give me an example that I didn't really understand, can someone help me in understanding it? Given a list of occupation and a list of grocery bought I want to recommend a list of ...
0
votes
0answers
16 views

Regarding Naive Bayes and conditional independence

We all have been talking about how Naive Bayes may, in some cases, not perform well due to the fact that this assumes conditional independence of features and MOSTLY, this is not true for real world ...
0
votes
0answers
27 views

Which naive Bayes?

I am attempting to use a naïve Bayes classifier in python (using scikit-learn), with two examples. The first example has 6 classes and 2 hypotheses, the 2nd example has 2 classes and 6 hypotheses. ...
0
votes
1answer
25 views

Turning categorized output into continuous

I'm using a NaiveBayes algorithm that generates categorized probabilities as output instead of continuous values, which is what I need for this webapp I'm working on. Unfortunately I can't switch ...
-1
votes
1answer
33 views

How to choose the right model after k-fold cross validation is done?

I'm using naive bayes to classify tweet into three classes. and i want to use k-fold cross validation to predict the right model, but i'm confused how to choose the right model after k-fold validation ...
0
votes
1answer
33 views

Independent variables in Naïve Bayes

In the perfect explanation of Bayes' Theorem here as far as I know features of the class should be independent. The question is how to prove statistically that two given features are independent? I ...
0
votes
0answers
28 views

What is the relationship between naive Bayes and Mahalanobis distance

Recently, I found a code project which uses the Mahalanobis distance to compute Bayes value, but I don't know why you can do that. Second, as I know naive Bayes is based on the Bayes rule, and how ...
0
votes
0answers
11 views

Estimating covariance for naive Bayes

I am a beginner in Pattern Recognition and started reading up Bayesian classifiers. I came across the case of naive Bayes with equal covariance in all dimensions. Given sufficient data, how does one ...
1
vote
0answers
30 views

In Kneser-Ney smoothing, how are unseen words handled?

From what I have seen, the (second-order) Kneser-Ney smoothing formula is in some way or another given as $ \begin{align} P^2_{KN}(w_n|w_{n-1}) &= \frac{\max \left\{ C\left(w_{n-1}, w_n\right) - ...
1
vote
0answers
37 views

Naive Bayes and text classification: which probability model and vectorizer combination makes sense?

I am wondering which combinations of Naive models can be paired with different vectorizing methods so that it makes sense. Let's say we have a simple binary spam-classification task. Multinomial ...
1
vote
0answers
26 views

Word probabilities in a Naive Bayes filter

While implementing a Naive Bayes filter, I stumbled across a problem with the calculation of the conditional probabilities $p(w|c)$ of a word $w \in \mathcal{W}$ given a class $c \in \mathcal{C}$. ...
1
vote
0answers
6 views

Smoothing strategies for features assuming values from countably infinite domains

I am in the midst of programming a simple Naive Bayes classifier as an exercise. It is supposed to perform word-sense disambiguation on natural language phrases, e.g. predicting the correct meaning of ...
0
votes
0answers
29 views

Parameter Estimation for Naive Bayes - Maximum a posteriori and Maximum Likelihood

I am wondering if I understand those terms correctly. To summarize my thoughts: In naive Bayes, our decision rule is basically the Maximum a posteriori (MAP) estimate of our hypothesis. We assign an ...
1
vote
1answer
31 views

Classification tips for a begginer

I'm doing a graduation work that involves applying Classification algorithms in a dataset of matches from Dota 2 (a popular MOBA game). Here's an explanation of the problem: Dota 2 matches are played ...
1
vote
0answers
27 views

Document classification problem

Assume we have $L$ labelled documents, and $U$ unlabeled ones, where all the documents from class $k$ were generated from a multinomial or Naive Bayes distribution with parameter $\theta_k$, and ...
0
votes
0answers
16 views

Entropy weighted Naive Bayes performs poorer than regular Naive Bayes?

I have a text classification problem, where there are many different classes, and the text to be classified is very short (about 1 sentence each): ...
0
votes
0answers
30 views

Naive Bayes classifier calculation

I'm trying to use naive Bayes classifier to classify my dataset.My questions are: 1- Usually when we try to calculate the likehood we use the formula: P(c|x)= P(c|x1) * P(c|x2)*...P(c|xn)*P(c) . But ...
0
votes
0answers
22 views

Post-process the output of a Multinomial Naive Bayes text classifier

I have a multinomial text classification application where there are other features than the words in text which can be useful to do the classification e.g, contains email address, contains an URL, ...
3
votes
3answers
112 views

In Naive Bayes, why bother with Laplacian smoothing when we have unknown words in the test set?

I was reading over Naive Bayes Classification today. I read, under the heading of Parameter Estimation with add 1 smoothing: "Let $c$ refer to a class (such as Positive or Negative), and let $w$ ...
1
vote
1answer
75 views

How to handle unseen features in a Naive Bayes classifier?

I am writing a naive bayes classifier for a text classification problem. I have a bunch of words and an associated label: ...
1
vote
1answer
42 views

Greater than 1 Naive Bayes Probabilities?

I am trying to train a Naive Bayes classifier. In addition to getting the most likely class as an output from the Naive Bayes classifier, I would also like to compute the probabilities associated with ...
1
vote
0answers
16 views

Variable coarsening in Naive Bayes

Say we have a binary classification problem that we solve with Naive Bayes. All features are categorical variables. Say we focus on a single feature that takes one of $N$ possible values. If $N$ is ...
0
votes
0answers
64 views

visualize the naive bayes with k-fold cv

I have done the classification using naive Bayes as a classifier, and applied 10-fold CV.I know that I can get the mean and variance of the result. However, how can I plot the classifier performance? ...
1
vote
1answer
113 views

Example of how the log-sum-exp trick works in Naive Bayes

I have read about the log-sum-exp trick in many places (e.g. here, and here) but have never seen an example of how it is applied specifically to the Naive Bayes classifier (e.g. with discrete features ...
1
vote
1answer
89 views

Gaussian Naive Bayes really equivalent to GMM with diagonal covariance matrices?

Murphy writes that a multivariate Gaussian used in a generative classifier ("Gaussian discriminant analysis"), i.e., $p(\mathbf x|y=c,\mathbf\theta) = \mathcal{N}(\mathbf x|\mathbf y_c,\mathbf ...
0
votes
2answers
152 views

Best feature selection method for naive Bayes classification

i want to make classification with naive Bayes. I have got about 100 Features. Numerical ones as well as categorical ones. Since i want only the most relevant ones to be included for the ...
0
votes
0answers
16 views

Conditional Probability With Naive Bayes

I have recently begun exploring R I have downloaded a data set that contains flight times for when a plane takes off at Origin and Lands at destination The data has up to three months’ worth of ...
2
votes
3answers
201 views

Naive Bayes: Imbalanced Dataset in Real-time Scenario

I am using scikit-learn Multinomial Naive Bayes classifier for binary text classification (classifier tells me whether the document belongs to the category X or not). I use a balanced dataset to train ...
0
votes
0answers
44 views

A Graphical Model for Fellegi-Sunter Record Linkage

I am trying to understand the Fellegi-Sunter Probability Model for Record Linkage problem. I am following the thesis at: http://www.inf.ed.ac.uk/publications/thesis/online/IM080663.pdf in order to ...
1
vote
1answer
36 views

Multinomial Naive Bayes performance with and without Bigrams/Stopwords

I have a dataset of 300 text documents that I have manually classified into three classes. The first two classes are relatively equal in size; the third class is about 40% larger than each of the ...
1
vote
0answers
46 views

How do we build a model with pyMC

I am a newbie programmer with pyMC. I have a model given as following $$\rho=\sqrt(x^2+y^2)$$ $$z=f(x,y)=\frac{c^3}{\log(c)+1}\frac{\log\rho}{\rho^2}+\frac{5.3}{r^2}\rho^{-3}$$ and I want to find the ...
1
vote
1answer
88 views

Using PyMC to obtain the posterior for the parameters of a parametric model?

I am a newbie with pyMC and I want to program an MCMC sampling for a complicated problem. I have a function given by in the following: $$g(r,c,\theta_1,\theta_2)=\frac{\delta_c}{\Sigma ...
0
votes
0answers
18 views

Are there any issues with the naive bayes classifier when the number of features are greater than the number of observations?

Say I have $f$ features and $n$ observations. If $f>>n$ are there any inherent issues that arise if a naive bayes classifier is used on such a data set? To be more concrete, let's say $f=200$ ...
1
vote
1answer
186 views

Naive Bayes with unbalanced classes

As a part of a project for the university is should train a Naive Bayes classifier to classify question and answers in three different categories, the task should be easy since that the 3 classes are ...
0
votes
4answers
44 views

Get rid of the influence of a predictor

I am currently working on a data mining project during an in internship, and I use among others methods decision trees. My problem is that I have a categorical predictor very influent on ...
1
vote
0answers
123 views

Overfitting naive Bayes

I understand naive Bayes is used largely in text classification. However, the number of features tend to outnumber the number of documents. Does this not result in overfitting where the number of ...
1
vote
2answers
34 views

Different number of samples affecting naive Bayes

I am trying to build a model of 7 classes text documents. However, i do not have equal number of samples for each class. I can have close to about 10k documents for class A but only about 100 ...
1
vote
0answers
114 views

Possible Problem in Naive Bayes Spam Filters

I realized that, never been observed words in training sets, decrease test probaility of documents being spam and not spam after i mentioned here and this link "... you need to account for it as a ...
1
vote
2answers
194 views

Naive Bayes non-Dictionary Term in Test Document

Using Laplacian Smoothing we can get rid of 0 probabilities if a term occur in spam and does not occur in ham class or vice versa. My question is about what if a term in test document does not occur ...
1
vote
0answers
27 views

What is delta in the maximization step in this EM algorithm?

The algorithm is used to classify english vs non-english tweets from unlabeled data. Given n observed tweets (x1 ... xn) where each tweet xi is a collection of d words (xi1 ... xid). y is the class ...
0
votes
1answer
107 views

Naive Bayes Python implementation differences

Am currently using Naive Bayes for a multi labelled text document classification problem. But I would like to know the differences (advantages and disadvantages) of using SkiLearn Naive Bayes or ...
1
vote
1answer
76 views

Machine Learning Algorithm Confusion

I made a small application about cricket prediction using Machine Learning. I took records of 10 years (2001-2011) of ODI matches and prepared a training set. Now to predict a win or loss for a ...
0
votes
1answer
56 views

Passing a single item and getting a prediction

I have completed a demo of a naive Bayes classifier for predicting whether an SMS message is spam or ham. My question is, how can I use this in a practical manner? It appears I would have to ...
0
votes
0answers
41 views

Why naive bayes classifier cannot perform correctly on 1 conditional probability?

I use e1071 package for naive bayes classification. My dataset is having 2 condition/feature (FF1 and FF2) to predict a class of signal (ELO or FP). When I used the following line of code: ...
0
votes
0answers
38 views

Why are Bayesian classifiers “robust to noise”?

In many different settings I've read that Bayesian classifiers like Naïve Bayes and Bayesian Networks are more robust to noise in the input data than other classifiers. I'm wondering what the evidence ...
0
votes
1answer
133 views

Naive Bayes: Continuous and Categorical Predictors

It's my understanding that most types of common classifiers (Support Vector Machine, for example) can take a mixture of categorical and continuous predictors. However, this doesn't seem to be true ...
0
votes
0answers
36 views

Probability of class membership given univariate normal distribution

Assuming a class is well described with a normal distribution of u and s, is it reasonable to calculate the probability of membership as: $Pr(x)=Pr(|x-u|)=2.0*\text{cdf}(|x-u|,s)$? I've briefly ...
1
vote
0answers
31 views

Obtaining and sampling from the posterior predictive of a naive Bayes classifier

I have trained a naive Bayes classifier with on a dataset with a dichotomous outcome and multinomial attributes (predictors). I managed to get a Maximum a posteriori (MAP) estimate which is good ...
0
votes
0answers
17 views

How to cross validate with optimal sample size selection?

My problem is as follows: I got a training set composed of texts, where 10% of observations are 1, and 90% are 0. To make it simpler, lets say that 1000 is 1, and 9000 is 0. I implement Naive Bayes ...