Questions tagged [naive-bayes]

A naive Bayes classifier is a simple probabilistic classifier based on applying Bayes' theorem with strong independence assumptions. A more descriptive term for the underlying probability model would be "independent feature model".

Filter by
Sorted by
Tagged with
0
votes
1answer
36 views

Need suggestion/guide on how to estimate unknown bayesian priors

Suppose I can only observe people who visit Starbucks. My posterior probabilities will be like $\Pr(\text{male} \mid \text{visits Starbucks})$, $\Pr(\text{has hair} \mid \text{visits Starbucks})$, ...
0
votes
1answer
278 views

How to use Naive Bayes with “not found” label?

I'm trying to do text detection thanks to Naive Bayes Algorithm. If I teach my tool: "Football is a great hobby" and assign it to the label "football", I'm totally fine with it detecting "I play ...
4
votes
2answers
3k views

Confused among Gaussian, Multinomial and Binomial Naive Bayes for Text Classification

I am doing text classification but I am confused which Naive Bayes model I should use. What I understood by reading answers from couple of places that Gaussian Naive Bayes can be used if the attribute ...
1
vote
0answers
51 views

Can inferencing come from incomplete rule sets?

I have some data for medical diagnosis, consisting of some rules about relationship of diseases and their symptoms, for example disease D1 frequently has symptom S1 ...
2
votes
1answer
2k views

Naive Bayes logarithmic probability

I am trying to do sentiment analysis using Naive Bayes and have a doubt regarding log. While calculating posterior probability in Naive Bayes classifier, we apply log to prevent underflows and very ...
2
votes
1answer
1k views

How does the beta prior - binomial conjugate theory hold in classifiers?

In our machine learning class, we were given an example of a naive bayes classifier. Say, you classify a day as being good/bad depending on 2 conditions (the "X" input) - weather(X1 - hot/cold) and ...
1
vote
1answer
2k views

Small Probabilities in Naive Bayes

I am trying to implement Naive Bayes, but I am encountering a problem. I have 5000 word features. Hence, every sample is a binary vector of length 5000. The true labels are 1 or 0. The value of P(...
0
votes
1answer
276 views

What non-Bayesian classifiers could be used in a naive Bayes model?

Bayes' theorem [is used] in the classifier's decision rule, but naive Bayes is not (necessarily) a Bayesian method. -Wikipedia. I have applied naive Bayes assumption to LDA, but it has a Bayes ...
0
votes
1answer
141 views

Training and validating a naive gaussian bayes classifier

I've been reading up on the theory of naive bayes classifiers, specifically the ones in which the probability $p(x|c)$ is a random vector. The problem which I'm trying to solve takes a 4 dimensional ...
2
votes
1answer
876 views

Classification problem: compare results of a decision tree, naive bayes and 1-nearest-neighbor classifier

These are the results of a classification problem using decision tree, naive bayes and 1-nearest-neighbor as classifiers. There are 10,000 data objects and these results were validated using 10-fold ...
5
votes
2answers
10k views

Bernoulli NB vs MultiNomial NB, How to choose among different NB algorithms?

I want to understand the logic behind using a specific type of NB algorithm for a particular dataset. I read about Naive Bayes but still few things are unclear. According to my understanding of NB ...
0
votes
0answers
478 views

Naive bayes performs worse than predicting the most common answer?

I have input X, with 22 binary features and 70000 examples. The target y is one of 4 possible categories. They are unbalanced, with the most common having a bit more than 51% of the data. When I train ...
2
votes
2answers
3k views

Naive Bayes: Mix unigrams and bigrams for text classification?

I'm creating a naive bayes text classifier, but I'm wondering if it's a good idea to break the text up into both unigrams and bigrams. Should I only use one method? Will having both variations mess ...
0
votes
1answer
3k views

Naive Bayes with Laplace Smoothing Probabilities Not Adding Up

Let c refer to a class (such as Positive or Negative), and let w refer to a token or word. Define $count(w,c) = $ $counts \ w \ in\ class \ c$ $count(c) = counts \ of \ words \ in \ class \...
1
vote
0answers
568 views

Predict stock market using twitter

I'm trying to predict the daily positivity or negativity of stock market value through Twitter. I researched a lot about this topic and I found this article to start. Basically, what I've done is get ...
1
vote
1answer
2k views

Standardisation in Naive Bayes?

Is it possible that the accuracy of Naive Bayes remain the same even after applying Standardisation . I have applied 2 Standardisation techniques : Min Max Scaling ( which squishes the range from 0-1 ...
2
votes
1answer
964 views

Why the probability greater than 100% when I use Naive Bayesian for classification

I use Naive Bayesian to train a dataSet: ...
2
votes
0answers
969 views

Confidence interval for naive bayes estimation

I have some simple model with 2 independent input variables A,B . I want to calculate some probability. $$ P(X|a,b) = P(a,b|X) P(X) / P(a,b) \\ P(X|a,b) = P(a|X) P(b|X) * P(X) / ( P(a) * ...
3
votes
2answers
949 views

Why does training naive Bayes on a data set in which all the features are repeated increase the confidence of the naive Bayes probability estimates?

I am looking for a toy example to understand this behavior. Preferebly a text classification one I read the following from http://people.cs.umass.edu/~mccallum/papers/crf-tutorial.pdf at page 7. ...
2
votes
1answer
400 views

Why Bernoulli Naive Bayes explicitly penalizes the non-occurrence of a feature while Multinominal NB simply ignoring it?

In the naive bayes page in Scikit-Lean, one of differences between Bernoulli Naive Bayes and Multinomial Naive Bayes is that it (bernoulli) explicitly penalizes the non-occurrence of a feature i ...
2
votes
1answer
6k views

With the Naive Bayes classifier, why do we have to normalize the probabilities after calculating the probabilities of each hypothesis?

In the Naive Bayes classifier, why do we have to normalize the probabilities after calculating the probabilities of each hypothesis?
1
vote
1answer
2k views

What is the effect of data scaling when compared between kNN, naive Bayes or logistic regression?

I am a complete noob and have just started with machine learning. However there was this one question I faced which I was unable to answer. What is the effect of data (feature) scaling on the three ...
2
votes
1answer
5k views

Text classification based on keywords

Totally new to ML so please bear with me. I'm trying to classify text from certain email messages and RSS feed entries. The texts should be classified as either relevant or irrelevant. The decision ...
3
votes
2answers
3k views

Laplace smoothing understanding implementation

Considering the data set given below Here if we have to classify new data point: D15 (O=Overcast, T=Cool, H=High, W=Strong) Then for P(No|Overcast, Cool, High, Strong) we have, (5/14) * 0 * (1/5) *...
1
vote
3answers
776 views

naiveBayes does not give expected probabilities

I do not understand how the naiveBayes method from the e1071 package is calculating probabilities when classes are perfectly separable (more generally in categories where there are only one class ...
4
votes
1answer
729 views

Random Forest / Random Ferns combining the result of each predictor

I am trying to understand the ways that the predictions of each Tree in a Random Forest or each Fern in Random Ferns are combined to form a single response. I did read somewhere (reference missing) ...
4
votes
1answer
12k views

When to use Bernoulli Naive Bayes?

Below is an example of a dummy scatter plot of x,y where BLUE (0) and RED(1) are the 'target'. The yellow dot is my input and I'...
1
vote
1answer
2k views

A simple numerical example for Kneser-Ney Smoothing

I'm working in a project trying to implement the Kneser-Key algorithm. I think I got up to the step of implementing this formula for bigrams: $P_{(KN)}(w_i|w_{i-1}) = \frac{max(c(w_{-1}, w_{1}) - \...
0
votes
1answer
127 views

Prove $P(A|B)=P(A\cap B)/P(B)$?

I've been doing some learning for Native Bayes classification. I came across this formula, but I'm having trouble remembering it because I don't know how to get this formula. Can anyone explain how to ...
30
votes
3answers
43k views

What algorithms need feature scaling, beside from SVM?

I am working with many algorithms: RandomForest, DecisionTrees, NaiveBayes, SVM (kernel=linear and rbf), KNN, LDA and XGBoost. All of them were pretty fast except for SVM. That is when I got to know ...
0
votes
1answer
65 views

Can you explain the following code for detection and removal of outliers?

My teacher wrote this to detect and remove Outliers from a data set. ...
2
votes
0answers
918 views

Can naive Bayes model this type of (approx. circular) decision boundary?

In a recent exam on machine learning I came across the following question: "Which of the following techniques can model the decision boundary depicted in the figure? (check all that apply)" See my ...
2
votes
0answers
60 views

How to compute the likelihood when uncertain about underlying distribution

In a Bayesian approach, when I have no previous idea about the likelihood of a given event, can I simply gather a lot of data and use that distribution as my likelihood (1) and as I gather more and ...
1
vote
0answers
664 views

Can Naive Bayes fit non-linear decision boundaries?

I am confused about Naive Bayes. Some sources e.g. here suggest only linear decision boundaries can be fit. Others here on SE or here on CV say or just suspect they can be non-linear. What is true? ...
0
votes
0answers
187 views

Naive Bayes, how to get conditional probability of continuous value

I know for nominal value, it is easy to get the conditional probability $\hat{p}(x_i|c)$. But for continuous value, as far as I know, I can assume $\hat{p}(x_i|c)\sim{\mathcal{N}(\mu_{c,i},\sigma^2_{c,...
2
votes
0answers
172 views

Is the LogSumExp trick really necessary for Naive Bayes? [duplicate]

The LogSumExp trick is often introduced in the context of Naive Bayes, where the computation of the class posteriors would lead to underflow. Specifically, we need to compute $$ p(y=c|\mathbf{x}) = \...
3
votes
2answers
2k views

Bayes classifier?

I'm trying to understand the Bayes Classifier. I don't really understand its purpose or how to apply it, but I think I understand the parts of the formula: $$P(Y = j \mid X = x_{0})$$ If I'm correct,...
1
vote
1answer
450 views

Condition of applying Naive Bayes classifier

I am confused with the condition of independence. According to Wiki https://en.wikipedia.org/wiki/Naive_Bayes_classifier, Naive Bayes assumes strong independence between features, so that for a given ...
4
votes
1answer
9k views

Difference between Bayes classifier, KNN classifier and Naive Bayes Classifier

Question 1 :- Is there any difference between Bayes Classifier and Naive Bayes Classifier ? Is there any fundamental difference ? I searched the web and unable to find a good solution. Question 2 :- ...
0
votes
1answer
1k views

How do I train sklearn model to predict square of an integer

I created sample training data with set of random numbers and their squares. But when I predict square of a new number, none of the sklearn models are predicting it correctly. Given below is my sample ...
2
votes
1answer
191 views

Naive Bays or Naive Bayes?

I have come across a number of papers that write "naive Bays" instead of "naive Bayes". Are these different algorithms or just a popular typo?
3
votes
2answers
82 views

conditional probability calculation confusion

Suppose I have 3 friends, each of them are independent and each of them have the probability of p to tell the truth about something (e.g. company, weather, stock, etc.). Suppose something happened (e....
4
votes
1answer
10k views

How to interpret the outcome tables of a naive Bayes classifier

I have calculated a Naive Bayes with the klaR package. The vector "RandomAssignment" is representing the 7 Categories. "ydf" is a data frame which represents the underlying data. ...
1
vote
1answer
5k views

Test/training data set split for Naive Bayes classifier after model finalized

I've been learning about Naive Bayes classifiers using the nltk package in Python. I'm working on a gender classification model. I have some labeled data for names with male/female probabilities, and ...
1
vote
0answers
64 views

How could I use a Bayes classifier to categorise emails?

I've been looking into using Bayes to classify incoming emails to one of several distinct "owners" (so more complex than a spam filter that only has two outcomes). I don't have a stats background, so ...
0
votes
1answer
196 views

Classification: training sets different sizes

I'm building a classifier for text analysis sentiment. I have a large training set for positive, neutral and negative mentions. Should the training data sets be similar in size? Currently my ...
0
votes
1answer
876 views

Regarding probabilites for naiveBayes algo

I have trained my data with naiveBayes algo in e1071 package. I have 6 classes in my data. I have predicted test data. the prediction returns only one class for each data point but I would like to ...
2
votes
1answer
532 views

Twitter Classification of tweets related to Ebola into 21 custom categories

I have a lot of twitter data (4GB) related to keyword Ebola. I want to classify the tweets into 21 categories. Categories :- Death - tweet is about death Health Care Workers - tweet is about ...
0
votes
1answer
278 views

Naive Bayes using 1d features vs one n-D features with Kernel Density Estimation - Independence assumption

Given a set of features $x_1,x_2,x_3, ... \in \mathbb{R}$ and output class variable $y \in \mathbb{R}$ I could do Naive Bayes using the independence assumption of $x_1, x_2, x_3, ...$ to predict the ...
4
votes
1answer
1k views

Analogy between Neural network and naive bayes

I am trying to understand the analogy between a single layer neural network and naive Bayes classifier. Particularly, I want to know if, in a neural network, the variables are independent given the ...

1
4 5
6
7 8
12