Questions tagged [naive-bayes]
A naive Bayes classifier is a simple probabilistic classifier based on applying Bayes' theorem with strong independence assumptions. A more descriptive term for the underlying probability model would be "independent feature model".
1
vote
0answers
30 views
Naive Bayes linearity [duplicate]
By theory I know that Naive Bayes Classifier is a linear one, but when I implemented the decision boundary it was a curve (not linear as shown below). Is there any explanation why this is happening? ...
0
votes
1answer
27 views
How valid is this Stacking Model (input features to weak learners are different)?
I have a set of features with 6 of them being categorical, 1 continuous and 2 textual in type.
I have to predict the labels ( 10 in number) for them.
I tried applying several models and came to a ...
1
vote
1answer
51 views
Counting frequencies in Multinomial Naive Bayes
I have noticed some ambiguity/inconsistency in how various authors calculate the p(word|label) estimate in Multinomial Naive Bayes. In some cases, this value is calculated by counting words without ...
0
votes
2answers
41 views
Derivation of the formula for the probability of a class, given conditionally independent attributes
The following is a formula that finds the posterior probability of a class (i.e. yes or no) given four conditionally independent attributes:
$$P(c|X) = P(x_1|c)\cdot P(x_2|c)\cdot P(x_3|c)\cdot P(x_4|...
1
vote
1answer
19 views
Calculating priors with large number of classes
I'm trying to classify users into ~1K different groups. I'm trying to build a MAP classifier and have estimated my prior and posterior distributions using large amounts of data. The issue that I've ...
1
vote
1answer
26 views
Gaussian Naive Bayes Classifier
There are three nice Bayes classifier techniques: Bernoulli, Multinomial, and Gaussian
If we have a dataset whose samples have continuous-valued features, then Gaussian Bayes Classifier is used.
In ...
2
votes
0answers
26 views
Determining the decision boundary for Naive Bayes
I'd like to know if this is a sensible idea and if there exist any already formed methods to do this (I'm new to the data science area).
Essentially, I have used Naive Bayes to accurately classify ...
0
votes
1answer
27 views
manual implementation of Gaussian naive bayesian returns posterior larger than 1
I try to implement Gaussian naive bayesian manually in R. I test my model on iris data set. I would like to build a predictive model. That is, I would like to ...
1
vote
1answer
35 views
Naive bayes computation of denominator
I'm wondering about the denominator in this computation :
...
0
votes
0answers
29 views
0
votes
0answers
13 views
Naive Bayes missclassification rate across classes
I have a dataset with income, age sex and education as categorical features. I used R to create a Naive Bayes classifier as follows: income ~ age + sex + education.
I got the following a-priori and ...
0
votes
0answers
16 views
Why is it easier to incorporate arbitrary features into discriminative models?
It is often stated, that when arbitrary features are implied, generative models (e.g. Naive Bayes) are a lesser fit than discriminative ones, mainly for being harder to build. How would you elucidate ...
0
votes
0answers
22 views
Classification with ONLY categorical data
Suppose I have a table with some factor characteristics of some plants. For instance, petal color, pollen color, and so on. What is the best way to classify that data? Is it feasible to use some of ...
0
votes
1answer
22 views
Are there useful applications for Bayes Nets (vs. Naive Bayes)?
I am trying to learn about Bayesian networks and try to make them work in the context of a simple prediction problem. But my question is more theoretical:
For argument's sake, assume we have a ...
0
votes
1answer
13 views
How to make a prediction with Bayes Classifier after computing MLE?
I'm trying to figure out the role of computing the MLE for classification/prediction purposes with the Bayes Classifier.
Let's say I'm given a set of data assumed to be Gaussian. I can then compute ...
1
vote
1answer
21 views
How to validate classification model with ordinal information
I have a Naive Bayes model that predicts 3 classes. As you increase each class it means that the condition is more severe. 0 means no condition, 1 is concern and 2 is that they have the condition. I ...
0
votes
0answers
43 views
Why do the posterior probabilities violate the axioms of probability when we apply Bayesian update without likelihood computation?
Suppose that the unknown parameter $\Theta$ is Bernoulli and we make $n$ observations $X_1,X_2,\ldots,X_n$, which are continuous random variables. Assuming that $X_1,X_2,\ldots,X_n$ are conditionally ...
2
votes
1answer
424 views
Why Naive Bayes classifier is known to be a bad estimator?
In scikit-learn documentation page for Naive Bayes, it states that: On the flip side, although naive Bayes is known as a decent classifier, it is known to be a bad estimator, so the probability ...
0
votes
0answers
10 views
Feature engineering for verb\non-verb classification
Suppose we have data of X = words, and for each word we have a label indicating whether the word is a verb or a non-verb. So, y = labels.
Assuming we can build all the unigrams and bigrams of each of ...
0
votes
0answers
41 views
Naive Bayes Feature Selection based on KL-Divergence
I was reading this research paper (link below). There's a section where KL divergence is used by Naive Bayes assign to a document 'd', a class 'c' such that d is most similar to c. In other words, the ...
0
votes
2answers
54 views
Interpretation of Naive Bayes Probabilities
I'm dealing with a Naive Bayes approach to a Multiclass Classification problem with 9 different classes in the target variable. Let's assume the following: I've fitted a model to my training data and ...
0
votes
0answers
37 views
Connections between logistic regression, information value and Kullback-Leibler
Suppose that we are interested in modeling a binary predictor $Y=0,1$ subject to $m$ predictors $x_1,...,x_m$. First, let us examine a simpler model of the impact of $x_j$ on the response $Y$. By the ...
2
votes
0answers
73 views
How to compute gain statistic for the multinomial Naive Bayes classifier from Jurafsky and Martin (2018)
I'm trying to figure out how to compute the gain statistic G(w) following the fitting of the multinomial Naive Bayes model. This statistic is described on p17 of the new edition of Jurafsky and ...
0
votes
0answers
66 views
Laymen's description of Laplace Smoothing
I have understood that Laplace Smoothing will provide a small non-zero value to the probability score. But still, I am missing something. Please if anyone can provide a better description, it will be ...
1
vote
1answer
30 views
Summing Posterior Probability of Naive Bayes
I'm trying to work through a toy example of Naive Bayes with text classification (spam/ham) to make sure I understand the intuition, but not understanding why my posterior probabilities are not ...
0
votes
1answer
24 views
Training data grow with K-NN and Naive Bayes
My doubt is about the grow of the training data using K-NNs and Naive Bayes. As it grows larger, does prediction (on test data) become also computationally harder?
0
votes
1answer
40 views
Naive Bayes compute P(y|X) with conditional prob-ability
Does the Naive Bayes algorithm accurately compute P(y|X), i.e., the conditional prob-ability of class given features?
0
votes
1answer
138 views
When does Bayesian classifier act as linear classifier?
I am reviewing my lectures in Machine Learning and my current topic is Bayesian Classifier. The context is the classification of two classes C1 and C2.
My book (neural networks and learning machines ...
1
vote
0answers
110 views
klaR NaiveBayes warnings “Numerical 0 probability for all classes…”
Relatively new to both R and classification, so sorry if this is a dumb question.
I'm working on a "first classification" level project using this dataset. My code below for reference.
...
0
votes
0answers
33 views
Bayes Rule Bayesian Risk and Decision
Good day,
When attempting this problem I came across some difficulties.
A humanitarian charity wishes to classify a village as being at either high or low risk of flooding. The following ...
0
votes
0answers
15 views
naive bayes text classification: can I look at the individual word probabilities?
Assume I use the Naive Bayes classification algorithm. My question is simple: can I rank the words according to their posterior probabilities? I want to have a measure of "importance" of the words in ...
1
vote
1answer
39 views
Naive Bayes Classifier Unclear
I read the following sentence regarding the Naive Bayes Classifier:
If large number of features have relatively minor effects, taken together, their combined impact could be quite large.
Could ...
1
vote
1answer
31 views
Measuring the contribution of word to a classification in a Naive Bayes document classifier
I have a multinomial Naive Bayes document classifier. I'm interested in knowing the contribution made by each word to a single classification.
That is, I'd like to be able to measure which words in ...
0
votes
0answers
41 views
Measure Naive Bayesian Classifier certainty and error of prediction
So I am fairly new to this so please be patient :)
I am using Naive Bayesian Classifier to find the probability of a class (Yes), then I use this probability in another process. Now I am being asked ...
2
votes
2answers
114 views
Understanding the parameters needed for a distribution in Bayes networks?
Since I have a discriminative mindset hardly can I intuit the so-called parameters needed to specify a distribution in a generative Bayesian Network. I'd like to borrow an example from this blog. If ...
0
votes
2answers
444 views
Maximum likelihood in Naive Bayes classifier
With regards to the Naive Bayes classificator, I have read the following in Wikipedia and wanted to know why it is like that:
"In many practical applications, parameter estimation for naive Bayes ...
1
vote
0answers
91 views
Naive Bayes and kNN accuracy
Assume that a large number of binary features are added to a dataset with two
class labels c1 and c2, such that for each added feature f, the class conditional
probability P(f = 0|c1) = P(f = 0|c2). ...
0
votes
1answer
51 views
What is the interpretation for the priors in the derivation of Laplace smoothing?
Laplace smoothing has a generalisation that can be justified with the use of Bayes formula. Let $f(x;\alpha,\beta)$ be the (non-normalised) beta distribution, i.e.
$$f(x;\alpha,\beta) = x^{\alpha-1}(...
0
votes
1answer
24 views
Bernoulli distribution/ SOME probability/conjugate prior
I would like to know what "SOME probability of seeing tail" means in the second answer here. I.e. how much is it?
EDIT: I do not understand how can I see that there is SOME probability of seeing Tail ...
1
vote
3answers
346 views
Random Forests performs far less better than Naive Bayes
In one of my Machine Learning courses I have to find the best predictor for this dataset and its binary target "Caesarian".
First of all, I tried to improve the datas : there are few features. I did ...
1
vote
1answer
209 views
Naïve Bayes vs Bayes classifier
While studying the naive Bayes classifier, two questions came to mind:
When (if this can happen) is the naive Bayes classifier equivalent to the Bayes classifier?
Why do I usually read in textbooks ...
0
votes
0answers
33 views
Intuitive logic behind Naive Bayes and Bayes Theorem. Why does Naive Bayes multiply/input prior probability twice?
In order to calculate Naive Bayes, we combine prior probability with test evidence and obtain posterior probability.
When calculating test instance, we combine the probability of the test with the ...
0
votes
1answer
157 views
R Naive Bayes and Laplace: Even turned off, works fine with unseen words in test data?
I'm trying to better understand Laplace+1 smoothing on Naive Bayes for text classification. Using the e1071 package in R, naiveBayes() function, I get some confusing results.
If I fit a model using ...
2
votes
0answers
46 views
My roc is low while precision and recall are high.Why is it so?
I bulit a naive bayes classifier from 60k vectors of text and each of the text is a 2000 dimension vector(Used bag of words for vectorization). Used simple cross validator to find the best alpha and ...
1
vote
1answer
21 views
When will Naive Bayes misclassify continuous training instances?
For the discrete case, we can say that Naive Bayes might misclassify training data due to things like the zero-frequency problem.
Why might Naive Bayes misclassify continuous training data?
0
votes
0answers
64 views
Naive Bayes where posterior almost always equals prior (in MATLAB)
I have trained a naive Bayes classifier in MATLAB using fitcnb (description link) and 11 variables, seven of which are numeric (normal) and four of which are categorical ("mvmn" distribution name). I ...
0
votes
0answers
28 views
Why is my version of naive bayes not working as well as the one from sklearn?
I've implemented my own version of the bernoulli naive bayes algorithm. However, its performance is not as good as the sklearn version. Could anyone explain how I can improve my code?
...
0
votes
0answers
23 views
possible to identify (2 or more) phases (left/right step) with SVM?
I am new to the field of machine learning and I am trying to create a tool which is able to identify if a left or a right step is performed on a wearable sensor which consists out of 3 sensors (...
3
votes
1answer
380 views
Why does Naive Bayes work better when the number of features >> sample size compared to more sophisticated ML algorithms?
According to this article
Because of the class independence assumption, naive Bayes classifiers
can quickly learn to use high dimensional features with limited
training data compared to more ...
0
votes
0answers
19 views
Classifying web pages into page type accurately
I'm trying to classify webpages as either "login", "registration/create an account", "contact us", "forgot password" for example.
My approach is the following:
Obtain plaintext of the page (just the ...
