Questions tagged [naive-bayes]
A naive Bayes classifier is a simple probabilistic classifier based on applying Bayes' theorem with strong independence assumptions. A more descriptive term for the underlying probability model would be "independent feature model".
0
votes
0answers
12 views
Naive Bayes missclassification rate across classes
I have a dataset with income, age sex and education as categorical features. I used R to create a Naive Bayes classifier as follows: income ~ age + sex + education.
I got the following a-priori and ...
0
votes
0answers
15 views
Why is it easier to incorporate arbitrary features into discriminative models?
It is often stated, that when arbitrary features are implied, generative models (e.g. Naive Bayes) are a lesser fit than discriminative ones, mainly for being harder to build. How would you elucidate ...
0
votes
0answers
18 views
Classification with ONLY categorical data
Suppose I have a table with some factor characteristics of some plants. For instance, petal color, pollen color, and so on. What is the best way to classify that data? Is it feasible to use some of ...
0
votes
1answer
19 views
Are there useful applications for Bayes Nets (vs. Naive Bayes)?
I am trying to learn about Bayesian networks and try to make them work in the context of a simple prediction problem. But my question is more theoretical:
For argument's sake, assume we have a ...
0
votes
1answer
12 views
How to make a prediction with Bayes Classifier after computing MLE?
I'm trying to figure out the role of computing the MLE for classification/prediction purposes with the Bayes Classifier.
Let's say I'm given a set of data assumed to be Gaussian. I can then compute ...
1
vote
1answer
19 views
How to validate classification model with ordinal information
I have a Naive Bayes model that predicts 3 classes. As you increase each class it means that the condition is more severe. 0 means no condition, 1 is concern and 2 is that they have the condition. I ...
0
votes
0answers
42 views
Why do the posterior probabilities violate the axioms of probability when we apply Bayesian update without likelihood computation?
Suppose that the unknown parameter $\Theta$ is Bernoulli and we make $n$ observations $X_1,X_2,\ldots,X_n$, which are continuous random variables. Assuming that $X_1,X_2,\ldots,X_n$ are conditionally ...
2
votes
1answer
132 views
Why Naive Bayes classifier is known to be a bad estimator?
In scikit-learn documentation page for Naive Bayes, it states that: On the flip side, although naive Bayes is known as a decent classifier, it is known to be a bad estimator, so the probability ...
0
votes
0answers
9 views
Feature engineering for verb\non-verb classification
Suppose we have data of X = words, and for each word we have a label indicating whether the word is a verb or a non-verb. So, y = labels.
Assuming we can build all the unigrams and bigrams of each of ...
0
votes
0answers
21 views
Naive Bayes Feature Selection based on KL-Divergence
I was reading this research paper (link below). There's a section where KL divergence is used by Naive Bayes assign to a document 'd', a class 'c' such that d is most similar to c. In other words, the ...
0
votes
2answers
43 views
Interpretation of Naive Bayes Probabilities
I'm dealing with a Naive Bayes approach to a Multiclass Classification problem with 9 different classes in the target variable. Let's assume the following: I've fitted a model to my training data and ...
0
votes
0answers
30 views
Connections between logistic regression, information value and Kullback-Leibler
Suppose that we are interested in modeling a binary predictor $Y=0,1$ subject to $m$ predictors $x_1,...,x_m$. First, let us examine a simpler model of the impact of $x_j$ on the response $Y$. By the ...
2
votes
0answers
66 views
How to compute gain statistic for the multinomial Naive Bayes classifier from Jurafsky and Martin (2018)
I'm trying to figure out how to compute the gain statistic G(w) following the fitting of the multinomial Naive Bayes model. This statistic is described on p17 of the new edition of Jurafsky and ...
0
votes
0answers
28 views
Laymen's description of Laplace Smoothing
I have understood that Laplace Smoothing will provide a small non-zero value to the probability score. But still, I am missing something. Please if anyone can provide a better description, it will be ...
0
votes
1answer
18 views
Summing Posterior Probability of Naive Bayes
I'm trying to work through a toy example of Naive Bayes with text classification (spam/ham) to make sure I understand the intuition, but not understanding why my posterior probabilities are not ...
0
votes
1answer
23 views
Training data grow with K-NN and Naive Bayes
My doubt is about the grow of the training data using K-NNs and Naive Bayes. As it grows larger, does prediction (on test data) become also computationally harder?
0
votes
1answer
39 views
Naive Bayes compute P(y|X) with conditional prob-ability
Does the Naive Bayes algorithm accurately compute P(y|X), i.e., the conditional prob-ability of class given features?
0
votes
1answer
73 views
When does Bayesian classifier act as linear classifier?
I am reviewing my lectures in Machine Learning and my current topic is Bayesian Classifier. The context is the classification of two classes C1 and C2.
My book (neural networks and learning machines ...
0
votes
0answers
56 views
klaR NaiveBayes warnings “Numerical 0 probability for all classes…”
Relatively new to both R and classification, so sorry if this is a dumb question.
I'm working on a "first classification" level project using this dataset. My code below for reference.
...
0
votes
0answers
32 views
Bayes Rule Bayesian Risk and Decision
Good day,
When attempting this problem I came across some difficulties.
A humanitarian charity wishes to classify a village as being at either high or low risk of flooding. The following ...
0
votes
0answers
14 views
naive bayes text classification: can I look at the individual word probabilities?
Assume I use the Naive Bayes classification algorithm. My question is simple: can I rank the words according to their posterior probabilities? I want to have a measure of "importance" of the words in ...
1
vote
1answer
35 views
Naive Bayes Classifier Unclear
I read the following sentence regarding the Naive Bayes Classifier:
If large number of features have relatively minor effects, taken together, their combined impact could be quite large.
Could ...
1
vote
0answers
19 views
Measuring the contribution of word to a classification in a Naive Bayes document classifier
I have a multinomial Naive Bayes document classifier. I'm interested in knowing the contribution made by each word to a single classification.
That is, I'd like to be able to measure which words in ...
0
votes
0answers
29 views
Measure Naive Bayesian Classifier certainty and error of prediction
So I am fairly new to this so please be patient :)
I am using Naive Bayesian Classifier to find the probability of a class (Yes), then I use this probability in another process. Now I am being asked ...
2
votes
2answers
86 views
Understanding the parameters needed for a distribution in Bayes networks?
Since I have a discriminative mindset hardly can I intuit the so-called parameters needed to specify a distribution in a generative Bayesian Network. I'd like to borrow an example from this blog. If ...
0
votes
2answers
242 views
Maximum likelihood in Naive Bayes classifier
With regards to the Naive Bayes classificator, I have read the following in Wikipedia and wanted to know why it is like that:
"In many practical applications, parameter estimation for naive Bayes ...
1
vote
0answers
49 views
Naive Bayes and kNN accuracy
Assume that a large number of binary features are added to a dataset with two
class labels c1 and c2, such that for each added feature f, the class conditional
probability P(f = 0|c1) = P(f = 0|c2). ...
0
votes
1answer
36 views
What is the interpretation for the priors in the derivation of Laplace smoothing?
Laplace smoothing has a generalisation that can be justified with the use of Bayes formula. Let $f(x;\alpha,\beta)$ be the (non-normalised) beta distribution, i.e.
$$f(x;\alpha,\beta) = x^{\alpha-1}(...
0
votes
1answer
24 views
Bernoulli distribution/ SOME probability/conjugate prior
I would like to know what "SOME probability of seeing tail" means in the second answer here. I.e. how much is it?
EDIT: I do not understand how can I see that there is SOME probability of seeing Tail ...
1
vote
3answers
198 views
Random Forests performs far less better than Naive Bayes
In one of my Machine Learning courses I have to find the best predictor for this dataset and its binary target "Caesarian".
First of all, I tried to improve the datas : there are few features. I did ...
0
votes
1answer
120 views
Naïve Bayes vs Bayes classifier
While studying the naive Bayes classifier, two questions came to mind:
When (if this can happen) is the naive Bayes classifier equivalent to the Bayes classifier?
Why do I usually read in textbooks ...
0
votes
0answers
26 views
Intuitive logic behind Naive Bayes and Bayes Theorem. Why does Naive Bayes multiply/input prior probability twice?
In order to calculate Naive Bayes, we combine prior probability with test evidence and obtain posterior probability.
When calculating test instance, we combine the probability of the test with the ...
0
votes
1answer
129 views
R Naive Bayes and Laplace: Even turned off, works fine with unseen words in test data?
I'm trying to better understand Laplace+1 smoothing on Naive Bayes for text classification. Using the e1071 package in R, naiveBayes() function, I get some confusing results.
If I fit a model using ...
2
votes
0answers
43 views
My roc is low while precision and recall are high.Why is it so?
I bulit a naive bayes classifier from 60k vectors of text and each of the text is a 2000 dimension vector(Used bag of words for vectorization). Used simple cross validator to find the best alpha and ...
1
vote
1answer
19 views
When will Naive Bayes misclassify continuous training instances?
For the discrete case, we can say that Naive Bayes might misclassify training data due to things like the zero-frequency problem.
Why might Naive Bayes misclassify continuous training data?
0
votes
0answers
53 views
Naive Bayes where posterior almost always equals prior (in MATLAB)
I have trained a naive Bayes classifier in MATLAB using fitcnb (description link) and 11 variables, seven of which are numeric (normal) and four of which are categorical ("mvmn" distribution name). I ...
0
votes
0answers
24 views
Why is my version of naive bayes not working as well as the one from sklearn?
I've implemented my own version of the bernoulli naive bayes algorithm. However, its performance is not as good as the sklearn version. Could anyone explain how I can improve my code?
...
0
votes
0answers
23 views
possible to identify (2 or more) phases (left/right step) with SVM?
I am new to the field of machine learning and I am trying to create a tool which is able to identify if a left or a right step is performed on a wearable sensor which consists out of 3 sensors (...
2
votes
1answer
280 views
Why does Naive Bayes work better when the number of features >> sample size compared to more sophisticated ML algorithms?
According to this article
Because of the class independence assumption, naive Bayes classifiers
can quickly learn to use high dimensional features with limited
training data compared to more ...
0
votes
0answers
18 views
Classifying web pages into page type accurately
I'm trying to classify webpages as either "login", "registration/create an account", "contact us", "forgot password" for example.
My approach is the following:
Obtain plaintext of the page (just the ...
0
votes
0answers
64 views
Accuracy Measure for Bayesian Network
I was trying with INSURANCE data in bnlearn. For measuring the model accuracy, I tried with different nodes for prediction. With each node, the prediction shows different accuracy rate. So, my doubt ...
3
votes
1answer
296 views
Should PCA be (always) done before Naive Bayes classification
According to Wikipedia page on Naive Bayes:
.. Naive Bayes classifiers are a family of simple "probabilistic
classifiers" based on applying Bayes' theorem with strong (naive)
independence ...
1
vote
0answers
44 views
Exploit grouped features for a classification problem
I've got a dataset of 100.000 labelled vectors, each one with a set of 100 binary features: $$F = \{ f_{1}, f_{2}, ..., f_{100} \}.$$
I am able to build a Bernoulli Naive Bayes classifier for ...
0
votes
0answers
61 views
When coding categorical predictors as dummy variables for a Naive Bayes classifier, should the reference levels be left out?
Perhaps another way of putting the question would be: "Do Naive Bayes classifiers have intercept terms?"
My statistical training tells me that a predictor with $k$ categories should be coded as $k-1$ ...
1
vote
0answers
19 views
Review 5-stars Multiclassification Model
So I'm super new to DataScience World. And I'm Trying to do a TextMining Work.
My goal is by reading user's reviews to predict their rating to a tech product.
Problem? Multiclassification model with ...
2
votes
2answers
556 views
Trouble understanding Bayes Theorem
I was watching a video on YouTube and i am not sure if the given solution is correct. Can someone confirm?
0
votes
0answers
42 views
How do document lengths affect Gaussian Naive Bayes?
I'm trying to understand Gaussian Naive Bayes.
I am training on a pre-processed subset of the 20 Newsgroup data. Each observation is around 500 attributes (words), and 1 class (of 5 possible).
I ...
0
votes
0answers
23 views
Testing if word-count vectors follow a multinomial distribution
I am attempting to make a Naive Bayes classifier for word count vectors (each document is represented as a vector of word counts). For this, I am using SciKit-Learn's MultinomialNB.
From what I ...
1
vote
1answer
756 views
Proving that Gaussian Naive Bayes Decision Boundary is Linear
I need to come up with a Proof that Gaussian Naive Bayes has a linear decision boundary (In this case for Y={0,1})
I tried to work it out, but I am not able to pull out the xi term as it is stuck in ...
0
votes
0answers
27 views
Predicting a combination
Question
Suppose we have a training set of families. Where each family is defined as such…
Family: A list of integers. Each integer is the age of one of the family members.
(e.g. with a 45 year ...
