Stack Exchange Network

Stack Exchange network consists of 175 Q&A communities including Stack Overflow, the largest, most trusted online community for developers to learn, share their knowledge, and build their careers.

Visit Stack Exchange

Questions tagged [naive-bayes]

A naive Bayes classifier is a simple probabilistic classifier based on applying Bayes' theorem with strong independence assumptions. A more descriptive term for the underlying probability model would be "independent feature model".

0
votes
0answers
12 views

Naive Bayes missclassification rate across classes

I have a dataset with income, age sex and education as categorical features. I used R to create a Naive Bayes classifier as follows: income ~ age + sex + education. I got the following a-priori and ...
0
votes
0answers
15 views

Why is it easier to incorporate arbitrary features into discriminative models?

It is often stated, that when arbitrary features are implied, generative models (e.g. Naive Bayes) are a lesser fit than discriminative ones, mainly for being harder to build. How would you elucidate ...
0
votes
0answers
18 views

Classification with ONLY categorical data

Suppose I have a table with some factor characteristics of some plants. For instance, petal color, pollen color, and so on. What is the best way to classify that data? Is it feasible to use some of ...
0
votes
1answer
19 views

Are there useful applications for Bayes Nets (vs. Naive Bayes)?

I am trying to learn about Bayesian networks and try to make them work in the context of a simple prediction problem. But my question is more theoretical: For argument's sake, assume we have a ...
0
votes
1answer
12 views

How to make a prediction with Bayes Classifier after computing MLE?

I'm trying to figure out the role of computing the MLE for classification/prediction purposes with the Bayes Classifier. Let's say I'm given a set of data assumed to be Gaussian. I can then compute ...
1
vote
1answer
19 views

How to validate classification model with ordinal information

I have a Naive Bayes model that predicts 3 classes. As you increase each class it means that the condition is more severe. 0 means no condition, 1 is concern and 2 is that they have the condition. I ...
0
votes
0answers
42 views

Why do the posterior probabilities violate the axioms of probability when we apply Bayesian update without likelihood computation?

Suppose that the unknown parameter $\Theta$ is Bernoulli and we make $n$ observations $X_1,X_2,\ldots,X_n$, which are continuous random variables. Assuming that $X_1,X_2,\ldots,X_n$ are conditionally ...
2
votes
1answer
132 views

Why Naive Bayes classifier is known to be a bad estimator?

In scikit-learn documentation page for Naive Bayes, it states that: On the flip side, although naive Bayes is known as a decent classifier, it is known to be a bad estimator, so the probability ...
0
votes
0answers
9 views

Feature engineering for verb\non-verb classification

Suppose we have data of X = words, and for each word we have a label indicating whether the word is a verb or a non-verb. So, y = labels. Assuming we can build all the unigrams and bigrams of each of ...
0
votes
0answers
21 views

Naive Bayes Feature Selection based on KL-Divergence

I was reading this research paper (link below). There's a section where KL divergence is used by Naive Bayes assign to a document 'd', a class 'c' such that d is most similar to c. In other words, the ...
0
votes
2answers
43 views

Interpretation of Naive Bayes Probabilities

I'm dealing with a Naive Bayes approach to a Multiclass Classification problem with 9 different classes in the target variable. Let's assume the following: I've fitted a model to my training data and ...
0
votes
0answers
30 views

Connections between logistic regression, information value and Kullback-Leibler

Suppose that we are interested in modeling a binary predictor $Y=0,1$ subject to $m$ predictors $x_1,...,x_m$. First, let us examine a simpler model of the impact of $x_j$ on the response $Y$. By the ...
2
votes
0answers
66 views

How to compute gain statistic for the multinomial Naive Bayes classifier from Jurafsky and Martin (2018)

I'm trying to figure out how to compute the gain statistic G(w) following the fitting of the multinomial Naive Bayes model. This statistic is described on p17 of the new edition of Jurafsky and ...
0
votes
0answers
28 views

Laymen's description of Laplace Smoothing

I have understood that Laplace Smoothing will provide a small non-zero value to the probability score. But still, I am missing something. Please if anyone can provide a better description, it will be ...
0
votes
1answer
18 views

Summing Posterior Probability of Naive Bayes

I'm trying to work through a toy example of Naive Bayes with text classification (spam/ham) to make sure I understand the intuition, but not understanding why my posterior probabilities are not ...
0
votes
1answer
23 views

Training data grow with K-NN and Naive Bayes

My doubt is about the grow of the training data using K-NNs and Naive Bayes. As it grows larger, does prediction (on test data) become also computationally harder?
0
votes
1answer
39 views

Naive Bayes compute P(y|X) with conditional prob-ability

Does the Naive Bayes algorithm accurately compute P(y|X), i.e., the conditional prob-ability of class given features?
0
votes
1answer
73 views

When does Bayesian classifier act as linear classifier?

I am reviewing my lectures in Machine Learning and my current topic is Bayesian Classifier. The context is the classification of two classes C1 and C2. My book (neural networks and learning machines ...
0
votes
0answers
56 views

klaR NaiveBayes warnings “Numerical 0 probability for all classes…”

Relatively new to both R and classification, so sorry if this is a dumb question. I'm working on a "first classification" level project using this dataset. My code below for reference. ...
0
votes
0answers
32 views

Bayes Rule Bayesian Risk and Decision

Good day, When attempting this problem I came across some difficulties. A humanitarian charity wishes to classify a village as being at either high or low risk of flooding. The following ...
0
votes
0answers
14 views

naive bayes text classification: can I look at the individual word probabilities?

Assume I use the Naive Bayes classification algorithm. My question is simple: can I rank the words according to their posterior probabilities? I want to have a measure of "importance" of the words in ...
1
vote
1answer
35 views

Naive Bayes Classifier Unclear

I read the following sentence regarding the Naive Bayes Classifier: If large number of features have relatively minor effects, taken together, their combined impact could be quite large. Could ...
1
vote
0answers
19 views

Measuring the contribution of word to a classification in a Naive Bayes document classifier

I have a multinomial Naive Bayes document classifier. I'm interested in knowing the contribution made by each word to a single classification. That is, I'd like to be able to measure which words in ...
0
votes
0answers
29 views

Measure Naive Bayesian Classifier certainty and error of prediction

So I am fairly new to this so please be patient :) I am using Naive Bayesian Classifier to find the probability of a class (Yes), then I use this probability in another process. Now I am being asked ...
2
votes
2answers
86 views

Understanding the parameters needed for a distribution in Bayes networks?

Since I have a discriminative mindset hardly can I intuit the so-called parameters needed to specify a distribution in a generative Bayesian Network. I'd like to borrow an example from this blog. If ...
0
votes
2answers
242 views

Maximum likelihood in Naive Bayes classifier

With regards to the Naive Bayes classificator, I have read the following in Wikipedia and wanted to know why it is like that: "In many practical applications, parameter estimation for naive Bayes ...
1
vote
0answers
49 views

Naive Bayes and kNN accuracy

Assume that a large number of binary features are added to a dataset with two class labels c1 and c2, such that for each added feature f, the class conditional probability P(f = 0|c1) = P(f = 0|c2). ...
0
votes
1answer
36 views

What is the interpretation for the priors in the derivation of Laplace smoothing?

Laplace smoothing has a generalisation that can be justified with the use of Bayes formula. Let $f(x;\alpha,\beta)$ be the (non-normalised) beta distribution, i.e. $$f(x;\alpha,\beta) = x^{\alpha-1}(...
0
votes
1answer
24 views

Bernoulli distribution/ SOME probability/conjugate prior

I would like to know what "SOME probability of seeing tail" means in the second answer here. I.e. how much is it? EDIT: I do not understand how can I see that there is SOME probability of seeing Tail ...
1
vote
3answers
198 views

Random Forests performs far less better than Naive Bayes

In one of my Machine Learning courses I have to find the best predictor for this dataset and its binary target "Caesarian". First of all, I tried to improve the datas : there are few features. I did ...
0
votes
1answer
120 views

Naïve Bayes vs Bayes classifier

While studying the naive Bayes classifier, two questions came to mind: When (if this can happen) is the naive Bayes classifier equivalent to the Bayes classifier? Why do I usually read in textbooks ...
0
votes
0answers
26 views

Intuitive logic behind Naive Bayes and Bayes Theorem. Why does Naive Bayes multiply/input prior probability twice?

In order to calculate Naive Bayes, we combine prior probability with test evidence and obtain posterior probability. When calculating test instance, we combine the probability of the test with the ...
0
votes
1answer
129 views

R Naive Bayes and Laplace: Even turned off, works fine with unseen words in test data?

I'm trying to better understand Laplace+1 smoothing on Naive Bayes for text classification. Using the e1071 package in R, naiveBayes() function, I get some confusing results. If I fit a model using ...
2
votes
0answers
43 views

My roc is low while precision and recall are high.Why is it so?

I bulit a naive bayes classifier from 60k vectors of text and each of the text is a 2000 dimension vector(Used bag of words for vectorization). Used simple cross validator to find the best alpha and ...
1
vote
1answer
19 views

When will Naive Bayes misclassify continuous training instances?

For the discrete case, we can say that Naive Bayes might misclassify training data due to things like the zero-frequency problem. Why might Naive Bayes misclassify continuous training data?
0
votes
0answers
53 views

Naive Bayes where posterior almost always equals prior (in MATLAB)

I have trained a naive Bayes classifier in MATLAB using fitcnb (description link) and 11 variables, seven of which are numeric (normal) and four of which are categorical ("mvmn" distribution name). I ...
0
votes
0answers
24 views

Why is my version of naive bayes not working as well as the one from sklearn?

I've implemented my own version of the bernoulli naive bayes algorithm. However, its performance is not as good as the sklearn version. Could anyone explain how I can improve my code? ...
0
votes
0answers
23 views

possible to identify (2 or more) phases (left/right step) with SVM?

I am new to the field of machine learning and I am trying to create a tool which is able to identify if a left or a right step is performed on a wearable sensor which consists out of 3 sensors (...
2
votes
1answer
280 views

Why does Naive Bayes work better when the number of features >> sample size compared to more sophisticated ML algorithms?

According to this article Because of the class independence assumption, naive Bayes classifiers can quickly learn to use high dimensional features with limited training data compared to more ...
0
votes
0answers
18 views

Classifying web pages into page type accurately

I'm trying to classify webpages as either "login", "registration/create an account", "contact us", "forgot password" for example. My approach is the following: Obtain plaintext of the page (just the ...
0
votes
0answers
64 views

Accuracy Measure for Bayesian Network

I was trying with INSURANCE data in bnlearn. For measuring the model accuracy, I tried with different nodes for prediction. With each node, the prediction shows different accuracy rate. So, my doubt ...
3
votes
1answer
296 views

Should PCA be (always) done before Naive Bayes classification

According to Wikipedia page on Naive Bayes: .. Naive Bayes classifiers are a family of simple "probabilistic classifiers" based on applying Bayes' theorem with strong (naive) independence ...
1
vote
0answers
44 views

Exploit grouped features for a classification problem

I've got a dataset of 100.000 labelled vectors, each one with a set of 100 binary features: $$F = \{ f_{1}, f_{2}, ..., f_{100} \}.$$ I am able to build a Bernoulli Naive Bayes classifier for ...
0
votes
0answers
61 views

When coding categorical predictors as dummy variables for a Naive Bayes classifier, should the reference levels be left out?

Perhaps another way of putting the question would be: "Do Naive Bayes classifiers have intercept terms?" My statistical training tells me that a predictor with $k$ categories should be coded as $k-1$ ...
1
vote
0answers
19 views

Review 5-stars Multiclassification Model

So I'm super new to DataScience World. And I'm Trying to do a TextMining Work. My goal is by reading user's reviews to predict their rating to a tech product. Problem? Multiclassification model with ...
2
votes
2answers
556 views

Trouble understanding Bayes Theorem

I was watching a video on YouTube and i am not sure if the given solution is correct. Can someone confirm?
0
votes
0answers
42 views

How do document lengths affect Gaussian Naive Bayes?

I'm trying to understand Gaussian Naive Bayes. I am training on a pre-processed subset of the 20 Newsgroup data. Each observation is around 500 attributes (words), and 1 class (of 5 possible). I ...
0
votes
0answers
23 views

Testing if word-count vectors follow a multinomial distribution

I am attempting to make a Naive Bayes classifier for word count vectors (each document is represented as a vector of word counts). For this, I am using SciKit-Learn's MultinomialNB. From what I ...
1
vote
1answer
756 views

Proving that Gaussian Naive Bayes Decision Boundary is Linear

I need to come up with a Proof that Gaussian Naive Bayes has a linear decision boundary (In this case for Y={0,1}) I tried to work it out, but I am not able to pull out the xi term as it is stuck in ...
0
votes
0answers
27 views

Predicting a combination

Question Suppose we have a training set of families. Where each family is defined as such… Family: A list of integers. Each integer is the age of one of the family members. (e.g. with a 45 year ...