Questions tagged [naive-bayes]
A naive Bayes classifier is a simple probabilistic classifier based on applying Bayes' theorem with strong independence assumptions. A more descriptive term for the underlying probability model would be "independent feature model".
586
questions
0
votes
0
answers
15
views
Naive Bayes fits multiple hyperplanes in the case of multiclass classification problems?
Naive Bayes differentiates feature distributions given target labels, and intuitively, it fits a hyperplane to the given data set.
But I do not fully understand whether Naive Bayes fits multiple ...
2
votes
1
answer
56
views
Understanding the application of MLE in Naive Bayes
I was looking at the Naive Bayes classifier models (Binomial, Multinomial and Gaussian) and trying to understand the theory behind them a bit better, but am unsure if I understand the MLE approach ...
- 173
0
votes
0
answers
21
views
Population stability index and Text Data Length
I'm training a language detection model using:
a training set, classified between English and not English sentences or small paragraps, where the length of the sentences can vary
a score set that is ...
1
vote
1
answer
32
views
Assuming Independence in Naive Bayes
If features of the Naive Bayes are not independent then what are the consequences of the results?
- 19
0
votes
1
answer
36
views
Understanding rare definition of the likelihood function and corresponding posterior from research paper
Reading the paper https://storage.googleapis.com/pub-tools-public-publication-data/pdf/b20467a5c27b86c08cceed56fc72ceadb875184a.pdf i came across a rare definition of the likelihood function that in ...
1
vote
1
answer
33
views
interpreting confusion matrix results
I have a dataset on unemployed individuals enrolled in a job training program where I am trying to predict whether 6 months post-enrolment they 1) gain employment, 2) stay unemployed, or 3) drop out ...
- 63
2
votes
2
answers
21
views
If I engineer a new feature such that feature C = feature A/feature B, must I drop features A and B from a Gaussian Naive Bayes model?
As the question asks, is it bad data science not to drop the dividend and divisor features when creating a new feature that is their quotient when working with a Naive Bayes model? My understanding of ...
- 137
0
votes
0
answers
13
views
Is it possible for the gains line to fall below the naive rule in a lift chart?
I created a naive Bayes model and generated this lift chart . Is it possible that my model could underperform the naiveBayes rule?
2
votes
1
answer
78
views
AUC - Logistic Regression versus LDA, and Naive Bayes
everyone!
I am a newbie on machine learning, and I am now interested on classification modeling.
I used logistic regression, linear discriminant analysis (LDA), and naive Bayes on my notebook DataCamp ...
- 33
0
votes
0
answers
13
views
Minimize risk and add rejection to model
I want to minimize the risk of a Gaussian model with a cost for false negatives and false positives. The model uses Naive Bayes algorithm and solves a binary classification problem:
$$P(x_i \mid y) = \...
- 1
0
votes
0
answers
113
views
NaiveBayes in R - Understand importance of Variables
I am working with a data set where the response variable is binary and 15-20 continuous and categorical variables.
I am using the naiveBayes library to compute the model. I am interested in ...
0
votes
1
answer
172
views
How to handle missing values NaiveBayes Scikit Learn
I am working with a dataset which has 34 features (numerical, nominal) and the target class. Several of the columns have missing values, especially one column has approximately 50% missing values.
I ...
- 1
0
votes
0
answers
13
views
Find conditional independence between the attributes of a categorical dataset
I have a high dimensional data set. I used feature selection method to reduce the dimensionality of the dataset. Originally, the dataset has 120 attributes which I minimized to 80 attributes after ...
- 133
0
votes
0
answers
25
views
Why are linear/logistic regression and naive bayes called "parametric" while SVM, random forests, neural nets are not? [duplicate]
This table is mentioned in What algorithms need feature scaling, beside from SVM?
It says that linear regression, logistic regression, and naive bayes are parametric, while KNN, decision trees, ...
- 31
1
vote
1
answer
23
views
Impact of Laplace smoothing on likelihood in Naive Bayes
When 1 is added to word count in Laplace Smoothing in Naive Bayes, the new probabilities either increase or decrease as shown below.
Though the problem of "zero" probability has been solved.
...
- 11
0
votes
0
answers
13
views
How to go about using one machine learning method to 'augment' another?
I'm in a machine learning class, and we've been asked to create some sort of... machine learning solution to a problem, using two different forms of machine learning. I'm doing something pretty basic, ...
- 101
0
votes
1
answer
70
views
Use Naive Bayes to label unlabeled data
I have an Excel file that includes all product information (web scraped from Zalando) of 10k dresses. So for each dress/line I have multiple features available (brand, color, neckline, length...)
I ...
0
votes
0
answers
14
views
Why would including a feature reduce performance of Naive Bayes (multi class prediction)
I have a simple Naive Bayes model, 5 features, trying to classify observations into 1 of 3 classes. I add a 6th feature and the test set performance, from 58% correct classified to 54% correctly ...
0
votes
1
answer
92
views
Naive-Bayes Iris R, Correct Implementation? [closed]
So I am trying to understand the naive Bayes classifier by implementing it in R. However I'm not sure if my implementation is correct.
Using the iris dataset and Sepal Width / Length as features. ...
- 179
1
vote
0
answers
14
views
Log Naive Bayes NLP dropping the denominator [duplicate]
I'm learning about the the Naive Bayes classification and I don't get what the squiggly alpha sign means and what it means that "Denominator remains constant for given input." Is it because ...
- 11
0
votes
1
answer
94
views
Computing a prior from two components in Naive Bayes
Given a model parameter $\theta$ that is composed of two distributions in a Naive Bayes classifier, how is $P(\theta)$ typically computed in practice?
More specifically, from the article of Nigam et ...
- 230
0
votes
0
answers
90
views
Negative Log Likelihood and Derivative of Gaussian Naive Bayes Classifer
I am trying to derive negative log likelihood of Gaussian Naive Bayes classifier and the derivatives of the parameters.
So there are class labels $y \in {1, ..., k}$, and real valued vector of $d$ ...
- 1
0
votes
0
answers
38
views
How to improve accuracy of natural's Naive Bayes classifier?
Note: I'm new to Machine Learning and NLP. This is my first project in this field.
I'm using NaturalNode/natural (https://github.com/NaturalNode/natural) to build a chat bot to help my users with ...
- 101
1
vote
0
answers
15
views
Laplace Smoothing in Naive Bayes [duplicate]
I'm reading up on Laplace Smoothing/Add-1 Smoothing in Naive Bayes and I'm given the formula $ \frac{Count(Feature=Value) + α}{N + α\cdot k} $.
In reference to the image above, if we have to classify ...
- 11
1
vote
0
answers
21
views
Oscillation of AdaBoost Training error
Adaboost, using weak learners as Gaussian Naive bayes, has oscillating/unpredictable training error as we increase the number of weak learners. Is there a specific reason for this? Y-axis is the ...
- 29
3
votes
1
answer
281
views
What attributes does Laplace Smoothing apply on in Naive Bayes
Consider the dataset:
Outlook
Temperature
Humidity
Play Golf?
Overcast
Cool
Low
Yes
Sunny
Hot
Low
Yes
Rainy
Cool
High
No
Sunny
Hot
High
No
Rainy
Cool
Low
Yes
There are 3 possible values for the ...
0
votes
0
answers
18
views
What is the $\prod_{i=1}^dP(x_i|y=1)$ in the Naive Bayes classifier?
Consider the Naive Bayes classifier. Suppose that $x=(x_1, x_2, \dots, x_d)^T$ and $x_i\in \{0, 1\}$ with labels $y_i\in \{0,1\}$. Let $P(y=1)=p=1-P(y=0)$ and $P(x_i=1|y=0)=p_{i0}$, $P(x_i=1|y=1)=p_{...
- 432
0
votes
0
answers
33
views
Two models trained on the same data, with very similar performance statistics, give very different results on unseen data
I am trying to develop a natural language processing model. My goal is to be simple, and basically say if something is good or bad for a topic.
I have a training set made up of around 15000 sentences. ...
- 103
1
vote
0
answers
47
views
How do you interpret the matrix confusion in this Naïve Bayes output?
Why are my correctly classified instances lower than incorrectly classified?
This was tested using Naive Bayes with option testing Cross-Validation set at 10 folds.
Here is the image of the results: ...
- 11
1
vote
0
answers
146
views
How to calculate the Bayesian Risk Classifier
I'm not exactly sure how to calculate the Bayesian risk Classifier $L(r^*)$ for $Y\in\{ 0,1 \}$.
For this scenario, assume:
$X\in\mathbb{X}=[0,1],Y\in\{ 0,1 \}$
$\pi_y=P(Y=y)=1/2$ for $y\in{0,1}$
...
- 57
2
votes
1
answer
104
views
Stuck on a step calculating Naive Bayes Classifier,
Using the example at 3Blue1Brown I constructed a table to help me remember Bayes theorem
where L=Librarian and S =Shy. I understand that $$P(S,L) = P(S|L)P(L) = P(L|S)P(S) = \frac{4}{210}$$
I am ...
- 439
0
votes
0
answers
26
views
How to use K-fold classifier for comparison of different models
I am learning machine learning and went through a term K-fold cross validation.
I also took notes from this site to enhance my understanding.
As per the tutorial if it is 3 fold cross validation and ...
- 133
0
votes
0
answers
243
views
How many divergent transitions are too many?
I am running a Bayesian linear mixed effects analysis. Four chains for 3000 iterations. I end up with four divergent transitions. Is this too many or can I proceed? How do I know if it's too many? I'...
- 1,691
0
votes
0
answers
15
views
Naive Bayes for data generation
NB is a classification method which according to Bishop's book is categorized in probabilistic generative methods. As far as I understood you can learn a join distribution from input-output pairs and ...
- 85
1
vote
0
answers
15
views
Base rate calculation for customer conversion
Question: What is the base rate of conversion for mobile versus desktop sites?
Total no of customers: 590381 Out of 590381, the Total no of customers that were converted: 701
These customers used ...
- 11
7
votes
2
answers
924
views
Intuition for why LDA is a special case of naive Bayes
The naive Bayes classifier assumes the regressors to be mutually independent, while linear discriminant analysis (LDA) allows them to be correlated. James et al. "An Introduction to Statistical ...
- 57.8k
5
votes
1
answer
293
views
Why is naive Bayes overconfident?
In the fourth edition of "Artificial Intelligence: a modern approach" by Russel and Norvig, they write in section 12.6, regarding the Naive Bayes Model for text classification, the following:...
- 171
0
votes
1
answer
1k
views
How can Naive Bayes overfit the data?
I know that Laplace smoothing results in a high bias of Naive Bayes. If the value of the smoothing parameter (alpha) is large, then the probability distribution will be uniform for all the features. ...
- 55
2
votes
1
answer
225
views
Parameters in Naive Bayes
This is from https://scikit-learn.org/stable/modules/naive_bayes.html
In the last line it says
and we can use Maximum A Posteriori (MAP) estimation to estimate
$P(y)$ and $P(x_i|y)$; the former is ...
- 21
1
vote
1
answer
1k
views
Product of two normal distributions (for Bayes Rule) is not product of normal output variables?
When we apply Bayes' rule in machine learning, we want to compute the posterior probability $P(y|X)$ by multiplying two probability distributions (the observed class-conditional likelihood $P(X|y)$ ...
- 3,300
0
votes
1
answer
207
views
Logic of Sklearn Bernoulli Naive Bayes Classifier when the the predictors are not even binary?
I know the mathematics behind the Naive Baye's Bernoulli Classifier Algorithm and it is used to calculate the probabilistic results. As we know the Bernoulli Naive Bayes Classifier uses binary ...
0
votes
1
answer
75
views
Binary Classification with OpenNLP
I'm trying to apply binary classification with OpenNLP. I could already successfully classify movies by different genres. The data sample has the form:
...
- 3
2
votes
1
answer
92
views
Does it make sense to use Gaussian Naive Bayes for a single feature?
I understand that 'Naive' Bayes refers to the approach where all the features are assumed to be independent. But I want to evaluate the performance of each feature individually before I combine all of ...
- 183
-1
votes
1
answer
45
views
How to re-write with Bayes' Rule as a function?
I'm a little bit stuck. Say I have a P(A | B ^ C). How could I re-write this as a function with the terms P(B|A ^ C), P(B|C) and P(C|B) and can anyone explain how this was done?
- 1
2
votes
2
answers
245
views
Naive Bayes - having trouble coming up with a case where Laplace smoothing changes the prediction
I'm thinking through the logic of Naive Bayes and encountered a brain teaser. I know that adding smoothing (alpha) to Naive Bayes can help to increase the accuracy of the model, which implies that it ...
- 123
1
vote
1
answer
484
views
How to choose prior in Laplace Smoothing Naive Bayes?
Please check the image for reference
How to choose the prior probability of each feature .
Should we use the same prior for every feature or different features use different prior.
for example ...
- 13
0
votes
0
answers
173
views
How should I handle Laplace smoothing in Naive Bayes in this example?
I have a toy dataset on animals, with 4 features and 2 possible classes (mammals vs non-mammals).
I have summarized the dataset ...
- 111
2
votes
1
answer
73
views
On the denominator of Bayes and Naive Bayes
There are many question on the topic but I think the comparison below is harder to find.
Let us assume that all variables in this example are binary for the sake of simplicity. The typical academic ...
- 541
0
votes
1
answer
515
views
Why does Multinomial Naive Bayes work well on discrete features?
I understand Multinomial Naive Bayes is a specific instance of Naive Bayes when the data distribution is assumed to be multinomial.
In the sklearn documentation for Multinomial Naive Bayes, it is ...
- 183
2
votes
1
answer
109
views
Logistic Regression and NaiveBayes with infinite dataset
I'm comparing the LR and NB performance on different datasets. And suddenly I am wondering what if we have a big dataset that is infinitely large (at least ensure both models trained to its best??). ...
- 21