Skip to main content

Questions tagged [naive-bayes]

A naive Bayes classifier is a simple probabilistic classifier based on applying Bayes' theorem with strong independence assumptions. A more descriptive term for the underlying probability model would be "independent feature model".

Filter by
Sorted by
Tagged with
0 votes
0 answers
10 views

Is multinomial naive Bayes classification not naive Bayes classification?

Suppose I am thinking about a classification problem and I have my features $X = (X_1,...,X_n)$ and my classification $C$ (taking values in some finite set of classes $c_1,...,c_k$). The naive Bayes ...
Sprotte's user avatar
  • 101
0 votes
1 answer
19 views

How to interpret plots of naive bayes classification model using naivebayes library in R

I'm using the 'naivebayes' library in R to run a three-label multiclassification model. When I plot the model, I get a series of charts--one for each predictor--that look like the following:enter ...
user avatar
0 votes
0 answers
10 views

Naive Bayer Classifier - Do we use dependence structure?

If we apply Naive Bayer Classifier and predict an unseen observation just by using the posterior probability calculated with Bayes theorem combined with the naive feature independence assumption, do ...
Marlon Brando's user avatar
1 vote
1 answer
71 views

Determining a mix of clusters by using naive Bayes classifiers

I have the following question which I cannot seem to find an easy answer to: given that we have two groups (let's call them 1 and 2), is it possible to determine a mixed percentage of the groups by ...
Tino D's user avatar
  • 236
0 votes
0 answers
20 views

Why do we add 2 to the denominator when doing laplace smoothing? [duplicate]

Every explanation of laplace smoothing for, e.g, spam filtering, includes the following: The solution is to never let any word probabilities be zero, by smoothing them upwards. Instead of starting ...
Foobar's user avatar
  • 349
0 votes
0 answers
46 views

Naive Bayes classification for multivalued marginal

x y z C 1 0 1 1 1 1 1 1 0 1 1 0 1 1 0 0 1 0 1 0 0 0 0 1 0 0 0 1 0 0 1 0 The dataset in the table above consisting of boolean variables x, y and z and a single boolean output variable C. I ...
Sena Yalçın's user avatar
1 vote
0 answers
19 views

Probability of an image containing a specific object, by combining the results of multiple dependent tests

I am trying to assign to images the probability of them containing a metal building. The images can contain either metal buildings, non-metal buildings or no building. Let $B$ be the event that an ...
lovonho's user avatar
  • 11
0 votes
0 answers
5 views

Which method should be used to determine the class ID of multiple SVM models?

I'm using Support Vector Machine(SVM) with image classification. Each SVM model results a linear model $$y = wx + b$$ Where $w$ and $b$ is the SVM parameters. If I have multiple SVM models, I will get ...
euraad's user avatar
  • 415
2 votes
1 answer
73 views

Naive Bayes with R - ISLR Default dataset

I'm currently reading chapter 4 of An Introduction to Statistical Learning by James et alter. I've been trying to go through the examples myself, replicating the calculations in R. Section $(4.4.4)$ ...
Matteo Campagnoli's user avatar
0 votes
0 answers
34 views

Interprete Multinomial Naive Bayes when working with real non-negative values

I am currently working on an algorithm that aims to reduce dimensions and map data within the non-negative orthant. Subsequently, the mapped data is utilized as input for a classifier. The classifiers ...
Thoth's user avatar
  • 123
1 vote
1 answer
92 views

Proving consistent/inconsistency of a fusion of KF estimates

I have a distributed fusion scenario with a single target where two sensor nodes $i,j$ estimate the true state $\mathbf{x}$ using a local Kalman filter. The (linear, Gaussian) measurement errors of ...
Nikhil Sharma's user avatar
4 votes
1 answer
95 views

What is the extension of Naive Bayes that breaks the conditonal independence?

I had the idea that we can overcome the conditional independence of features within Naive Bayes classification by assuming that we have latent (hidden) sub-classes. Let me explain. For example, if we ...
Roman's user avatar
  • 594
1 vote
1 answer
72 views

Naive Bayes is a "special case" of logistic regression - which other models?

Suppose $Y \in \{0, 1\}$ is a response variable and $X = (X_1, \cdots, X_p)$ are covariates with $X_j \in \{0, 1\}$ for each $j = 1, \cdots, p$. In the Naive Bayes model, we assume conditional ...
TheProofIsTrivium's user avatar
1 vote
1 answer
62 views

In Naïve Bayes, why do we estimate Pr(W|H)*Pr(H) instead of Pr(W)

This wikipedia article describes spam filtering using Naïve Bayes: https://en.wikipedia.org/wiki/Naive_Bayes_spam_filtering It says P(S|W) is given as ...
BasilTomato's user avatar
1 vote
0 answers
27 views

What type of classifier is Naive Bayes?

I am currently studying the Naive Bayes method (a classification method) and I am having quite some trouble classifying it as a hard or soft classifier. Below follows a quick introduction on the ...
xyz's user avatar
  • 185
2 votes
1 answer
307 views

Equivalence of Logistic regression to Gaussian naive bayes

I was revisiting the differences between logistic regression and Naive Bayes, and had a conceptual question. A logistic regression classifier makes intuitive sense to me as a classifier that directly ...
user9740643's user avatar
3 votes
3 answers
339 views

prior & posterior probability in Bayesian Decision Theory

Learning Bayesian decision theory (specifically in Machine Learning) recently, couldn't figure out what do the posterior possibility $P(c|x)$ and the prior possibility $P(x|c)$ mean exactly. Anybody ...
Bog's user avatar
  • 31
1 vote
1 answer
51 views

Posterior Probabilities in terms log odds ratio

From the book Bayesian Decision Analysis Principles and Practice, I am trying to prove $$\begin{aligned} \mathbb{P}(I=i\mid X=x)=\frac{\exp(O(i,1\mid x))}{1+\sum_{k=2}^n \exp(O(k,1\mid x))} \end{...
user avatar
1 vote
0 answers
15 views

Can Perceptron and Naive Bayes classifier create a vertical decision boundary in a two-dimensional graph?

A decision boundary like in the picture.
Xuan Viet Duc Pham's user avatar
1 vote
0 answers
81 views

Naive Bayes Classification

Consider the binary classification problem where class label Y ∈ {0, 1} and each training example X has 2 binary attributes X = [X1, X2] ∈ {0, 1}^2. Assume that class priors are given P(Y = 0) = P(Y = ...
harford's user avatar
  • 11
1 vote
1 answer
231 views

Is the Bayes Optimal Classifier actually the optimal classifier?

From a theoretical perspective is the Bayesian Optimal Classifier (BOC) the best possible classifier one can make? Better than NN and GBDT? Let's say that we have two distributions $P(X,Y)$ and $P(X',...
Carlos Mougan's user avatar
2 votes
1 answer
164 views

Logistic regression vs naive bayes and random forest

I have a dataset that is a high dimensional imbalanced dataset. The dataset is a categorical data set and I applied label encoder to transfer categorical values into numerical values. the dataset is a ...
Encipher's user avatar
  • 175
1 vote
1 answer
50 views

In a multinomial naive Bayes classifier, is the feature vector always a histogram?

In the Wikipedia definition, the feature vector is defined as a histogram, as well as in this popular and well-done YouTube video. However, if the features are words, then the variable is nominal and, ...
john987's user avatar
  • 11
3 votes
1 answer
95 views

Why there is no alpha parameter for GaussianNB()?

Why there is no alpha argument ( smoothing parameter in Laplace smoothing) for GaussianNB() in sklearn library? ? Although BernoulliNB() and MultinomialNB() have an alpha parameter but GaussianNB() ...
AAA's user avatar
  • 45
1 vote
1 answer
328 views

Dependent Features and Naive Bayes

Naive Bayes assumes that the features given their classes are independent, and hence : $$P(y~|~x_1, \ldots, x_n)= \frac{P(y)P(x_1,\ldots, x_n~|~ y) }{P(x_1,\ldots,x_n)}$$ Will become : $$ P(y~|~ x_1,\...
AAA's user avatar
  • 45
2 votes
0 answers
38 views

Simplistic linear estimator for a probability vector

I am working on a problem where the unknown probabilities $p_i$ are related to observed rates/frequencies $\pi_\alpha$ as $$ \pi_\alpha = \sum_iW_{\alpha i}p_i, $$ where $W_{\alpha i}$ is known (...
Roger V.'s user avatar
  • 4,196
2 votes
1 answer
580 views

Understanding the application of MLE in Naive Bayes

I was looking at the Naive Bayes classifier models (Binomial, Multinomial and Gaussian) and trying to understand the theory behind them a bit better, but am unsure if I understand the MLE approach ...
user3629892's user avatar
1 vote
1 answer
67 views

Assuming Independence in Naive Bayes

If features of the Naive Bayes are not independent then what are the consequences of the results?
NMA's user avatar
  • 19
0 votes
1 answer
57 views

Understanding rare definition of the likelihood function and corresponding posterior from research paper

Reading the paper https://storage.googleapis.com/pub-tools-public-publication-data/pdf/b20467a5c27b86c08cceed56fc72ceadb875184a.pdf i came across a rare definition of the likelihood function that in ...
stewardbranson's user avatar
1 vote
1 answer
86 views

interpreting confusion matrix results

I have a dataset on unemployed individuals enrolled in a job training program where I am trying to predict whether 6 months post-enrolment they 1) gain employment, 2) stay unemployed, or 3) drop out ...
maldini1990's user avatar
2 votes
2 answers
103 views

If I engineer a new feature such that feature C = feature A/feature B, must I drop features A and B from a Gaussian Naive Bayes model?

As the question asks, is it bad data science not to drop the dividend and divisor features when creating a new feature that is their quotient when working with a Naive Bayes model? My understanding of ...
NaiveBae's user avatar
  • 257
0 votes
0 answers
33 views

Is it possible for the gains line to fall below the naive rule in a lift chart?

I created a naive Bayes model and generated this lift chart . Is it possible that my model could underperform the naiveBayes rule?
Regis Maria O'Connor's user avatar
2 votes
1 answer
198 views

AUC - Logistic Regression versus LDA, and Naive Bayes

everyone! I am a newbie on machine learning, and I am now interested on classification modeling. I used logistic regression, linear discriminant analysis (LDA), and naive Bayes on my notebook DataCamp ...
GregOliveira's user avatar
0 votes
0 answers
492 views

NaiveBayes in R - Understand importance of Variables

I am working with a data set where the response variable is binary and 15-20 continuous and categorical variables. I am using the naiveBayes library to compute the model. I am interested in ...
Michael Lamontagne's user avatar
0 votes
1 answer
685 views

How to handle missing values NaiveBayes Scikit Learn

I am working with a dataset which has 34 features (numerical, nominal) and the target class. Several of the columns have missing values, especially one column has approximately 50% missing values. I ...
Panos's user avatar
  • 1
0 votes
0 answers
31 views

Why are linear/logistic regression and naive bayes called "parametric" while SVM, random forests, neural nets are not? [duplicate]

This table is mentioned in What algorithms need feature scaling, beside from SVM? It says that linear regression, logistic regression, and naive bayes are parametric, while KNN, decision trees, ...
tomjavg's user avatar
  • 31
1 vote
1 answer
58 views

Impact of Laplace smoothing on likelihood in Naive Bayes

When 1 is added to word count in Laplace Smoothing in Naive Bayes, the new probabilities either increase or decrease as shown below. Though the problem of "zero" probability has been solved. ...
Arpit Saxena's user avatar
0 votes
1 answer
647 views

Use Naive Bayes to label unlabeled data

I have an Excel file that includes all product information (web scraped from Zalando) of 10k dresses. So for each dress/line I have multiple features available (brand, color, neckline, length...) I ...
Mara Socquet's user avatar
0 votes
1 answer
201 views

Naive-Bayes Iris R, Correct Implementation? [closed]

So I am trying to understand the naive Bayes classifier by implementing it in R. However I'm not sure if my implementation is correct. Using the iris dataset and Sepal Width / Length as features. ...
user2550228's user avatar
1 vote
0 answers
14 views

Log Naive Bayes NLP dropping the denominator [duplicate]

I'm learning about the the Naive Bayes classification and I don't get what the squiggly alpha sign means and what it means that "Denominator remains constant for given input." Is it because ...
Yi Yao Tan's user avatar
0 votes
1 answer
112 views

Computing a prior from two components in Naive Bayes

Given a model parameter $\theta$ that is composed of two distributions in a Naive Bayes classifier, how is $P(\theta)$ typically computed in practice? More specifically, from the article of Nigam et ...
user avatar
1 vote
0 answers
28 views

Laplace Smoothing in Naive Bayes [duplicate]

I'm reading up on Laplace Smoothing/Add-1 Smoothing in Naive Bayes and I'm given the formula $ \frac{Count(Feature=Value) + α}{N + α\cdot k} $. In reference to the image above, if we have to classify ...
Dithering's user avatar
1 vote
0 answers
46 views

Oscillation of AdaBoost Training error

Adaboost, using weak learners as Gaussian Naive bayes, has oscillating/unpredictable training error as we increase the number of weak learners. Is there a specific reason for this? Y-axis is the ...
A J's user avatar
  • 29
4 votes
1 answer
2k views

What attributes does Laplace Smoothing apply on in Naive Bayes

Consider the dataset: Outlook Temperature Humidity Play Golf? Overcast Cool Low Yes Sunny Hot Low Yes Rainy Cool High No Sunny Hot High No Rainy Cool Low Yes There are 3 possible values for the ...
UnsurelyStuck's user avatar
1 vote
0 answers
78 views

How do you interpret the matrix confusion in this Naïve Bayes output?

Why are my correctly classified instances lower than incorrectly classified? This was tested using Naive Bayes with option testing Cross-Validation set at 10 folds. Here is the image of the results: ...
Mark's user avatar
  • 11
1 vote
0 answers
343 views

How to calculate the Bayesian Risk Classifier

I'm not exactly sure how to calculate the Bayesian risk Classifier $L(r^*)$ for $Y\in\{ 0,1 \}$. For this scenario, assume: $X\in\mathbb{X}=[0,1],Y\in\{ 0,1 \}$ $\pi_y=P(Y=y)=1/2$ for $y\in{0,1}$ ...
Major Redux's user avatar
3 votes
1 answer
328 views

Stuck on a step calculating Naive Bayes Classifier,

Using the example at 3Blue1Brown I constructed a table to help me remember Bayes theorem where L=Librarian and S =Shy. I understand that $$P(S,L) = P(S|L)P(L) = P(L|S)P(S) = \frac{4}{210}$$ I am ...
Kirsten's user avatar
  • 793
0 votes
0 answers
911 views

How many divergent transitions are too many?

I am running a Bayesian linear mixed effects analysis. Four chains for 3000 iterations. I end up with four divergent transitions. Is this too many or can I proceed? How do I know if it's too many? I'...
Dave's user avatar
  • 2,621
1 vote
0 answers
15 views

Base rate calculation for customer conversion

Question: What is the base rate of conversion for mobile versus desktop sites? Total no of customers: 590381 Out of 590381, the Total no of customers that were converted: 701 These customers used ...
Arjun's user avatar
  • 11
10 votes
2 answers
3k views

Intuition for why LDA is a special case of naive Bayes

The naive Bayes classifier assumes the regressors to be mutually independent, while linear discriminant analysis (LDA) allows them to be correlated. James et al. "An Introduction to Statistical ...
Richard Hardy's user avatar

1
2 3 4 5
12