Questions tagged [naive-bayes]

A naive Bayes classifier is a simple probabilistic classifier based on applying Bayes' theorem with strong independence assumptions. A more descriptive term for the underlying probability model would be "independent feature model".

Filter by
Sorted by
Tagged with
2 votes
1 answer
43 views

Equivalence of Logistic regression to Gaussian naive bayes

I was revisiting the differences between logistic regression and Naive Bayes, and had a conceptual question. A logistic regression classifier makes intuitive sense to me as a classifier that directly ...
3 votes
3 answers
211 views

prior & posterior probability in Bayesian Decision Theory

Learning Bayesian decision theory (specifically in Machine Learning) recently, couldn't figure out what do the posterior possibility $P(c|x)$ and the prior possibility $P(x|c)$ mean exactly. Anybody ...
  • 31
0 votes
1 answer
30 views

Posterior Probabilities in terms log odds ratio

From the book Bayesian Decision Analysis Principles and Practice, I am trying to prove $$\begin{aligned} \mathbb{P}(I=i\mid X=x)=\frac{\exp(O(i,1\mid x))}{1+\sum_{k=2}^n \exp(O(k,1\mid x))} \end{...
user avatar
0 votes
0 answers
20 views

Multilevel (Hierarchical) Bayesian Model in R

I have my dataset with different mutations as unit of analysis. These mutations belong to 5 different classes. Also, I have collected, 9 features about these mutations. In other words I have 12 ...
0 votes
0 answers
33 views

How to derive Complement Naive Bayes rule

I have been trying to use probability rules to mathematically derive the complement naive bayes (CNB) classifier that is developed in this document by Rennie et al.. It is also currently implemented ...
1 vote
0 answers
11 views

Can Perceptron and Naive Bayes classifier create a vertical decision boundary in a two-dimensional graph?

A decision boundary like in the picture.
0 votes
0 answers
12 views

Should you use Naive Bayes/LDA when N features ≥ N datapoints or vice versa?

First of all, I'm lumping LDA in with Naive Bayes because LDA is a special case of Naive Bayes. I read in ISLR that Naive Bayes and LDA are good for instances where you have a small amount of N ...
  • 513
0 votes
0 answers
11 views

How to incorporate document metadata into a multinomial Naive Bayes model? (in R)

I've built a naïve Bayes classifier that assigns observations to categories based on a text description. I've converted the text to a document-term matrix and fed that to the ...
0 votes
0 answers
12 views

Can you use Naive Bayes with categorical and continuous features? [duplicate]

Can you use Naive Bayes with categorical and continuous features? I read somewhere that Naive Bayes can only be used with categorical features. Is this true or not? I think the post I read intended to ...
  • 513
1 vote
0 answers
32 views

Naive Bayes Classification

Consider the binary classification problem where class label Y ∈ {0, 1} and each training example X has 2 binary attributes X = [X1, X2] ∈ {0, 1}^2. Assume that class priors are given P(Y = 0) = P(Y = ...
  • 11
0 votes
0 answers
14 views

Estimators for gaussian distribution including label for data point (Bayes classification - ML)

I am struggling to understand how I am supposed to derive two estimators for a standard gaussian distribution in bayes classification. I have a data set $ \chi = [x^t,r^t]_{i=1}^N$, and two classes. I ...
1 vote
1 answer
59 views

Is the Bayes Optimal Classifier actually the optimal classifier?

From a theoretical perspective is the Bayesian Optimal Classifier (BOC) the best possible classifier one can make? Better than NN and GBDT? Let's say that we have two distributions $P(X,Y)$ and $P(X',...
0 votes
0 answers
13 views

Is it normal for my Naive Bayes classifier to have its output probabilities so close to each other?

I am currently building a Naive Bayes classifier for language detection, the model is supposed to take a sentence as an input and return the top 3 language guesses. Since I want to avoid the curse of ...
1 vote
1 answer
92 views

Logistic regression vs naive bayes and random forest

I have a dataset that is a high dimensional imbalanced dataset. The dataset is a categorical data set and I applied label encoder to transfer categorical values into numerical values. the dataset is a ...
  • 155
1 vote
1 answer
41 views

In a multinomial naive Bayes classifier, is the feature vector always a histogram?

In the Wikipedia definition, the feature vector is defined as a histogram, as well as in this popular and well-done YouTube video. However, if the features are words, then the variable is nominal and, ...
  • 11
3 votes
1 answer
63 views

Why there is no alpha parameter for GaussianNB()?

Why there is no alpha argument ( smoothing parameter in Laplace smoothing) for GaussianNB() in sklearn library? ? Although BernoulliNB() and MultinomialNB() have an alpha parameter but GaussianNB() ...
  • 45
1 vote
1 answer
123 views

Dependent Features and Naive Bayes

Naive Bayes assumes that the features given their classes are independent, and hence : $$P(y~|~x_1, \ldots, x_n)= \frac{P(y)P(x_1,\ldots, x_n~|~ y) }{P(x_1,\ldots,x_n)}$$ Will become : $$ P(y~|~ x_1,\...
  • 45
2 votes
0 answers
35 views

Simplistic linear estimator for a probability vector

I am working on a problem where the unknown probabilities $p_i$ are related to observed rates/frequencies $\pi_\alpha$ as $$ \pi_\alpha = \sum_iW_{\alpha i}p_i, $$ where $W_{\alpha i}$ is known (...
  • 3,354
0 votes
0 answers
37 views

Naive Bayes fits multiple hyperplanes in the case of multiclass classification problems?

Naive Bayes differentiates feature distributions given target labels, and intuitively, it fits a hyperplane to the given data set. But I do not fully understand whether Naive Bayes fits multiple ...
2 votes
1 answer
283 views

Understanding the application of MLE in Naive Bayes

I was looking at the Naive Bayes classifier models (Binomial, Multinomial and Gaussian) and trying to understand the theory behind them a bit better, but am unsure if I understand the MLE approach ...
0 votes
0 answers
34 views

Population stability index and Text Data Length

I'm training a language detection model using: a training set, classified between English and not English sentences or small paragraps, where the length of the sentences can vary a score set that is ...
1 vote
1 answer
44 views

Assuming Independence in Naive Bayes

If features of the Naive Bayes are not independent then what are the consequences of the results?
  • 19
0 votes
1 answer
43 views

Understanding rare definition of the likelihood function and corresponding posterior from research paper

Reading the paper https://storage.googleapis.com/pub-tools-public-publication-data/pdf/b20467a5c27b86c08cceed56fc72ceadb875184a.pdf i came across a rare definition of the likelihood function that in ...
1 vote
1 answer
53 views

interpreting confusion matrix results

I have a dataset on unemployed individuals enrolled in a job training program where I am trying to predict whether 6 months post-enrolment they 1) gain employment, 2) stay unemployed, or 3) drop out ...
2 votes
2 answers
42 views

If I engineer a new feature such that feature C = feature A/feature B, must I drop features A and B from a Gaussian Naive Bayes model?

As the question asks, is it bad data science not to drop the dividend and divisor features when creating a new feature that is their quotient when working with a Naive Bayes model? My understanding of ...
  • 237
0 votes
0 answers
23 views

Is it possible for the gains line to fall below the naive rule in a lift chart?

I created a naive Bayes model and generated this lift chart . Is it possible that my model could underperform the naiveBayes rule?
2 votes
1 answer
132 views

AUC - Logistic Regression versus LDA, and Naive Bayes

everyone! I am a newbie on machine learning, and I am now interested on classification modeling. I used logistic regression, linear discriminant analysis (LDA), and naive Bayes on my notebook DataCamp ...
0 votes
0 answers
14 views

Minimize risk and add rejection to model

I want to minimize the risk of a Gaussian model with a cost for false negatives and false positives. The model uses Naive Bayes algorithm and solves a binary classification problem: $$P(x_i \mid y) = \...
  • 1
0 votes
0 answers
265 views

NaiveBayes in R - Understand importance of Variables

I am working with a data set where the response variable is binary and 15-20 continuous and categorical variables. I am using the naiveBayes library to compute the model. I am interested in ...
0 votes
1 answer
366 views

How to handle missing values NaiveBayes Scikit Learn

I am working with a dataset which has 34 features (numerical, nominal) and the target class. Several of the columns have missing values, especially one column has approximately 50% missing values. I ...
  • 1
0 votes
0 answers
38 views

Find conditional independence between the attributes of a categorical dataset

I have a high dimensional data set. I used feature selection method to reduce the dimensionality of the dataset. Originally, the dataset has 120 attributes which I minimized to 80 attributes after ...
  • 155
0 votes
0 answers
26 views

Why are linear/logistic regression and naive bayes called "parametric" while SVM, random forests, neural nets are not? [duplicate]

This table is mentioned in What algorithms need feature scaling, beside from SVM? It says that linear regression, logistic regression, and naive bayes are parametric, while KNN, decision trees, ...
  • 31
1 vote
1 answer
39 views

Impact of Laplace smoothing on likelihood in Naive Bayes

When 1 is added to word count in Laplace Smoothing in Naive Bayes, the new probabilities either increase or decrease as shown below. Though the problem of "zero" probability has been solved. ...
0 votes
1 answer
233 views

Use Naive Bayes to label unlabeled data

I have an Excel file that includes all product information (web scraped from Zalando) of 10k dresses. So for each dress/line I have multiple features available (brand, color, neckline, length...) I ...
0 votes
1 answer
165 views

Naive-Bayes Iris R, Correct Implementation? [closed]

So I am trying to understand the naive Bayes classifier by implementing it in R. However I'm not sure if my implementation is correct. Using the iris dataset and Sepal Width / Length as features. ...
1 vote
0 answers
14 views

Log Naive Bayes NLP dropping the denominator [duplicate]

I'm learning about the the Naive Bayes classification and I don't get what the squiggly alpha sign means and what it means that "Denominator remains constant for given input." Is it because ...
0 votes
1 answer
104 views

Computing a prior from two components in Naive Bayes

Given a model parameter $\theta$ that is composed of two distributions in a Naive Bayes classifier, how is $P(\theta)$ typically computed in practice? More specifically, from the article of Nigam et ...
1 vote
0 answers
20 views

Laplace Smoothing in Naive Bayes [duplicate]

I'm reading up on Laplace Smoothing/Add-1 Smoothing in Naive Bayes and I'm given the formula $ \frac{Count(Feature=Value) + α}{N + α\cdot k} $. In reference to the image above, if we have to classify ...
1 vote
0 answers
40 views

Oscillation of AdaBoost Training error

Adaboost, using weak learners as Gaussian Naive bayes, has oscillating/unpredictable training error as we increase the number of weak learners. Is there a specific reason for this? Y-axis is the ...
  • 29
3 votes
1 answer
1k views

What attributes does Laplace Smoothing apply on in Naive Bayes

Consider the dataset: Outlook Temperature Humidity Play Golf? Overcast Cool Low Yes Sunny Hot Low Yes Rainy Cool High No Sunny Hot High No Rainy Cool Low Yes There are 3 possible values for the ...
1 vote
0 answers
61 views

How do you interpret the matrix confusion in this Naïve Bayes output?

Why are my correctly classified instances lower than incorrectly classified? This was tested using Naive Bayes with option testing Cross-Validation set at 10 folds. Here is the image of the results: ...
  • 11
1 vote
0 answers
248 views

How to calculate the Bayesian Risk Classifier

I'm not exactly sure how to calculate the Bayesian risk Classifier $L(r^*)$ for $Y\in\{ 0,1 \}$. For this scenario, assume: $X\in\mathbb{X}=[0,1],Y\in\{ 0,1 \}$ $\pi_y=P(Y=y)=1/2$ for $y\in{0,1}$ ...
3 votes
1 answer
174 views

Stuck on a step calculating Naive Bayes Classifier,

Using the example at 3Blue1Brown I constructed a table to help me remember Bayes theorem where L=Librarian and S =Shy. I understand that $$P(S,L) = P(S|L)P(L) = P(L|S)P(S) = \frac{4}{210}$$ I am ...
  • 639
0 votes
0 answers
451 views

How many divergent transitions are too many?

I am running a Bayesian linear mixed effects analysis. Four chains for 3000 iterations. I end up with four divergent transitions. Is this too many or can I proceed? How do I know if it's too many? I'...
  • 2,161
1 vote
0 answers
15 views

Base rate calculation for customer conversion

Question: What is the base rate of conversion for mobile versus desktop sites? Total no of customers: 590381 Out of 590381, the Total no of customers that were converted: 701 These customers used ...
  • 11
9 votes
2 answers
2k views

Intuition for why LDA is a special case of naive Bayes

The naive Bayes classifier assumes the regressors to be mutually independent, while linear discriminant analysis (LDA) allows them to be correlated. James et al. "An Introduction to Statistical ...
5 votes
1 answer
429 views

Why is naive Bayes overconfident?

In the fourth edition of "Artificial Intelligence: a modern approach" by Russel and Norvig, they write in section 12.6, regarding the Naive Bayes Model for text classification, the following:...
  • 171
0 votes
1 answer
2k views

How can Naive Bayes overfit the data?

I know that Laplace smoothing results in a high bias of Naive Bayes. If the value of the smoothing parameter (alpha) is large, then the probability distribution will be uniform for all the features. ...
2 votes
1 answer
350 views

Parameters in Naive Bayes

This is from https://scikit-learn.org/stable/modules/naive_bayes.html In the last line it says and we can use Maximum A Posteriori (MAP) estimation to estimate $P(y)$ and $P(x_i|y)$; the former is ...
  • 21
3 votes
1 answer
2k views

Product of two normal distributions (for Bayes Rule) is not product of normal output variables?

When we apply Bayes' rule in machine learning, we want to compute the posterior probability $P(y|X)$ by multiplying two probability distributions (the observed class-conditional likelihood $P(X|y)$ ...

1
2 3 4 5
12