Questions tagged [bayes-optimal-classifier]

The tag has no usage guidance.

Filter by
Sorted by
Tagged with
2
votes
2answers
113 views

Improve Adaboost that using weighted logistic regression instead of decision trees

I implemented Adaboost that using weighted logistic regression instead of decision trees and I managed to get to 0.5% error, I'm trying to improve it for days with no success and I know it possible to ...
2
votes
1answer
17 views

Use different Naive Bayes classifiers to target different data

I am practicing using the Naive Bayes classifier to predict whether people get a stroke or not, but, I am confused with two classifiers. One is categorical Naive Bayes, another is Gaussian Naive Bayes....
0
votes
0answers
12 views

What is the difference between the class v and the hypothesis h? Some dumb example needed

In this example, the labels are "no/yes" which are enough to perform a Naive Bayes. But if i perform a Bayesian optimal classifier so P(v|x,D) = sum_h_of P(v|h,D) * P(h|D) with v the label ...
1
vote
0answers
26 views

How to proof that the bayes optimal classifier is optimal for a continuous domain

Exercise 3.7 from the book »Understanding Machine Learning: From Theory to Algorithms«, Shalev-Shwartz and Ben-David, states the following: The Bayes optimal predictor: Show that for every ...
0
votes
0answers
9 views

Existence of an optimal learning algorithm for a given distribution

I am working on Exercise 3.8 from Understanding Machine Learning by Shai Shalev-Shwartz and Shai Ben-David. In it we consider the binary classification problem where $\mathcal{X}$ is a set of data ...
0
votes
0answers
41 views

multi class text classification with multiple dependent variable to one set of predictors in r

I am doing multi class classification in text in r on a dataset containing two columns; feedback and topics. Some feedback has been assigned to more than one topics and some more than two but most of ...
1
vote
1answer
132 views

Bayes Optimal Classifier for multinomial classification

I understand the meaning and how to deduce a Bayes optimal classifier in binary classification, but I am not sure how to derive this in the context of multinomial classification. Do we use the naive ...
0
votes
0answers
26 views

Naive Bayes: why not to compute the likelihood probability directly? [duplicate]

A Bayes classifier assigns to an observation $X$ the class $Y$ that maximizes: $P(Y|X) \varpropto P(Y)P(X|Y)$ I wonder why not to estimate both $P(Y)$ and $P(X|Y)$ directly from the training data ...
0
votes
0answers
41 views

Bayes Optimal vs Naive Bayes

I am a 3rd year CS student and I am currently studying machine learning. We have recently been covering Bayes classifiers and I am very confused by the Bayes Optimal Classifier. For Naive Bayes, we ...
1
vote
0answers
204 views

LDA and Fisher LDA - are their weight vectors always equivalent?

Linear Discriminant Analysis (LDA) and Fisher Linear Discriminant Analysis (FLDA) both project high-dimensional observations to univariate classification scores using different rationals and ...
0
votes
0answers
177 views

How to find the optimal classifier for a given loss function?

For a binary classification problem (labels being 0 and 1) and a classifier $g$ we consider the loss function $L(g)=P[Y\neq g(X)]$. It is known that the optimal classifier $g^*$ that minimizes $L$ is $...
0
votes
1answer
30 views

Penalization term for unfairness

I am reading [1], where the researchers do a logistic regression, but add to the loss function the following penalization term for fairness $ R^{AVD}_{FP}(\theta; S) = \left\lvert \dfrac{\sum\limits_{...
3
votes
0answers
51 views

Question about using Bayesian rule as a classification for continuous data set

Please note that my question is not about coding. I am now learning Bayesian classification and I think I understand it in a discrete case. I have trouble understanding it for multivariate continuous ...
2
votes
1answer
302 views

Combining Classifiers with different Precision and Recall values

Suppose I have two binary classifiers, A and B. Both are trained on the same set of data, and produce predictions on a different (but same for both classifiers) set of data. The precision for A is ...
1
vote
0answers
58 views

Can we increase the accuracy of a classifier using sketches?

I am using a sketch technique to improve the memory of a standard classifier (naive Bayes) with data streams. The sketch technique is composed of a sketch table (hash table) means the true values can ...
1
vote
1answer
150 views

What is D in Optimal Bayes Classification (Machine Learning by Tom Mitchell Ed2)?

I'm reading chapter 6 from "Machine Learning" by Tom Mitchell, 2nd edition. It seems like the author changes in each paragraph what "D" is without saying anything, but it becomes really confussing at ...
1
vote
0answers
46 views

Bayes classifier in terms of generative and discriminant analysis approaches

For simplicity I will only discuss binary classification. If $p_k(x) = P(X \mid Y = k)$ for $k = 0,1$, then Bayes classifier $h$ minimizes the risk $P(h(x) \neq Y)$. It is well known that $$h(x) = 1\{...
0
votes
0answers
190 views

Is the Bayes optimal classifier well defined?

The Bayes optimal classifier (BOC) is defined as follows. When data $D$ is given, the classifier returns the value $$\text{argmax}_{y\in Y} \sum_{h} P(y\mid h) P(h\mid D)\text{,}$$ where the $Y$ is a ...
1
vote
0answers
433 views

Finding the error probability of an optimal bayes classifier analytically

I have two classes $\omega_1,\omega_2$ with equal prior probability $P(\omega_1)=P(\omega_2)=0.5$. And the points in 2D are distributed $\mathcal{N}(\mu_i,\Sigma), \mu_1=(0,0)^T, \mu_1=(4,4)^T, \Sigma=...
1
vote
1answer
163 views

error probability of decision function

If I have a binary calssification task with prior probability $p(0) = 0.6$, and I make two decisions. 1) solely based on the prior probability i.e. I make prediction 0 60% of the time and prediction ...
8
votes
1answer
6k views

Bayes optimal classifier vs Likelihood Ratio

I am getting slightly confused by all the probabilistic classifiers. The bayes optimal classifier is given as $ max (p(x|C)p(C)) $ and if all classes have equal prior then it reduces to $ max (p(x|C))...
1
vote
1answer
449 views

Can the Bayes Optimal Predictor be generalized?

I'm reading Understanding Machine Learning by Shai and Shai. In it, the Bayes Optimal Predictor is defined as $$f_{\mathcal{D}}(x) = \mathbb{1}[\mathbb{P}[y = 1 | x] \geq 1/2]$$ Where $\mathcal{D}$ ...
2
votes
1answer
403 views

Why classifiers report the class with maximum posterior probability as the predicted class?

When we train a classifier to predict $y \in \{1, \dots, K\}$ given an input $x$, classification is done by reporting the class with the highest posterior probability as the prediction; that is: $$ \...
5
votes
0answers
3k views

Help with a proof of Bayes classifier optimality

I have a class assignment to provide a proof that Bayes classifier for the two label version is optimal in that it's error rate is always ${\le}$ any other classifier. I've never worked through a ...
9
votes
4answers
13k views

Calculating the error of Bayes classifier analytically

If two classes $w_1$ and $w_2$ have normal distribution with known parameters ($M_1$, $M_2$ as their means and $\Sigma_1$,$\Sigma_2$ are their covariances) how we can calculate error of the Bayes ...