# Questions tagged [bayes-optimal-classifier]

The tag has no usage guidance.

25 questions
Filter by
Sorted by
Tagged with
113 views

### Improve Adaboost that using weighted logistic regression instead of decision trees

I implemented Adaboost that using weighted logistic regression instead of decision trees and I managed to get to 0.5% error, I'm trying to improve it for days with no success and I know it possible to ...
17 views

### Use different Naive Bayes classifiers to target different data

I am practicing using the Naive Bayes classifier to predict whether people get a stroke or not, but, I am confused with two classifiers. One is categorical Naive Bayes, another is Gaussian Naive Bayes....
12 views

### What is the difference between the class v and the hypothesis h? Some dumb example needed

In this example, the labels are "no/yes" which are enough to perform a Naive Bayes. But if i perform a Bayesian optimal classifier so P(v|x,D) = sum_h_of P(v|h,D) * P(h|D) with v the label ...
26 views

### How to proof that the bayes optimal classifier is optimal for a continuous domain

Exercise 3.7 from the book »Understanding Machine Learning: From Theory to Algorithms«, Shalev-Shwartz and Ben-David, states the following: The Bayes optimal predictor: Show that for every ...
9 views

### Existence of an optimal learning algorithm for a given distribution

I am working on Exercise 3.8 from Understanding Machine Learning by Shai Shalev-Shwartz and Shai Ben-David. In it we consider the binary classification problem where $\mathcal{X}$ is a set of data ...
41 views

### multi class text classification with multiple dependent variable to one set of predictors in r

I am doing multi class classification in text in r on a dataset containing two columns; feedback and topics. Some feedback has been assigned to more than one topics and some more than two but most of ...
132 views

### Bayes Optimal Classifier for multinomial classification

I understand the meaning and how to deduce a Bayes optimal classifier in binary classification, but I am not sure how to derive this in the context of multinomial classification. Do we use the naive ...
26 views

### Naive Bayes: why not to compute the likelihood probability directly? [duplicate]

A Bayes classifier assigns to an observation $X$ the class $Y$ that maximizes: $P(Y|X) \varpropto P(Y)P(X|Y)$ I wonder why not to estimate both $P(Y)$ and $P(X|Y)$ directly from the training data ...
41 views

### Bayes Optimal vs Naive Bayes

I am a 3rd year CS student and I am currently studying machine learning. We have recently been covering Bayes classifiers and I am very confused by the Bayes Optimal Classifier. For Naive Bayes, we ...
204 views

### LDA and Fisher LDA - are their weight vectors always equivalent?

Linear Discriminant Analysis (LDA) and Fisher Linear Discriminant Analysis (FLDA) both project high-dimensional observations to univariate classification scores using different rationals and ...
177 views

51 views

### Question about using Bayesian rule as a classification for continuous data set

Please note that my question is not about coding. I am now learning Bayesian classification and I think I understand it in a discrete case. I have trouble understanding it for multivariate continuous ...
302 views

### Combining Classifiers with different Precision and Recall values

Suppose I have two binary classifiers, A and B. Both are trained on the same set of data, and produce predictions on a different (but same for both classifiers) set of data. The precision for A is ...
58 views

### Can we increase the accuracy of a classifier using sketches?

I am using a sketch technique to improve the memory of a standard classifier (naive Bayes) with data streams. The sketch technique is composed of a sketch table (hash table) means the true values can ...
150 views

### What is D in Optimal Bayes Classification (Machine Learning by Tom Mitchell Ed2)?

I'm reading chapter 6 from "Machine Learning" by Tom Mitchell, 2nd edition. It seems like the author changes in each paragraph what "D" is without saying anything, but it becomes really confussing at ...
46 views

For simplicity I will only discuss binary classification. If $p_k(x) = P(X \mid Y = k)$ for $k = 0,1$, then Bayes classifier $h$ minimizes the risk $P(h(x) \neq Y)$. It is well known that $$h(x) = 1\{... 0answers 190 views ### Is the Bayes optimal classifier well defined? The Bayes optimal classifier (BOC) is defined as follows. When data D is given, the classifier returns the value$$\text{argmax}_{y\in Y} \sum_{h} P(y\mid h) P(h\mid D)\text{,}$$where the Y is a ... 0answers 433 views ### Finding the error probability of an optimal bayes classifier analytically I have two classes \omega_1,\omega_2 with equal prior probability P(\omega_1)=P(\omega_2)=0.5. And the points in 2D are distributed \mathcal{N}(\mu_i,\Sigma), \mu_1=(0,0)^T, \mu_1=(4,4)^T, \Sigma=... 1answer 163 views ### error probability of decision function If I have a binary calssification task with prior probability p(0) = 0.6, and I make two decisions. 1) solely based on the prior probability i.e. I make prediction 0 60% of the time and prediction ... 1answer 6k views ### Bayes optimal classifier vs Likelihood Ratio I am getting slightly confused by all the probabilistic classifiers. The bayes optimal classifier is given as  max (p(x|C)p(C))  and if all classes have equal prior then it reduces to  max (p(x|C))... 1answer 449 views ### Can the Bayes Optimal Predictor be generalized? I'm reading Understanding Machine Learning by Shai and Shai. In it, the Bayes Optimal Predictor is defined as$$f_{\mathcal{D}}(x) = \mathbb{1}[\mathbb{P}[y = 1 | x] \geq 1/2]$$Where \mathcal{D} ... 1answer 403 views ### Why classifiers report the class with maximum posterior probability as the predicted class? When we train a classifier to predict y \in \{1, \dots, K\} given an input x, classification is done by reporting the class with the highest posterior probability as the prediction; that is:$$ \...
I have a class assignment to provide a proof that Bayes classifier for the two label version is optimal in that it's error rate is always ${\le}$ any other classifier. I've never worked through a ...
If two classes $w_1$ and $w_2$ have normal distribution with known parameters ($M_1$, $M_2$ as their means and $\Sigma_1$,$\Sigma_2$ are their covariances) how we can calculate error of the Bayes ...