# Questions tagged [bayes-optimal-classifier]

The tag has no usage guidance.

32 questions
Filter by
Sorted by
Tagged with
51 views

### Modeling the probability distributions for the Bayes classifier

According to the Wikipedia, the Bayes classifier assumes knowledge of the distributions of $X | Y$, where $X$ and $Y$ are the random variables of the features and the classes, respectively. Let's ...
• 371
1 vote
49 views

### Is Fisher's discriminant analysis equivalent to the Bayes optimal LDA when the no. of classes is greater than two and covariances are all equal?

P.S. While I gave a brief background to make the question complete, informed readers can move to the questions 1 and 2 towards the end of this post, right after 'what are not clear to me are:'. Fisher'...
• 761
989 views

### Should a language model like GPT-3 be directly used to perform classification?

The OpenAI API enables classification by sampling from GPT-3 given a prompt. Is estimating posterior probabilities a more statistically sound approach? Below is a specification of what "...
• 1,420
1 vote
67 views

### How do I minimize the cost from errors?

Bayes' Optimal Classifier is known to achieve the minimum error rate for a dataset $x_1, \ldots, x_n, x_i \in \mathbb{R}^d$. Suppose that each error had a cost associated with it. For example, in a ...
• 255
642 views

### Use Naive Bayes to label unlabeled data

I have an Excel file that includes all product information (web scraped from Zalando) of 10k dresses. So for each dress/line I have multiple features available (brand, color, neckline, length...) I ...
204 views

### Derive the criterion for minimizing the expected loss when there is a general loss matrix and general prior probabilities for the classes

In the book "Pattern Recognition and Machine Learning" I am trying to do exercise 1.23 (p.63): Derive the criterion for minimizing the expected loss when there is a general loss matrix and ...
• 309
792 views

### What is the relation between Linear Classifier and Linear Decission Boundary (or Non Linear Decision Boundary)?

As we know (Wikipedia Definition): Linear Classifier makes a classification decision based on the linear combination of the feature vectors. Mathematically : $y = f(\sum w_i x_i)$ So , $f$ is our ...
1 vote
686 views

### How would you find a p threshold for a binary classification prediction? [duplicate]

Lets say that there's a binary classification problem where $X$ ∈ $R_p$ and $Y ∈ \{0,1\}$ and $Pr(Y = 1 | X = x) = p$ for $p$ in $[0,1]$. There is a loss function $L_{falseneg} > 0$ for false ...
• 11
122 views

### This decision is the best we can do if we have no prior information about the probabilities of the two classes?

I am currently studying the textbook Learning with kernels: support vector machines, regularization, optimization and beyond by Schölkopf and Smola. Chapter 1.2 A Simple Pattern Recognition Algorithm ...
• 1,902
234 views

### How is this a "Bayes classifier"?

I am currently studying the textbook Learning with kernels: support vector machines, regularization, optimization and beyond by Schölkopf and Smola. Chapter 1.2 A Simple Pattern Recognition Algorithm ...
• 1,902
72 views

### How $\dfrac{P_t(\mathbf{\mathrm{x}}, y)}{P_s(\mathbf{\mathrm{x}}, y)} = \dfrac{P_t(\mathbf{\mathrm{x}})}{P_s(\mathbf{\mathrm{x})}}$?

Related: Empirical Risk Minimization: Rewriting the expected loss using Bayes' rule and the definition of expectation I am currently studying Transfer Learning by Qiang Yang, Yu Zhang, Wenyuan Dai,...
• 1,902
465 views

### Empirical Risk Minimization: Rewriting the expected loss using Bayes' rule and the definition of expectation

I am currently studying Transfer Learning by Qiang Yang, Yu Zhang, Wenyuan Dai, and Sinno Jialin Pan. Chapter 2.2 Instance-Based Noninductive Transfer Learning says the following: As mentioned ...
• 1,902
1k views

### Improve Adaboost that using weighted logistic regression instead of decision trees

I implemented Adaboost that using weighted logistic regression instead of decision trees and I managed to get to 0.5% error, I'm trying to improve it for days with no success and I know it possible to ...
158 views

### Use different Naive Bayes classifiers to target different data

I am practicing using the Naive Bayes classifier to predict whether people get a stroke or not, but, I am confused with two classifiers. One is categorical Naive Bayes, another is Gaussian Naive Bayes....
• 207
247 views

### How to proof that the bayes optimal classifier is optimal for a continuous domain

Exercise 3.7 from the book »Understanding Machine Learning: From Theory to Algorithms«, Shalev-Shwartz and Ben-David, states the following: The Bayes optimal predictor: Show that for every ...
• 263
1 vote
254 views

### Bayes Optimal Classifier for multinomial classification

I understand the meaning and how to deduce a Bayes optimal classifier in binary classification, but I am not sure how to derive this in the context of multinomial classification. Do we use the naive ...
1k views

### LDA and Fisher LDA - are their weight vectors always equivalent?

Linear Discriminant Analysis (LDA) and Fisher Linear Discriminant Analysis (FLDA) both project high-dimensional observations to univariate classification scores using different rationals and ...
• 1,854
734 views

• 2,261
71 views

### Question about using Bayesian rule as a classification for continuous data set

Please note that my question is not about coding. I am now learning Bayesian classification and I think I understand it in a discrete case. I have trouble understanding it for multivariate continuous ...
• 316
833 views

### Combining Classifiers with different Precision and Recall values

Suppose I have two binary classifiers, A and B. Both are trained on the same set of data, and produce predictions on a different (but same for both classifiers) set of data. The precision for A is ...
1 vote
59 views

### Can we increase the accuracy of a classifier using sketches?

I am using a sketch technique to improve the memory of a standard classifier (naive Bayes) with data streams. The sketch technique is composed of a sketch table (hash table) means the true values can ...
• 103
1 vote
188 views

### What is D in Optimal Bayes Classification (Machine Learning by Tom Mitchell Ed2)?

I'm reading chapter 6 from "Machine Learning" by Tom Mitchell, 2nd edition. It seems like the author changes in each paragraph what "D" is without saying anything, but it becomes really confussing at ...
1 vote
58 views

• 2,251
3k views

### Help with a proof of Bayes classifier optimality

I have a class assignment to provide a proof that Bayes classifier for the two label version is optimal in that it's error rate is always ${\le}$ any other classifier. I've never worked through a ...
• 695
If two classes $w_1$ and $w_2$ have normal distribution with known parameters ($M_1$, $M_2$ as their means and $\Sigma_1$,$\Sigma_2$ are their covariances) how we can calculate error of the Bayes ...