# Questions tagged [bayes-optimal-classifier]

The tag has no usage guidance.

29 questions
Filter by
Sorted by
Tagged with
9 views

### Posteriori Probability, Parameter Vectors and the explicit form of Bayes Classifier

I'm working on some linear discriminant analysis problems to develop a better understanding of the underlying mechanisms for machine learning algorithms. I'd like to develop a better understanding of ...
37 views

### Expected value of given random variables [closed]

Consider $f(X,Y):=\underset{x \sim \mathcal{D}}{\mathbb{E}}_X \left[\underset{y \sim \mathcal{D}}{\mathbb{E}}_{Y | x}\left[\mathbb{\mathbf{1}}_{h(x) \neq y} | X = x\right]\right]$ where $\mathbf{1}$ ...
25 views

### Derive the criterion for minimizing the expected loss when there is a general loss matrix and general prior probabilities for the classes

In the book "Pattern Recognition and Machine Learning" I am trying to do exercise 1.23 (p.63): Derive the criterion for minimizing the expected loss when there is a general loss matrix and ...
41 views

### What is the relation between Linear Classifier and Linear Decission Boundary (or Non Linear Decision Boundary)?

As we know (Wikipedia Definition): Linear Classifier makes a classification decision based on the linear combination of the feature vectors. Mathematically : $y = f(\sum w_i x_i)$ So , $f$ is our ...
144 views

### How would you find a p threshold for a binary classification prediction?

Lets say that there's a binary classification problem where $X$ ∈ $R_p$ and $Y ∈ \{0,1\}$ and $Pr(Y = 1 | X = x) = p$ for $p$ in $[0,1]$. There is a loss function $L_{falseneg} > 0$ for false ...
114 views

### This decision is the best we can do if we have no prior information about the probabilities of the two classes?

I am currently studying the textbook Learning with kernels: support vector machines, regularization, optimization and beyond by Schölkopf and Smola. Chapter 1.2 A Simple Pattern Recognition Algorithm ...
189 views

### How is this a "Bayes classifier"?

I am currently studying the textbook Learning with kernels: support vector machines, regularization, optimization and beyond by Schölkopf and Smola. Chapter 1.2 A Simple Pattern Recognition Algorithm ...
53 views

### How $\dfrac{P_t(\mathbf{\mathrm{x}}, y)}{P_s(\mathbf{\mathrm{x}}, y)} = \dfrac{P_t(\mathbf{\mathrm{x}})}{P_s(\mathbf{\mathrm{x})}}$?

Related: Empirical Risk Minimization: Rewriting the expected loss using Bayes' rule and the definition of expectation I am currently studying Transfer Learning by Qiang Yang, Yu Zhang, Wenyuan Dai,...
202 views

### Empirical Risk Minimization: Rewriting the expected loss using Bayes' rule and the definition of expectation

I am currently studying Transfer Learning by Qiang Yang, Yu Zhang, Wenyuan Dai, and Sinno Jialin Pan. Chapter 2.2 Instance-Based Noninductive Transfer Learning says the following: As mentioned ...
390 views

### Improve Adaboost that using weighted logistic regression instead of decision trees

I implemented Adaboost that using weighted logistic regression instead of decision trees and I managed to get to 0.5% error, I'm trying to improve it for days with no success and I know it possible to ...
55 views

### Use different Naive Bayes classifiers to target different data

I am practicing using the Naive Bayes classifier to predict whether people get a stroke or not, but, I am confused with two classifiers. One is categorical Naive Bayes, another is Gaussian Naive Bayes....
104 views

### How to proof that the bayes optimal classifier is optimal for a continuous domain

Exercise 3.7 from the book »Understanding Machine Learning: From Theory to Algorithms«, Shalev-Shwartz and Ben-David, states the following: The Bayes optimal predictor: Show that for every ...
152 views

### Bayes Optimal Classifier for multinomial classification

I understand the meaning and how to deduce a Bayes optimal classifier in binary classification, but I am not sure how to derive this in the context of multinomial classification. Do we use the naive ...
573 views

### LDA and Fisher LDA - are their weight vectors always equivalent?

Linear Discriminant Analysis (LDA) and Fisher Linear Discriminant Analysis (FLDA) both project high-dimensional observations to univariate classification scores using different rationals and ...
386 views

58 views

### Question about using Bayesian rule as a classification for continuous data set

Please note that my question is not about coding. I am now learning Bayesian classification and I think I understand it in a discrete case. I have trouble understanding it for multivariate continuous ...
517 views

### Combining Classifiers with different Precision and Recall values

Suppose I have two binary classifiers, A and B. Both are trained on the same set of data, and produce predictions on a different (but same for both classifiers) set of data. The precision for A is ...
58 views

### Can we increase the accuracy of a classifier using sketches?

I am using a sketch technique to improve the memory of a standard classifier (naive Bayes) with data streams. The sketch technique is composed of a sketch table (hash table) means the true values can ...
177 views

### What is D in Optimal Bayes Classification (Machine Learning by Tom Mitchell Ed2)?

I'm reading chapter 6 from "Machine Learning" by Tom Mitchell, 2nd edition. It seems like the author changes in each paragraph what "D" is without saying anything, but it becomes really confussing at ...
54 views

For simplicity I will only discuss binary classification. If $p_k(x) = P(X \mid Y = k)$ for $k = 0,1$, then Bayes classifier $h$ minimizes the risk $P(h(x) \neq Y)$. It is well known that $$h(x) = 1\{... 0answers 197 views ### Is the Bayes optimal classifier well defined? The Bayes optimal classifier (BOC) is defined as follows. When data D is given, the classifier returns the value$$\text{argmax}_{y\in Y} \sum_{h} P(y\mid h) P(h\mid D)\text{,}$$where the Y is a ... 0answers 540 views ### Finding the error probability of an optimal bayes classifier analytically I have two classes \omega_1,\omega_2 with equal prior probability P(\omega_1)=P(\omega_2)=0.5. And the points in 2D are distributed \mathcal{N}(\mu_i,\Sigma), \mu_1=(0,0)^T, \mu_1=(4,4)^T, \Sigma=... 1answer 184 views ### error probability of decision function If I have a binary calssification task with prior probability p(0) = 0.6, and I make two decisions. 1) solely based on the prior probability i.e. I make prediction 0 60% of the time and prediction ... 1answer 6k views ### Bayes optimal classifier vs Likelihood Ratio I am getting slightly confused by all the probabilistic classifiers. The bayes optimal classifier is given as  max (p(x|C)p(C))  and if all classes have equal prior then it reduces to  max (p(x|C))... 1answer 551 views ### Can the Bayes Optimal Predictor be generalized? I'm reading Understanding Machine Learning by Shai and Shai. In it, the Bayes Optimal Predictor is defined as$$f_{\mathcal{D}}(x) = \mathbb{1}[\mathbb{P}[y = 1 | x] \geq 1/2]$$Where \mathcal{D} ... 1answer 461 views ### Why classifiers report the class with maximum posterior probability as the predicted class? When we train a classifier to predict y \in \{1, \dots, K\} given an input x, classification is done by reporting the class with the highest posterior probability as the prediction; that is:$$ \...
I have a class assignment to provide a proof that Bayes classifier for the two label version is optimal in that it's error rate is always ${\le}$ any other classifier. I've never worked through a ...
If two classes $w_1$ and $w_2$ have normal distribution with known parameters ($M_1$, $M_2$ as their means and $\Sigma_1$,$\Sigma_2$ are their covariances) how we can calculate error of the Bayes ...