# Bayes decision rule and thresholding

The best possible classification is for a set of samples drawn from any probability distribution is given by the Bayes decision rule.

For any distribution, the rule is given by

$$f(x) = 1 \quad\text{if}\quad ~\eta(x) \geq \frac{1}{2} \quad\text{else}\quad 0$$

Plugin classifiers estimates the sample posterior probability and makes decisions based on this value. But in most of the practical problems, the best possible classification is achieved using a value very different from $$\eta = \frac{1}{2}$$. Mostly this value is found using thresholding i.e. select $$\eta$$ value as the one which results in lowest classification error. I run many experiments on many datasets (balanced and imbalanced datasets) and most of the time the best $$\eta$$ value is found in the extreme regime (like near 0.2 or 0.7 or 0.8 etc). Why it is so ?. It should be very close to 0.5, according to theory right ?.

• Why should it be close to 0.5?
– Tim
Aug 20, 2021 at 13:04