2
$\begingroup$

The best possible classification is for a set of samples drawn from any probability distribution is given by the Bayes decision rule.

For any distribution, the rule is given by

$$ f(x) = 1 \quad\text{if}\quad ~\eta(x) \geq \frac{1}{2} \quad\text{else}\quad 0 $$

Plugin classifiers estimates the sample posterior probability and makes decisions based on this value. But in most of the practical problems, the best possible classification is achieved using a value very different from $\eta = \frac{1}{2}$. Mostly this value is found using thresholding i.e. select $\eta$ value as the one which results in lowest classification error. I run many experiments on many datasets (balanced and imbalanced datasets) and most of the time the best $\eta$ value is found in the extreme regime (like near 0.2 or 0.7 or 0.8 etc). Why it is so ?. It should be very close to 0.5, according to theory right ?.

$\endgroup$
1
  • 2
    $\begingroup$ Why should it be close to 0.5? $\endgroup$
    – Tim
    Aug 20, 2021 at 13:04

2 Answers 2

1
$\begingroup$

In practice, the threshold value should be chosen depending on the priori, i.e., the probability indicating how likely a data belongs to a specific class. The priori should be considered when you perform Maximum Likelihood Estimation. An illustration (said prof. Andrew Ng) is that suppose in your database, 90% indicates people having no cancer and 10% indicates having cancer. So most of the case you should predict a person doesn't have cancer by improve the value of the threshold. Intuitively it means that unless you are pretty sure (high probability) that a person has cancer, you predict there's no cancer. Hope it does some help to you.

$\endgroup$
0
$\begingroup$

contrary to Duc Nguyen, if you're priming that old baysian pump with an input threshold, your assumptions about input should be

  1. be independent of how much or in which direction sample/data is skewed
  2. be predictable/reasonable for the dataset aka not an outlier
  3. be close to the overall mean (depends on what youre trying to classify)

while i don't think there's any necessity for it to be close to 0.5, the baysian priming is a chicken-and-egg situation so you can be forgiven for assuming there is.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.