0
$\begingroup$

The best possible classification is for a set of samples drawn from any probability distribution is given by the Bayes decision rule.

For any distribution, the rule is given by

$f(x) = 1 ~if ~\eta(x) \geq \frac{1}{2} ~else~ 0$

Plugin classifiers estimates the sample posterior probability and makes decisions based on this value. But in most of the practical problems, the best possible classification is achieved using a value very different from $\eta = \frac{1}{2}$. Mostly this value is found using thresholding i.e. select $\eta$ value as the one which results in lowest classification error. I run many experiments on many datasets (balanced and imbalanced datasets) and most of the time the best $\eta$ value is found in the extreme regime (like near 0.2 or 0.7 or 0.8 etc). Why it is so ?. It should be very close to 0.5, according to theory right ?.

$\endgroup$
0
$\begingroup$

In practice, the threshold value should be chosen depending on the priori, i.e., the probability indicating how likely a data belongs to a specific class. The priori should be considered when you perform Maximum Likelihood Estimation. An illustration (said prof. Andrew Ng) is that suppose in your database, 90% indicates people having no cancer and 10% indicates having cancer. So most of the case you should predict a person doesn't have cancer by improve the value of the threshold. Intuitively it means that unless you are pretty sure (high probability) that a person has cancer, you predict there's no cancer. Hope it does some help to you.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.