# Worst classifier

What is the worst classier that learns badly in practical problems?

Edit: Especially bad on test data..

Thanks

-
Mathematically speaking, in binary classification, knowing the worst is equivalent to knowing the best. If you know the best you reverse the rule and it becomes the worst. Why do you want the worst rule ? what do you mean by worst ? –  robin girard Jan 5 '11 at 17:07

It is usually the statistician using the classifier that is the problem (for picking the wrong tool for the particular problem at hand, or for using it incorrectly). The "no free lunch" theorems show there is no a-priori distinction between classifiers (which works best/worst depends on the data), so I'd say there isn't a worst classifier in a general sense. However, some classifiers are easier to shoot yourself in the foot with than others, in which case, I would nominate artificial neural networks (the over-fit very easily and need proper architecture selection and none of the solutions to such problems are reliable).

-
Hah! I think my proposal is worse than yours :-). –  whuber Jan 4 '11 at 21:18
The no-free lunch theorem suggest that there will be problems where it works well and other algorithms dont! ;-P –  Dikran Marsupial Jan 4 '11 at 21:21
Thank you. Do you have a favorite reference or references to these NFL theorems? –  whuber Jan 4 '11 at 22:09
The original papers were by David Wolpert, there are some references and info at no-free-lunch.org . "no free lunch for the sandwich" would be a favourite reference, but only because of the title ;o) –  Dikran Marsupial Jan 4 '11 at 22:51
Consider the binary case. If you don't know the proportions of the two classes, then the worst you can do is to flip a fair coin in each case: the expected error rate is $1/2$. If you do know the proportions, and the smaller of the two is $p$, say, then you should always classify objects in that category: the expected error rate is $1-p \gt 1/2$. (However, it could be argued that this procedure is not worse. After observing it sufficiently long, you would notice its error rate is significantly greater than $1/2$ and then you would have the option of negating it--that is, assigning the opposite classification--and that would have an error rate of only $p \lt 1/2$.)