# Classification between most probable classes - what is its proper name?

I have to classify data between N classes.

At the first step, I make a classification between N classes with low threshold (for example, 10%). When first step is over, I have $M$ labels assigned to my datum $(M \leqslant N)$.

At the second step, I make normal classification with normal threshold between the $M$ classes to get the final decision.

What is the proper name of such policy?

• It is not clear to me what you meant by steps. Is it an algorithm? Do you reduce the number of classes each step? Do you want to classify $N$ classes using $M$ classes? – a.desantos Sep 16 '13 at 11:06
• Yes, I reduce the number of classes at first step (from $M$ to $N$), and only then I make the final classification (between $M$ classes). – Felix Sep 16 '13 at 11:09
• What? from $M$ to $N$? Wouldn't be from $N$ to $M$ given that $M\leq N$? In my opinion what you are doing it is a mixture between feature reduction and principal component analysis. If you explain yourself better, my response could be more helpful and oriented. – a.desantos Sep 16 '13 at 11:14
• Oh, holy Jesus! Of course, from $N$ to $M$. ^__^ – Felix Sep 16 '13 at 11:14

Many problems are caused by classification or premature classification. Think about using a continuous probability accuracy scoring rule. If you are interested in that I can send you a good reference.

• Certainly. In most of the cases I have to provide a probability estimation for the classification results. Give me the reference, if possible. – Felix Sep 17 '13 at 20:00
• @article {cal12ext, author = {Van Calster, Ben and Van Belle, Vanya and Vergouwe, Yvonne and Timmerman, Dirk and Van Huffel, Sabine and Steyerberg, Ewout W.}, title = {Extending the $c$-statistic to nominal polytomous outcomes: the Polytomous Discrimination Index}, journal = Statistics in Medicine, volume = 31, number = 23, issn = {1097-0258}, url = {dx.doi.org/10.1002/sim.5321}, doi = {10.1002/sim.5321}, pages = {2610--2626}, keywords = {polytomous risk prediction, discrimination, model performance, c-statistic, polytomous discrimination index}, year = 2012} – Frank Harrell Sep 18 '13 at 19:34

I think the name is going to largely depend on how exactly you carry out the process, so unfortunately the best I'll be able to do is throw some keywords at you with a little context.

Do you use the $M$ outputs of the first classifier as features for the second? If so this sounds like Cascading Classifiers. I think this term would also include the case where you did something like had a different classifier for each of the $\binom{N}{M}$ possible results of the first step.

Another term you might look at is stacking, although this doesn't really sound like what you're after.

• At first, I make a coarse classification with a low threshold, when I just save the list of classes, for which the coarse decision is above a small threshold. Of course, I don't know, what classes will form the list. Let there be M classes. And at second, I do the classification among the M classses and make the final decisions. – Felix Sep 17 '13 at 9:57
• @Felix, I understand that. I think if you're looking for the correct terminology the information you're leaving out about how you actually perform each step (how does the information from step 1 figure into step 2) is going to be important. For example, does the second classifier give you $P(y=k|x)$ for $k=1,\ldots,N$ and you just pick $\max_{k\in M}\{P(y=k|x)\}$ for the final classification, essentially allowing the first to veto any decision made by the second? Or is the second classifier doing something more like $\max_{k\in M}\{P(y=k|x,M)\}$? – alto Sep 17 '13 at 15:58
• The second classifier does $\max_{k \in M} \lbrace P(y=k|x,M)\rbrace$. – Felix Sep 17 '13 at 20:03
• Since the second classifier is conditioning on the input as well as the output of the first classifier, I would put it under the heading Cascading Classifiers. This is highly subjective and other opinions are likely to be equally valid. – alto Sep 17 '13 at 20:32

In my opinion, if there exist $N$ classes and after classification you realize that only $M\leq N$ are present in your result, then it is intuitive to think that your classification can be improved by using only $M$ classes.

However this is dangerous as if your data is supposed to have $N$ classes, the fact that a certain set has $M$ classes does not mean you can assume that your classifier may have $M$ classes because you are missing information for future data.

First of all, and answering your question, I do not think your procedure is a proper policy.

On the other hand, if during classification you want to gather several classes into one (since they have few representatives for instance) you are using similar to 'pruning' in decision tree like a 'merge' of classes. Maybe that is the name you are looking for.