2
$\begingroup$

Using any supervised classifier, we can usually get the probability that a data point $x$ belongs to each class $y_i$, i.e. $P(y_i|x)$.

However, in the case where the data-point x may belong to none of the known classes, how can we get the probability $P(?|x)$, which means that x belongs to an unknown class with probability $P(?|x)$ ?

$\endgroup$
2
$\begingroup$

I don't think probability of belonging to something unknown can be computed. For known classes, you have their characteristics (e.g. centroid, covariance matrix); for a new class in statu nascendi and perhaps represented so far by that single data pount x, you can say nothing.

The question of initiating a new class is a question of co-outliers. If the classifying functions (e.g. discriminant functions or such) bear unusual values or unusual combination of values for point x, the values not observed in the training data, then x is an outlier in relation to all the known classes. When you get many points similar to x you might speak of an emergent class.

$\endgroup$
  • $\begingroup$ ttnphns, so roughly you think that the problem of detecting a new probably new class is equivalent to detecting some similar outliers. I was more thinking about since the known classes are characterised by some characteristics (e.g. centroid etc), then we can maybe define a probability according to how much a new data-point is not characterized by characteristics of all the known classes (e.g. distance to centroids etc.). So it will be "the probability of belonging to known a class + "probability of belonging to an unknonw class" = 1 $\endgroup$ – shn Mar 18 '13 at 16:17
  • $\begingroup$ The sum of probabilities of belonging to known classes is 1. There is no option to belong to a new class until that class is described. To describe it, unusual data must collect. The data are described not by probabilities of belonging but by the original variables or the extracted classification functions. $\endgroup$ – ttnphns Mar 18 '13 at 16:29
  • $\begingroup$ So the question turns out to: how to explore unknown regions of the space in order to query the label of points lying in such regions ? $\endgroup$ – shn Mar 18 '13 at 16:38
  • $\begingroup$ Yes, re-study the updated data, decide to introduce the new class, so it is "known class" now, re-train classifiers. $\endgroup$ – ttnphns Mar 18 '13 at 17:00
  • $\begingroup$ What do you call "the updated data" are the new data points that are far (distance) from the points belonging to the known classes ? $\endgroup$ – shn Mar 18 '13 at 17:27

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.