0
$\begingroup$

This might be completely unfeasible or obvious, but I wanted to see if anyone has come across a similar problem. Typical classification methods (decision tree, random forest, etc) require at least two groups to determine group differences, however is it possible to have a singular group, and classify it based on the probability that it's derived from that group or not?

I guess the comparison that comes to mind is a shapiro-Wilk test which determines if a vector is derived from the normal distribution. Is there a method or modelling technique in which you could take a set of variables that characterise a single group, and perform a test to give the probability that it's derived from that group?

Thanks, and apologies if this is really obvious.

$\endgroup$
  • $\begingroup$ As noted by others, there are such methods designed for outlier detection, anomaly detection, or novelty detection, e.g. one-class SVM. $\endgroup$ – Tim Jan 19 '17 at 17:11
4
$\begingroup$

Well, if there is a single group, there is no classification problem, so strictly speaking this does not make sense. But what you're describing has resemblance to the idea of outlier detection, where you decide if a pattern or signal is coming from the group that you know already, or it is in fact an outlier or anomaly.

There are many approaches to (and names for, ie. novelty detection, anomaly detection) outlier detection, as you need some kind of assumptions on the distribution of the known group, and they may differ across the actual problem you are talking about, but it's quite a common problem as well. In fact, the test you are describing is also a form of outlier detection, where the group that 'you know already' is the set of vectors that are 'typically' the result of a multivariate normal distribution.

There is a good section in the scikit-learn documentation that you can start with and may lead you to other interesting sources.

$\endgroup$
1
$\begingroup$

Echoing some of the points Gijs already mentioned: if there is one group, there is another group, even if the second group is loosely defined as "observations which are heterogeneous". Most clustering algorithms are based on location, but it is possible to cluster based on spread. Using pattern-mixture modeling, you could assign probabilities to observations according to the likelihood that they belong to one "group" versus another. Rather than using off-the-shelf clustering algorithms, pattern-mixture models require you to specify the likelihoods in question which define these "groups". You could have both groups having a normal distribution but differing in terms of mean and/or standard deviation, or one group could take a normal distribution while the other takes, say, an exponential distribution. A test could then be simply performed with maximum likelihood, provided the "one-group" (no outlier) model is nested in the pattern-mixture model.

$\endgroup$
  • $\begingroup$ I don't think either answer really addresses the question of classification. Gijs had it right there is no classification problem with one group. $\endgroup$ – Michael Chernick Jan 19 '17 at 16:59
  • $\begingroup$ @MichaelChernick the OP's question could have been better framed. I took it to mean basically outlier detection, where a "central cluster" is defined by having less spread than a less prevalent "diffuse (or heterogeneous) cluster". This would imply two or more clusters, of course, but can still be situated in the broader context of the problem. $\endgroup$ – AdamO Jan 19 '17 at 17:35
  • $\begingroup$ I understand I didn't penalize either of you for your answers. They are reasonable and informative. $\endgroup$ – Michael Chernick Jan 19 '17 at 17:40

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.