33
$\begingroup$

The second question is that I found in a discussion somewhere on the web talking about "supervised clustering", as far as I know, clustering is unsupervised, so what is exactly the meaning behind "supervised clustering" ? What is the difference with respect to "classification" ?

There are many links talking about that:

http://www.cs.uh.edu/docs/cosc/technical-reports/2005/05_10.pdf

http://books.nips.cc/papers/files/nips23/NIPS2010_0427.pdf

http://engr.case.edu/ray_soumya/mlrg/supervised_clustering_finley_joachims_icml05.pdf

http://www.public.asu.edu/~kvanlehn/Stringent/PDF/05CICL_UP_DB_PWJ_KVL.pdf

http://www.machinelearning.org/proceedings/icml2007/papers/366.pdf

http://www.cs.cornell.edu/~tomf/publications/supervised_kmeans-08.pdf

http://jmlr.csail.mit.edu/papers/volume6/daume05a/daume05a.pdf

etc ...

Improve this question
$\endgroup$
3

4 Answers 4

Reset to default
5
$\begingroup$

My naive understanding is that classification is performed where you have a specified set of classes and you want to classify a new thing/dataset into one of those specified classes.

Alternatively, clustering has nothing to start with and you use all the data (including the new one) to separate into clusters.

Both use distance metrics to decide how to cluster/classify. The difference is that classification is based off a previously defined set of classes whereas clustering decides the clusters based on the entire data.

Again my naive understand is that supervised clustering still clusters based on the entire data and thus would be clustering rather than classification.

In reality i'm sure the theory behind both clustering and classification are inter-twinned.

Improve this answer
$\endgroup$
1
  • 1
    $\begingroup$ I humbly disagree. You're suggesting that "classification" is by definition and by default a supervised process, which is not true. Classification is divided into supervised and unsupervised cases, the latter being synonymous to clustering. $\endgroup$
    – Digio
    Commented Aug 20, 2015 at 8:46
23
+100
$\begingroup$

I don't think I know more than you do, but the links you posted do suggest answers. I'll take http://www.cs.cornell.edu/~tomf/publications/supervised_kmeans-08.pdf as an example. Basically they state: 1) clustering depends on a distance. 2) successful use of k-means requires a carefully chosen distance. 3) Given training data in the form of sets of items with their desired partitioning, we provide a structural SVM method that learns a distance measure so that k-means produces the desired clusterings. In this case there is a supervised stage to the clustering, with both training data and learning. The purpose of this stage is to learn a distance function so that applying k-means clustering with this distance will be hopefully optimal, depending on how well the training data resembles the application domain. All the usual caveats appropriate to machine learning and clustering still apply.

Further quoting from the article: Supervised clustering is the task of automatically adapting a clustering algorithm with the aid of a training set consisting of item sets and complete partitionings of these item sets.. That seems a reasonable definition.

Improve this answer
$\endgroup$
7
  • 2
    $\begingroup$ The problem is simply: why do you want to learn a distance measure from a set of labelled training data, and then apply this distance measure with a clustering method; why you would not just use a supervised method. In other words, you want to do clustering (i.e. partitioning your dataset into clusters), but you assume that you already have the complete desired partitioning and that you will use it to learn a distance measure, then apply clustering on this dataset using this learned distance. At best, you'll get the same partitions that you used to learn the distance measure ! You already have $\endgroup$
    – shn
    Commented Oct 10, 2012 at 17:58
  • $\begingroup$ Where you write "then apply clustering on this datase" substitute "then apply clustering on similar datasets". It is this scenario: in experiment X we have data A and B. A is for clustering, B helps with learning the distance. B sets a gold standard and is presumably expensive to obtain. In subsequent experiments X2, X3 .. we obtain A but cannot afford to obtain B. $\endgroup$
    – micans
    Commented Oct 11, 2012 at 9:10
  • $\begingroup$ Ok, now when you say "learning a distance" from a dataset B: do you mean "learning some distance threshold value" or "learning a distance metric function" (a sort of parametrised dissimilarity measure) ? $\endgroup$
    – shn
    Commented Oct 11, 2012 at 11:17
  • 1
    $\begingroup$ I mean the second, "learning a distance metric function". Upon more reading by the way, my simple A and B formulation above can be found in the quoted manuscript: "Given training examples of item sets with their correct clusterings, the goal is to learn a similarity measure so that future sets of items are clustered in a similar fashion." $\endgroup$
    – micans
    Commented Oct 11, 2012 at 11:30
  • 1
    $\begingroup$ Well, it seems then that "supervised clustering" is very similar to what is called "semi-supervised clustering". Until now, I don't really see any difference. By the way, in some other papers, the "(semi-)supervised clustering" do not refer to "creating a modified distance function" to be used to cluster future datasets in a similar fashion; it is rather about "modifying the clustering algorithm itself" without changing the distance function ! $\endgroup$
    – shn
    Commented Oct 11, 2012 at 11:48
6
$\begingroup$

Some definitions:

Supervised clustering is applied on classified examples with the objective of identifying clusters that have high probability density to a single class.

Unsupervised clustering is a learning framework using a specific object functions, for example a function that minimizes the distances inside a cluster to keep the cluster tight.

Semi-supervised clustering is to enhance a clustering algorithm by using side information in clustering process.

Advances in Neural Networks -- ISNN 2010

Without using too much jargon since I'm a novice in this area, the way I understand the supervised clustering is more the less like this:

In supervised clustering you start from the Top-Down with some predefined classes and then using a Bottom-Up approach you find which objects fit better into your classes.

For example, you performed an study regarding the favorite type of oranges in a population.
From the many types of oranges you found that a particular 'kind' of oranges is the preferred one.
However, that type of orange is very delicate and labile to infections, climate change and other environmental agents.
So you want to cross it over with other species that is very resistant to those insults.
Then you go to the lab and found some genes that are responsible for the juicy and sweet taste of one type, and for the resistant capabilities of the other type.
You perform several experiments and you end with let's say hundred different subtypes of oranges.
Now you are interested just in those subtypes that fit perfectly the properties described.
You don't want to perform the same study in your population again...
You know the properties you are looking for in your perfect orange.
So you run your cluster analysis and select the ones that fit best your expectations.

Improve this answer
$\endgroup$
1
  • $\begingroup$ This is a nice answer but fails to define what Classification is. $\endgroup$
    – adunaic
    Commented May 17, 2020 at 21:34
0
$\begingroup$

My interpretation has to do with the number of training samples you have per class.

If you have a lot of training samples per class, then you can reasonably train a classifier and you have a classification use case.

If you only have training samples for a fraction of the classes then a classifier would have poor performance, but a clusterer could be useful. You can optimize this clusterer with the labels you have (optimize the distance, features etc...) and hopefully this optimization will be useful on unlabelled data. You have a (semi) supervised clustering use case.

Improve this answer
$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.