Take the 2-minute tour ×
Cross Validated is a question and answer site for people interested in statistics, machine learning, data analysis, data mining, and data visualization. It's 100% free, no registration required.

The second question is that I found in a discussion somewhere on the web talking about "supervised clustering", as far as I know, clustering is unsupervised, so what is exactly the meaning behind "supervised clustering" ? What is the difference with respect to "classification" ?

There are many links talking about that:

http://www.cs.uh.edu/docs/cosc/technical-reports/2005/05_10.pdf

http://books.nips.cc/papers/files/nips23/NIPS2010_0427.pdf

http://engr.case.edu/ray_soumya/mlrg/supervised_clustering_finley_joachims_icml05.pdf

http://www.public.asu.edu/~kvanlehn/Stringent/PDF/05CICL_UP_DB_PWJ_KVL.pdf

http://www.machinelearning.org/proceedings/icml2007/papers/366.pdf

http://www.cs.cornell.edu/~tomf/publications/supervised_kmeans-08.pdf

http://jmlr.csail.mit.edu/papers/volume6/daume05a/daume05a.pdf

etc ...

share|improve this question
    
please give link of "discussion somewhere on the web" –  Atilla Ozgur Sep 25 '12 at 21:37
1  
add comment

3 Answers

up vote 1 down vote accepted
+100

I don't think I know more than you do, but the links you posted do suggest answers. I'll take http://www.cs.cornell.edu/~tomf/publications/supervised_kmeans-08.pdf as an example. Basically they state: 1) clustering depends on a distance. 2) successful use of k-means requires a carefully chosen distance. 3) Given training data in the form of sets of items with their desired partitioning, we provide a structural SVM method that learns a distance measure so that k-means produces the desired clusterings. In this case there is a supervised stage to the clustering, with both training data and learning. The purpose of this stage is to learn a distance function so that applying k-means clustering with this distance will be hopefully optimal, depending on how well the training data resembles the application domain. All the usual caveats appropriate to machine learning and clustering still apply.

Further quoting from the article: Supervised clustering is the task of automatically adapting a clustering algorithm with the aid of a training set consisting of item sets and complete partitionings of these item sets.. That seems a reasonable definition.

share|improve this answer
    
The problem is simply: why do you want to learn a distance measure from a set of labelled training data, and then apply this distance measure with a clustering method; why you would not just use a supervised method. In other words, you want to do clustering (i.e. partitioning your dataset into clusters), but you assume that you already have the complete desired partitioning and that you will use it to learn a distance measure, then apply clustering on this dataset using this learned distance. At best, you'll get the same partitions that you used to learn the distance measure ! You already have –  shn Oct 10 '12 at 17:58
    
Where you write "then apply clustering on this datase" substitute "then apply clustering on similar datasets". It is this scenario: in experiment X we have data A and B. A is for clustering, B helps with learning the distance. B sets a gold standard and is presumably expensive to obtain. In subsequent experiments X2, X3 .. we obtain A but cannot afford to obtain B. –  micans Oct 11 '12 at 9:10
    
Ok, now when you say "learning a distance" from a dataset B: do you mean "learning some distance threshold value" or "learning a distance metric function" (a sort of parametrised dissimilarity measure) ? –  shn Oct 11 '12 at 11:17
    
I mean the second, "learning a distance metric function". Upon more reading by the way, my simple A and B formulation above can be found in the quoted manuscript: "Given training examples of item sets with their correct clusterings, the goal is to learn a similarity measure so that future sets of items are clustered in a similar fashion." –  micans Oct 11 '12 at 11:30
    
Well, it seems then that "supervised clustering" is very similar to what is called "semi-supervised clustering". Until now, I don't really see any difference. By the way, in some other papers, the "(semi-)supervised clustering" do not refer to "creating a modified distance function" to be used to cluster future datasets in a similar fashion; it is rather about "modifying the clustering algorithm itself" without changing the distance function ! –  shn Oct 11 '12 at 11:48
show 1 more comment

My naive understanding is that classification is performed where you have a specified set of classes and you want to classify a new thing/dataset into one of those specified classes.

Alternatively, clustering has nothing to start with and you use all the data (including the new one) to separate into clusters.

Both use distance metrics to decide how to cluster/classify. The difference is that classification is based off a previously defined set of classes whereas clustering decides the clusters based on the entire data.

Again my naive understand is that supervised clustering still clusters based on the entire data and thus would be clustering rather than classification.

In reality i'm sure the theory behind both clustering and classification are inter-twinned.

share|improve this answer
    
Clasifiers performs a supervised learning: they use the entire data and the class labels associated to each data-point. Clustering methods performs unsupervised learning: they use the entire data without having the class labels associated to each data-point. Now you didn't really answered what is "supervised clustering", it seems that for you it is like clustering, however, clustering is unsupervised ! Then, my understanding is that a supervised clustering method is actually a classifier since it performs supervised learning. –  shn Oct 10 '12 at 14:35
add comment

Some definitions:

Supervised clustering is applied on classified examples with the objective of identifying clusters that have high probability density to a single class.

Unsupervised clustering is a learning framework using a specific object functions, for example a function that minimizes the distances inside a cluster to keep the cluster tight.

Semi-supervised clustering is to enhance a clustering algorithm by using side information in clustering process.

Advances in Neural Networks -- ISNN 2010

Without using too much jargon since I'm a novice in this area, the way I understand the supervised clustering is more the less like this:

In supervised clustering you start from the Top-Down with some predefined classes and then using a Bottom-Up approach you find which objects fit better into your classes.

For example, you performed an study regarding the favorite type of oranges in a population.
From the many types of oranges you found that a particular 'kind' of oranges is the preferred one.
However, that type of orange is very delicate and labile to infections, climate change and other environmental agents.
So you want to cross it over with other species that is very resistant to those insults.
Then you go to the lab and found some genes that are responsible for the juicy and sweet taste of one type, and for the resistant capabilities of the other type.
You perform several experiments and you end with let's say hundred different subtypes of oranges.
Now you are interested just in those subtypes that fit perfectly the properties described.
You don't want to perform the same study in your population again...
You know the properties you are looking for in your perfect orange.
So you run your cluster analysis and select the ones that fit best your expectations.

share|improve this answer
add comment

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.