Supervised clustering or classification? The second question is that I found in a discussion somewhere on the web talking about "supervised clustering", as far as I know, clustering is unsupervised, so what is exactly the meaning behind "supervised clustering" ? What is the difference with respect to "classification" ?
There are many links talking about that:
http://www.cs.uh.edu/docs/cosc/technical-reports/2005/05_10.pdf
http://books.nips.cc/papers/files/nips23/NIPS2010_0427.pdf
http://engr.case.edu/ray_soumya/mlrg/supervised_clustering_finley_joachims_icml05.pdf
http://www.public.asu.edu/~kvanlehn/Stringent/PDF/05CICL_UP_DB_PWJ_KVL.pdf
http://www.machinelearning.org/proceedings/icml2007/papers/366.pdf
http://www.cs.cornell.edu/~tomf/publications/supervised_kmeans-08.pdf
http://jmlr.csail.mit.edu/papers/volume6/daume05a/daume05a.pdf
etc ...
 A: Some definitions:
Supervised clustering is applied on classified examples with the objective of identifying clusters that have high probability density to a single class.
Unsupervised clustering is a learning framework using a specific object functions, for example a function that minimizes the distances inside a cluster to keep the cluster tight.
Semi-supervised clustering is to enhance a clustering algorithm by using side information in clustering process.
Advances in Neural Networks -- ISNN 2010
Without using too much jargon since I'm a novice in this area, the way I understand the supervised clustering is more the less like this:
In supervised clustering you start from the Top-Down with some predefined classes and then using a Bottom-Up approach you find which objects fit better into your classes.
For example, you performed an study regarding the favorite type of oranges in a population.
From the many types of oranges you found that a particular 'kind' of oranges is the preferred one.
However, that type of orange is very delicate and labile to infections, climate change and other environmental agents.
So you want to cross it over with other species that is very resistant to those insults.
Then you go to the lab and found some genes that are responsible for the juicy and sweet taste of one type, and for the resistant capabilities of the other type.
You perform several experiments and you end with let's say hundred different subtypes of oranges.
Now you are interested just in those subtypes that fit perfectly the properties described.
You don't want to perform the same study in your population again... 
You know the properties you are looking for in your perfect orange.
So you run your cluster analysis and select the ones that fit best your expectations.
A: My naive understanding is that classification is performed where you have a specified set of classes and you want to classify a new thing/dataset into one of those specified classes.
Alternatively, clustering has nothing to start with and you use all the data (including the new one) to separate into clusters.
Both use distance metrics to decide how to cluster/classify.  The difference is that classification is based off a previously defined set of classes whereas clustering decides the clusters based on the entire data.
Again my naive understand is that supervised clustering still clusters based on the entire data and thus would be clustering rather than classification.
In reality i'm sure the theory behind both clustering and classification are inter-twinned.
A: I don't think I know more than you do, but the links you posted do suggest answers. I'll take http://www.cs.cornell.edu/~tomf/publications/supervised_kmeans-08.pdf as an example. Basically they state: 1) clustering depends on a distance. 2) successful use of k-means requires a carefully chosen distance. 3) Given training data in the form of sets of items with their desired partitioning, we provide a structural SVM method that learns a distance measure so that k-means produces the desired clusterings. In this case there is a supervised stage to the clustering, with both training data and learning. The purpose of this stage is to learn a distance function so that applying k-means clustering with this distance will be hopefully optimal, depending on how well the training data resembles the application domain. All the usual caveats appropriate to machine learning and clustering still apply.
Further quoting from the article: Supervised clustering is the task of automatically adapting a clustering algorithm with the aid of a training set consisting of item sets and complete partitionings of these item sets.. That seems a reasonable definition.
A: My interpretation has to do with the number of training samples you have per class.
If you have a lot of training samples per class, then you can reasonably train a classifier and you have a classification use case.
If you only have training samples for a fraction of the classes then a classifier would have poor performance, but a clusterer could be useful. You can optimize this clusterer with the labels you have (optimize the distance, features etc...) and hopefully this optimization will be useful on unlabelled data. You have a (semi) supervised clustering use case.
