Hi I have data set with a set of variables and known class labels. I am trying to compare why a supervised approach will work theoretically better compared to a unsupervised approach for classification vs clustering in this case. If I remove the labels from my data, I can cluster the vector of independent variables of each row in my dataset and then figure out the labels based on the similarities of the data points in a cluster. Please suggest a way to understand why supervised methods may outperform unsupervised methods in my case. This may be a very rudimentary question but it is very important for me to know. What does the class label add mathematically (minimizing the generalization error may be) that clustering does not? Please let me know if you all need more info.

  • 1
    $\begingroup$ The class label adds the ground truth, real world knowledge. It is therefore the 'gold standard'. You can do your clustering and then see for yourself how well it performed in putting the observations with the same labels together. $\endgroup$ – Zhubarb Aug 14 '14 at 7:51

First thing is more information allows better decisions(Assuming the information is not redundant).

Suppose you use K-means for clustering. Then in simple terms you are assuming something about the data(Distribution being spherical for all clusters for example). Without labels model does not know if the clusters formed are better or worse. Suppose you data is in n dimensional space and there are two classes each lying in two separate parallel planes then you assumption of the data is wrong assuming you use kmeans and We are assuming the clusters will give us correct labels. So this wont work in this situation. But if you have two spherical cloud of data then kmeans with two cluster will do the trick.

But in supervised learning setting, we can make any linear/non-linear boundary to make correct prediction on the training set(and use validation set to control model complexity assuming validation error might be a surrogate to generalization error). With a way to control the complexity of the model, the objective function directly minimize the training error and if we choose correct complexity through cross validation we might be trying to minimize the generalization error directly.

In unsupervised model, the objective function in no way tries to directly minimize even the training error.


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.