# Understanding the difference between Supervised and unsupervised learning?

I have been reading about the Supervised and Unsupervised learning. What I came to know through this link is that in case of Supervised learning you have a set of input and a set of labels which are then used in the training phase whereas in case Unsupervised learning there are no labels. Now my basic question is:

1. Since there are no labels available in case of Unsupervised learning so does that mean that there is no training phase in unsupervised learning?

2. If there is a training phase in Unsupervised learning and we do not use any labels then what is the importance of training phase and how is it performed since we do not have any error or reward signal to evaluate a potential solution in training phase? Isn't it equal to the normal testing phase where we have the set of data but no labels and we predict the labels?

3. I was reading about a Nonparametric bayesian modeling method and the introduction said that-"The method is unsupervised and has no free parameter that require tuning". In this what is the meaning of no free parameter that require tuning? Is it that there is no requirement of training phase?

Please bear with my lack of understanding about these topics as I am new to this and would love if you could provide some explanation to clear out my doubts.

• It would help to have a link or citation to where you read that about nonparametric Bayesian modeling. I think there are versions of that that are not unsupervised. Jul 15, 2014 at 13:07
• I was reading about a method developed at MIT which is nonparametric bayesian modeling method Jul 15, 2014 at 13:08
• @gung the method is called corsscat.The intro said that it has no free parameter that require tuning. what does that mean? Jul 15, 2014 at 13:15

1.

Lets look at a simple example of trying to predict housing prices. Assume we have a dataset that looks like

Cost |  Sq Ft  | N bedroom
100K    1,800     4
120K    1,300     3
220K    2,200     5


In the case of supervised learning we would know the cost (these are our y labels) and we would use our set of features (Sq ft and N bedrooms) to build a model to predict the housing cost. The formula would look like

Cost ~ Sq Ft + N bedrooms


Now in unsupervised learning we would not know the cost of the house but we still would know the features. Therefore, we would train a model and try to group the types of houses together that are similar. For an example of this look at k-means clustering (http://scikit-learn.org/stable/modules/clustering.html#clustering)

This is a great, free, book which covers this very nicely (http://web.stanford.edu/~hastie/local.ftp/Springer/OLD/ESLII_print4.pdf)

2.

Each type of learning method (may) have a set of parameters which are called model parameters. The training phase is used to find out the optimal set of parameters which generalizes you data the best. That book also gives very nice information on different learning methods are there parameters.

For example, in the leaning algorithm called SVM there is a term that looks like $\exp(-\gamma|x-x^{2}|)$. In this example the $\gamma$ parameter is what we try to optimize using the training data.

• Thanks for reference. I am trying to get it but what does then it means when we say "The unsupervised method has no free parameter that require tuning"? Jul 15, 2014 at 13:18
• It simply means that the functional form of the model has no free parameters (like the $\gamma$ parameter) which I gave you an example of. Jul 15, 2014 at 13:22
• ok. So in above example if we do not have the output cost during training phase to verify incase of unsupervised learning and we train a model to do the prediction then isn't it same as the testing phase? Jul 15, 2014 at 13:42
• I think you are still very confused about all the topics being discussed here and suggest you watch the first two weeks, and week 10, of the lectures here (class.coursera.org/ml-005/lecture/preview) This will cover exactly what you are looking to know and is probably the best reference material on the web for beginners. Jul 15, 2014 at 13:46