Cross Validated is a question and answer site for people interested in statistics, machine learning, data analysis, data mining, and data visualization. It only takes a minute to sign up.
Sign up to join this community
In https://huyenchip.com/machine-learning-systems-design/design-a-machine-learning-system.html#design-a-machine-learning-system-dwGQI5R, I came across the sentence:
Similarly, you can use unsupervised learning to learn labels for your data, then use those labels for supervised learning.
I have never heard of the bolded part before. How exactly do you use unsupervised learning to "learn labels" for labeless data?
Normally, you don't (and you don't believe everything someone writes somewhere on the internet).
What the writer probably meant (at least that's my interpretation) is that you can use clustering to identify the clusters, declare each cluster to be a class for itself, and use these "classes" to learn class boundaries or other rules for "classifying" new data.
This approach, however, is likely to suffer from severe generalisation issues, if it works at all. If the true classes overlap, clustering won't be able to identify them and the clusters will not correspond to the classes. Even if the clusters/classes are well separated, lack of true labels will prevent you from tuning hyperparameters and ensuring good generalisation. So, it is a theoretically possible concept, but unlikely to work in practice.
I also stumbled over the preceding sentence in the blog you quoted:
An income prediction task can be regression if we output raw numbers, but if we quantize the income into different brackets and predict the bracket, it becomes a classification problem.
Again, it is theoretically possible, but not a recommended approach. By treating income prediction as a classification task we ignore (lose information about) the similarity between different "classes". The bracket [20,000 - 30,000] is closer to the bracket [30,000 - 40,000] than to [150,000 - 200,000]. Classification wouldn't take this into account. See my answer here for more details.
This pops up a lot when labelling your full data set is expensive and time consuming. A simple example would be labelling product reviews into buckets such as:
What we may do is label a small fraction of our dataset and then we can cluster the word vecs, do knn with them, or do some analysis to pull out keywords to then label the rest (although technically not unsupervised but the easiest to explain). For example, the word 'price' pops up mostly for reviews about price (unsurprisingly). So, if we see that word we can just label it price and let the machine learn the label and hopefully generalize better than just mapping keywords to labels (it usually does). Alternatively, with clustering we would hope that reviews with the word 'price' would get lumped with the other price labels.
Obviously, this approach will add error over labelling everything but it can definitely get you closer to your end goal.
This type of approach is called 'semi-supervised' learning.
Unsupervised methods usually assign data points to clusters, which could be considered algorithmically generated labels. We don't "learn" labels in the sense that there is some true target label we want to identify, but rather create labels and assign them to the data. An unsupervised clustering will identify natural groups in the data, and you can interpret those groups to come up with meaningful labels instead of "Cluster 1", "Cluster 2", etc. - perhaps a patient cluster represents some aspect of biology, or some group of transactions represents fraud. The clustering assigns arbitrary categorical "labels" which can be further analyzed to discern whether they represent true, meaningful classes in your data.
If you have a useful clustering, you can then use those labels in a supervised manner to train a classifier. Rather than clustering every patient or transaction dataset and hoping to find the same clusters, you can train a classifier to use cluster-discriminative gene signatures or fraud profiles in order to directly assign the "labels" you discovered through unsupervised clustering to new data.
