2
$\begingroup$

gradient boosting is a supervised learning algorithm that splits/grows decision trees to improve predictions iteratively.

hierarchical clustering is an unsupervised learning algorithm that splits/grows dendrograms to partition features.

Both decision trees and dendrograms look very similar, so how are they different in how they split binarily? is clustering and boosting interrelated because of them

$\endgroup$

1 Answer 1

2
$\begingroup$

In supervised learning you already have class labels. So the decision tree splits based on your features which give the maximum entropy/Gini impurity gain over that class label. Intuitively, Decision Trees look forward to create as pure nodes as possible by splitting on several features, such that the leaf nodes have near 0 entropy (this also depends on the depth of tree that you'd like to build).

Now hierarchical clustering is a part of unsupervised learning algorithm. You don't have class labels in case of unsupervised learning. So what unsupervised learning algorithms intuitively do is that they look for items that are close/similar in the space of the features based on the similarity scores/distance measures. In hierarchical clustering, suppose you are looking at agglomerative clustering. What it does is that it creates clusters based on the distances/similarities of the points from one another. If two clusters are quite far apart/have low similarity score, they would be combined together only if there is no other cluster closer to it geometrically. There are various methods of combining clusters, such as max, min, group average, ward's method, etc. Divisive algorithm also similar to that, except that they go from one big cluster to small clusters, i.e. they look to break a large cluster into small cluster based on the distances/similarity measures.

Note that boosting can be applied to many types of applications. There have been some publications about boosting in clustering, where in several weak learners are combined, in which the hard points which are tough to cluster are sent through multiple weak learners. You may check this paper out: https://www.sciencedirect.com/science/article/abs/pii/S016786550300285X#:~:text=The%20boost%2Dclustering%20algorithm%20is,provide%20a%20new%20data%20partitioning.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.