Are there any classifiers that can do hierarchal classification? For example, we may have some text and want to predict it's class in an ontology (which is a tree). As a concrete example, let's say we have some text such as "iphone 7 case" and we want to learn it maps to the path [electronics -> smart-phones & accessories -> cases] in an ontology of products. See imagenet for an example of images with hierarchal label structure).

The common naive solution is to flatten the problem into a multi-class classification problem but this simplification has two issues that a model aware of hierarchy could address:

  1. If a particular class is rare or even missing in the training data than a multi-class classifier does not have enough information to make a good prediction. A hierarchal classifier could utilize related examples that have a common ancestry with the rare class though.

  2. A multi-class classifier's loss function does not give any credit if the predicted class is "almost right" (e.g. most of the path is correct but it may have predicted the child, a parent, or a sibling instead of the correct class). A hierarchal classifier's loss function could give partial credit if the path was mostly correct.

Does anyone know of any models that can do hierarchal classification?


1 Answer 1


There is a whole literature on the subject. The function hclust in R has a wide array of algorithms that do hierarchical clustering: Complete Linkage Analysis, and Ward's D2 minimiazation, Unweighted Pair Group Method with Arithmetic Mean (UPGMA), McQuitty's weighted PGMA, Weighted Pair Group Median Cluster (WPGMC) and Unweighted Pair Group Centroid Clustering (UPGMC, not sure why the M is there). These are discussed in Murtagh's 1985 text "Multidimensional Clustering Algorithms", a more modern reference is Legendre/Legendre's "Numerical Ecology". Freely available lecture notes are available here.Your concerns are inline with the overall rational for hierarchical clustering.

Your last question is for models that do hierarchical clustering. I am not sure which of the above algorithms are actually a model-based approach.

  • 1
    $\begingroup$ I am not sure how hierarchal clustering would be useful for classification since an ontology tree already defines the hierarchal structure (so there is no hierarchal structure to learn). $\endgroup$ Commented Apr 10, 2018 at 20:43
  • $\begingroup$ @anthonybell Perhaps I misunderstood the question. Are you saying you have a known hierarchy and you wish to classify observations according to it? $\endgroup$
    – AdamO
    Commented Apr 10, 2018 at 21:04
  • 1
    $\begingroup$ yes the target of classification is a node in a tree that's already defined (such as wordnet, imagenet, taxonomy) $\endgroup$ Commented Apr 10, 2018 at 21:46
  • $\begingroup$ @anthonybell that doesn't make sense. How do you classify a node? It is a structure you've already set. What are your data? What are you observing and how does it relate to this "tree"? $\endgroup$
    – AdamO
    Commented Apr 10, 2018 at 22:04
  • 1
    $\begingroup$ The input is text/image and the output is a node in a tree (see imagenet as an example). $\endgroup$ Commented Apr 10, 2018 at 22:35

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.