Effective data visualizations for large numbers of classes

Question

Experimenting with a few approaches to visualize error modes/rates for models with a large number (100-500) of classes. For smaller numbers of classes have been using ROC graphs (with each class represented as a separate line) confusion matrices, and some simple bar charts just displaying accuracy.

When it comes to hundreds of classes though, these visualizations are generally ineffective at conveying information. Have been looking at confusion matrix variants such as CMO (confusion matrix ordering), and some other confusion-matrix-centric options. Things like displaying groups of classes with high mutual confusion rates, and pulling out strong vertical and horizontal lines.

However, outside of expecting people to scroll through individual error rates for hundreds of classes, or the confusion-matrix variants above, I haven't found any techniques that are effective for hundreds of classes.

Have people found certain visualizations that are successful in these cases? I'm specifically dealing with classification in this case, though I'm dealing both with single-label and multi-label scenarios (where individual class labels either are or are not mutually exclusive). However, suggestions for other types of problems that may be adapted to classification would also be very helpful.

Answer 1

If you can look at only one number per category, there are some bar chart extensions to handle 100s of categories: wrapped bars, piled bars, packed bars, and combined "other" bars. Additionally treemaps and 1-D labeled scatter plots can work. All of those work best when the categories are ordered by a ranking, but I think wrapping could work with less space efficiency with any ordering if you need, say, alphabetical lookup. Labeling is always a challenge -- some of these sacrifice labeling of smaller items (relying on hover only).

For something more than one number per category, there is the small multiples technique. Here is a grid of 400 ROC curves, with background color representing some metric such as area.

Labeling is still a challenge. For a static view, one could label each row and column and provide a legend for the category of each row-column combination.