2
$\begingroup$

I'm running many combinations of hyperparameters, such that each combination of transformation (e.g. preprocessing, classifier) is combined with all others in a Cartesian product, along with a fit metric (accuracy). One possible solution would be a table where each column is a classifier, each row is a preprocessor, however in this case, I'm working with greater than two hyperparameters.

What's a good, interactive way to display this data to a user (e.g. a person interested in exploring the data)?

Improve this question
$\endgroup$
2
  • $\begingroup$ You just want people to see that these combinations exist, or you want to show them something about each combination (such as a fit metric for a model w/ that combination of hyperparameters), or want them to compare fit metrics w/ different combinations, etc? Who are the "users"? What is the context in which they will use your display? Etc. $\endgroup$
    – gung - Reinstate Monica
    Commented Oct 9, 2017 at 17:40
  • $\begingroup$ Are there 3 types of hyperparameters? How many options exist w/i each type? Are you planning on creating some kind of application for users? $\endgroup$
    – gung - Reinstate Monica
    Commented Oct 9, 2017 at 18:04

1 Answer 1

Reset to default
1
$\begingroup$

Displaying in a 3-dimensional plot, that you can move around, zoom and so on is not a terrible option.

You'll need to project the high-dimensional data onto 3 dimensions of course. Two methods to do this are t-SNE, and PCA.

PCA projects onto axes which can maximize the amount of variance of the resulting plot, and minimize the residual maintenance that was lost during the projection. This is a fairly straightforward projection to understand intuitively. The downside is that you'll lose correlations that need some more manifold-like projection to show.

t-SNE is sort of the opposite: it projects onto a potentially very convoluted, complex manifold, that doesnt need to have any kind of global coherence in any way. It can represent local structure fairly well, and handle high-dimensional manifolds, but it loses any sense of the actual global structure.

As an example of t-SNE, if you have two interlocking rings, t-SNE can show them as two flat, separated, non-interlocking rings. This page https://distill.pub/2016/misread-tsne/ shows some very interesting examples:

enter image description here

At a practical level, an implementation of both a t-SNE projector and viewer, and a PCA projector and viewer is in the Tensorflow Tensorboard. https://www.tensorflow.org/programmers_guide/embedding#visualizing_embeddings

Improve this answer
$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.