3
$\begingroup$

Reading about t-SNE, and looking at the pretty plots, it seems to be very good at separating things that we "expect" to be separate in low dimensions. Why wouldn't we use this to do dimensionality reduction before using some sort of classification algorithm (something data-hungry like a DNN for example)?

EDIT: to rephrase and generalize slightly further, since t-SNE preserves separatbility so well when it is nicely tuned, why not go for t-SNE in 2 or 3-D and then a nonlinear classifier, instead of a standard $k$-dimension reduction like PCA or ICA?

Improve this question
$\endgroup$
5
  • $\begingroup$ Possible duplicate: stats.stackexchange.com/questions/263539/… $\endgroup$
    – generic_user
    Commented Oct 26, 2017 at 13:10
  • 1
    $\begingroup$ I think this is a little different, just because KMM for example is a generative approach, and as the answer well points out much information is lost in terms of density. But if separation is preserved, then why not run a complicated discriminative approach? $\endgroup$
    – bibliolytic
    Commented Oct 26, 2017 at 13:24
  • $\begingroup$ IDK, is separation truly preserved? $\endgroup$
    – generic_user
    Commented Oct 26, 2017 at 13:44
  • $\begingroup$ Heuristically speaking I'd say yes, to look at van Maaten's papers $\endgroup$
    – bibliolytic
    Commented Nov 1, 2017 at 16:01
  • $\begingroup$ "something data-hungry like a DNN for example" neural nets don't mind high dimensional space, but if you want to use a projection (to make it interpretable), you might as well let the nnet pick which one to use: arxiv.org/abs/1902.10527 $\endgroup$
    – John Madden
    Commented Apr 5, 2023 at 13:19

1 Answer 1

Reset to default
1
$\begingroup$

I agree that this sounds like a great idea. Indeed, some of my t-SNE plots have shown great separation of the data in just two dimensions. The trouble is that t-SNE gives no ability to map new points into the low-dimensional space, unlike, for instance, PCA, which gives you a linear transformation to apply to new points. Thus, you cannot make predictions on new data or even perform standard techniques like using test sets, cross validation, or bootstrap validation.

Improve this answer
$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.