2
$\begingroup$

I have implemented dimensionality reduction using PCA, AE, and t-SNE! I am trying to compare the output of the three methods by finding the reconstruction error. I have managed to reconstruct the data from PCA and AE with no issues. however, I couldn't figure out how to reconstruct it from t-SNE. I have looked through the literature and can't yet find any straightforward answer. My question has two parts:

  1. Can i reconstruct the data from the output by t-SNE?

  2. If so, how is it done? (any references or implementations would be great)

$\endgroup$
  • 3
    $\begingroup$ If you are able to perfectly reconstruct your original data from your PCA, then you haven't done any dimensionality reduction. Typically, you run PCA, and then only keep the principal components which account for a significant amount of the variation in your data, and throw away the rest. This yields a lower-dimensional dataset compared to what you started with, from which you cannot perfectly reconstruct your original data. If you're just running PCA and keeping all the PCs, you haven't reduced dimensionality at all, you've just mapped your data onto a different coordinate system! $\endgroup$ – Nuclear Wang Sep 17 '18 at 17:55
  • 1
    $\begingroup$ @NuclearWang If the data lie in a low dimensional linear subspace, then PCA can indeed reduce the dimensionality while maintaining perfect reconstruction performance. Of course, this is a reduction in the extrinsic dimensionality, but not the intrinsic dimensionality, and reducing beyond the intrinsic dimensionality would introduce error into the reconstructions, as you mentioned. $\endgroup$ – user20160 Sep 17 '18 at 22:42
4
$\begingroup$

You probably misunderstood how t-SNE works. Instead of performing some invertible transformation, it finds an embedding of a dataset into a low-dimensional space by minimizing some non-convex loss function.

Quoting the author's website:

Once I have a t-SNE map, how can I embed incoming test points in that map?

t-SNE learns a non-parametric mapping, which means that it does not learn an explicit function that maps data from the input space to the map. Therefore, it is not possible to embed test points in an existing map (although you could re-run t-SNE on the full dataset). A potential approach to deal with this would be to train a multivariate regressor to predict the map location from the input data. Alternatively, you could also make such a regressor minimize the t-SNE loss directly, which is what I did in this paper.

To put it very bluntly: you put the data in and you get some embedding out. But there is no way to use this "fitted" embedding for new points or to recover their original representations. These are, among others, the reasons why t-SNE is actually mainly aimed for data visualization.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.