4
$\begingroup$

I have implemented dimensionality reduction using PCA, AE, and t-SNE! I am trying to compare the output of the three methods by finding the reconstruction error. I have managed to reconstruct the data from PCA and AE with no issues. however, I couldn't figure out how to reconstruct it from t-SNE. I have looked through the literature and can't yet find any straightforward answer. My question has two parts:

  1. Can i reconstruct the data from the output by t-SNE?

  2. If so, how is it done? (any references or implementations would be great)

$\endgroup$
3
  • 3
    $\begingroup$ If you are able to perfectly reconstruct your original data from your PCA, then you haven't done any dimensionality reduction. Typically, you run PCA, and then only keep the principal components which account for a significant amount of the variation in your data, and throw away the rest. This yields a lower-dimensional dataset compared to what you started with, from which you cannot perfectly reconstruct your original data. If you're just running PCA and keeping all the PCs, you haven't reduced dimensionality at all, you've just mapped your data onto a different coordinate system! $\endgroup$ Sep 17, 2018 at 17:55
  • 2
    $\begingroup$ @NuclearWang If the data lie in a low dimensional linear subspace, then PCA can indeed reduce the dimensionality while maintaining perfect reconstruction performance. Of course, this is a reduction in the extrinsic dimensionality, but not the intrinsic dimensionality, and reducing beyond the intrinsic dimensionality would introduce error into the reconstructions, as you mentioned. $\endgroup$
    – user20160
    Sep 17, 2018 at 22:42
  • $\begingroup$ You don't answer his question. No, it's not possible to reconstruct the data after a t-SNE plot has been done. But with PCA, it's possible. $\endgroup$
    – euraad
    Dec 2, 2021 at 23:03

2 Answers 2

5
$\begingroup$

You probably misunderstood how t-SNE works. Instead of performing some invertible transformation, it finds an embedding of a dataset into a low-dimensional space by minimizing some non-convex loss function.

Quoting the author's website:

Once I have a t-SNE map, how can I embed incoming test points in that map?

t-SNE learns a non-parametric mapping, which means that it does not learn an explicit function that maps data from the input space to the map. Therefore, it is not possible to embed test points in an existing map (although you could re-run t-SNE on the full dataset). A potential approach to deal with this would be to train a multivariate regressor to predict the map location from the input data. Alternatively, you could also make such a regressor minimize the t-SNE loss directly, which is what I did in this paper.

To put it very bluntly: you put the data in and you get some embedding out. But there is no way to use this "fitted" embedding for new points or to recover their original representations. These are, among others, the reasons why t-SNE is actually mainly aimed for data visualization.

$\endgroup$
0
$\begingroup$

Okay, I have a good news for you. I'm doing some research on tsne as well. Definitely normal tsne is a non-parametric mapping, meaning there is no function which is learned while transforming the data from higher dimension to lower dimension. So, parametric-tsne comes to our rescue where it learns this mapping. There is implementation of the same here.

$\endgroup$
5
  • $\begingroup$ This yields a function to map from higher dimensions to lower dimensions, but the OP is trying to go the other way. Mapping back from lower dimensions to higher dimensions is in most cases fundamentally impossible; information is usually irrevocably lost when you reduce the dimensionality in the first place. $\endgroup$ Aug 21, 2023 at 14:29
  • $\begingroup$ true but that is what happens when u reduce the dimension by any dimensionality technique, right? Here u can change the layers similar to AE and use the transform method to get the reconstructed data $\endgroup$ Aug 21, 2023 at 14:38
  • $\begingroup$ Correct, nothing particularly special about t-SNE in this regard - unless you have redundant or colinear features, any dimensionality reduction method will lose some information about the original data. A good DR method will keep the "relevant" signal, but you'll never be able to reconstruct the original data perfectly. $\endgroup$ Aug 21, 2023 at 14:43
  • $\begingroup$ This does not provide an answer to the question. Once you have sufficient reputation you will be able to comment on any post; instead, provide answers that don't require clarification from the asker. - From Review $\endgroup$ Aug 21, 2023 at 14:47
  • $\begingroup$ Yes but if I want to reduce the dimension of data based on local nighborhood? The link which I shared above uses an AE with the tsne loss function which tries to prevent the local neighbourhood. $\endgroup$ Aug 21, 2023 at 14:49

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.