As far as I know, both autoencoders and t-SNE are used for nonlinear dimensionality reduction. What are the differences between them and why should I use one versus another?
$\begingroup$
$\endgroup$
asked May 20, 2015 at 23:39
Add a comment
|
4 Answers
$\begingroup$
$\endgroup$
3
Both of them try to find a lower dimensionality embedding of your data. However, there are different minimization problems. More specifically, an autoencoder tries to minimize the reconstruction error, while t-SNE tries to find a lower dimensional space and at the same time it tries to preserve the neighborhood distances. As a result of this attribute, t-SNE is usually preferred for plots and visualizations.
answered May 23, 2015 at 20:14
-
1$\begingroup$ So in this sense, does it mean autoencoder is better to find lower dimension when the lower dimension is more than 3D? (Because I assume if the lower dimension is large, t-SNE may not work that well?) $\endgroup$ May 23, 2015 at 23:58
-
3$\begingroup$ There are many types of autoencoders (sparse, variational, stacked, convolutional etc.) depending on your task. They can be very efficient in discovering lower dimensional embeddings, based on reconstruction error. Therefore, if your task is to find an optimal lower dimensional space (even 2D) I would suggest you to pick the right autoencoder for your task. If you have to do visualisations then t-SNE would probably be your choice. as the neighbouring distances preservation, can result better visualisations. $\endgroup$ May 24, 2015 at 1:05
-
$\begingroup$ References to some of these would be (very very) nice. :) $\endgroup$ Jun 11, 2021 at 1:27
$\begingroup$
$\endgroup$
2
[Autoencoders] primarily focus on maximizing the variance of the data in the latent space, as a result of which autoencoders are less successful in retaining the local structure of the data in the latent space than manifold learners...
From "Learning a Parametric Embedding by Preserving Local Structure", Laurens van der Maaten (https://lvdmaaten.github.io/publications/papers/AISTATS_2009.pdf)
-
$\begingroup$ This is not "the t-SNE paper". Standard t-SNE is non-parametric. $\endgroup$– amoebaJul 1, 2019 at 8:14
-
1$\begingroup$ Thanks @amoeba for your comment on this. I edited the post. $\endgroup$ Jul 1, 2019 at 8:24
$\begingroup$
$\endgroup$
Autoencoder and t-SNE can be used together for better visualization in high dimensional data, as described in [1]:
For 2D visualization specifically, t-SNE is probably the best algorithm around, but it typically requires relatively low-dimensional data. So a good strategy for visualizing similarity relationships in high-dimensional data is to start by using an autoencoder to compress your data into a low-dimensional space (e.g. 32 dimensional), then use t-SNE for mapping the compressed data to a 2D plane.
[1] https://blog.keras.io/building-autoencoders-in-keras.html
$\begingroup$
$\endgroup$
Autoencoder is designed to preserve previous data in a 2-norm sense, which can be thought as preserve the kinetic energy of the data, if data is velocity.
While t-SNE, use KL divergence which is not symmetrical, it will lead to t-SNE focus more on local structure, while autoencoder tends to keep overall L2 error small, which is in a global sense.
answered Oct 24, 2017 at 5:32