Skip to main content
195 votes

Clustering on the output of t-SNE

The problem with t-SNE (and UMAP) is that it does not preserve distances nor density. It only to some extent preserves nearest-neighbors. The difference is subtle, but affects any density- or distance ...
Erich Schubert's user avatar
124 votes
Accepted

Why do we use Kullback-Leibler divergence rather than cross entropy in the t-SNE objective function?

KL divergence is a natural way to measure the difference between two probability distributions. The entropy $H(p)$ of a distribution $p$ gives the minimum possible number of bits per message that ...
user20160's user avatar
  • 32.8k
104 votes

Are there cases where PCA is more suitable than t-SNE?

$t$-SNE is a great piece of Machine Learning but one can find many reasons to use PCA instead of it. Of the top of my head, I will mention five. As most other computational methodologies in use, $t$-...
usεr11852's user avatar
  • 44.7k
72 votes

Clustering on the output of t-SNE

I would like to provide a somewhat dissenting opinion to the well argued (+1) and highly upvoted answer by @ErichSchubert. Erich does not recommend clustering on the t-SNE output, and shows some toy ...
amoeba's user avatar
  • 106k
43 votes
Accepted

Why is t-SNE not used as a dimensionality reduction technique for clustering or classification?

The main reason that $t$-SNE is not used in classification models is that it does not learn a function from the original space to the new (lower) dimensional one. As such, when we would try to use our ...
usεr11852's user avatar
  • 44.7k
35 votes
Accepted

Should data be centered+scaled before applying t-SNE?

Centering shouldn't matter since the algorithm only operates on distances between points, however rescaling is necessary if you want the different dimensions to be treated with equal importance, since ...
jon_simon's user avatar
  • 2,049
33 votes
Accepted

Intuitive explanation of how UMAP works, compared to t-SNE

You said that your understanding of t-SNE is based on https://www.youtube.com/watch?v=NEaUSP4YerM and you are looking for an explanation of UMAP on a similar level. I watched this video and it is ...
amoeba's user avatar
  • 106k
26 votes
Accepted

How can t-SNE or UMAP embed new (test) data, given that they are nonparametric?

Great question. I will answer it using t-SNE because I assume it is familiar to more people. I think UMAP is very promising and is a great contribution but to be honest I am getting a little bit ...
amoeba's user avatar
  • 106k
26 votes

Intuitive explanation of how UMAP works, compared to t-SNE

The main difference between t-SNE and UMAP is the interpretation of the distance between objects or "clusters". I use the quotation marks since both algorithms are not meant for clustering - they are ...
Edgar's user avatar
  • 1,701
24 votes
Accepted

What is the meaning of the axes in t-SNE?

Individual axes in t-SNE have no meaning at all. Algorithms such as MDS, SNE, t-SNE, etc. only care about pairwise distances between points. They try to position the points on a plane such that the ...
amoeba's user avatar
  • 106k
23 votes
Accepted

t-SNE versus MDS

PCA selects influential dimensions by eigenanalysis of the N data points themselves, while MDS selects influential dimensions by eigenanalysis of the $N^2$ data points of a pairwise distance matrix. ...
aminorex's user avatar
  • 346
21 votes

Are there cases where PCA is more suitable than t-SNE?

https://stats.stackexchange.com/a/249520/7828 is an excellent general answer. I'd like to focus a bit more on your problem. You apparently want to see how your samples relate with respect to your 7 ...
Has QUIT--Anony-Mousse's user avatar
20 votes
Accepted

Choosing the hyperparameters using T-SNE for classification

I routinely use $t$-SNE (alongside clustering techniques - more on this in the end) to recognise/assess the presence of clusters in my data. Unfortunately to my knowledge there is no standard way to ...
usεr11852's user avatar
  • 44.7k
18 votes

Should dimensionality reduction for visualization be considered a "closed" problem, solved by t-SNE?

Definitely not. I agree that t-SNE is an amazing algorithm that works extremely well and that was a real breakthrough at the time. However: it does have serious shortcomings; some of the ...
amoeba's user avatar
  • 106k
16 votes
Accepted

How to determine parameters for t-SNE for reducing dimensions?

I highly reccomend the article How to Use t-SNE Effectively. It has great animated plots of the tsne fitting process, and was the first source that actually gave me an intuitive understanding of what ...
Zach's user avatar
  • 24k
13 votes

Are there any versions of t-SNE for streaming data?

When dealing with streaming data, you might not want/need to embed all the points in history in a single t-SNE map. As an alternative, you can perform an online embedding by following these simple ...
Stéphane Deny's user avatar
12 votes
Accepted

Why does larger perplexity tend to produce clearer clusters in t-SNE?

The larger the perplexity, the more non-local information will be retained in the dimensionality reduction result. Yes, I believe that this is a correct intuition. The way I think about perplexity ...
amoeba's user avatar
  • 106k
11 votes

When is t-SNE misleading?

Out of the box, tSNE has a few hyperparameters, the main one being perplexity. Remember that heuristically, perplexity defines a notion of similarity for tSNE and a universal perplexity is used for ...
Alex R.'s user avatar
  • 14k
11 votes

Clustering on the output of t-SNE

I think with large perplexity t-SNE can reconstruct the global topology, as indicated in https://distill.pub/2016/misread-tsne/. From the fish image, I sampled 4000 points for t-SNE. With a large ...
renxwise's user avatar
  • 127
10 votes
Accepted

Comparing t-SNE solutions using their Kullback-Leibler divergences

Unfortunately, no; comparing the optimality of a perplexity parameter through the correspond $KL(P||Q)$ divergence is not a valid approach. As I explained in this question: "The perplexity parameter ...
usεr11852's user avatar
  • 44.7k
10 votes

What's wrong with t-SNE vs PCA for dimensional reduction using R?

You have to understand what TSNE does before you use it. It starts by building a neighboorhood graph between feature vectors based on distance. The graph ...
Conic's user avatar
  • 259
10 votes
Accepted

What classification algorithm should one use after seeing that t-SNE separates classes well?

First a brief answer, and then a longer comment: Answer SNE techniques compute an N ×N similarity matrix in both the original data space and in the low-dimensional embedding space in such a way that ...
Zhubarb's user avatar
  • 8,339
10 votes
Accepted

Why can't t-SNE capture a simple parabola structure?

Three general remarks: t-SNE is excellent at preserving cluster structure but is not very good at preserving continuous "manifold structure". One famous toy example is the Swiss roll data set, and it ...
amoeba's user avatar
  • 106k
10 votes
Accepted

t-SNE with mixed continuous and binary variables

Disclaimer: I only have tangential knowledge on the topic, but since no one else answered, I will give it a try Distance is important Any dimensionality reduction technique based on distances (tSNE, ...
Martin Modrák's user avatar
9 votes
Accepted

PCA too slow when both n,p are large: Alternatives?

Question 1: Let's say you have observed a data matrix $X \in \mathbb R^{n \times p}$. From this you can compute the eigendecomposition $X^T X = Q \Lambda Q^T$. The question now is: if we get new data ...
jld's user avatar
  • 20.4k
9 votes

How to interpret t-SNE plot?

Unlike PCA, axes in the low dimensional space don't have a particular meaning. In fact, one could arbitrarily rotate the low dimensional points and the t-SNE cost function wouldn't change. Furthermore,...
user20160's user avatar
  • 32.8k
9 votes
Accepted

Why does t-SNE not separate linearly separable classes?

Yes. You can use the following code to convince yourself. ...
RUser4512's user avatar
  • 10.4k
8 votes

Are there any versions of t-SNE for streaming data?

There is a recently published variant, called A-tSNE, which supports dynamically adding new data and refining clusters either based on interest areas or by user input. The paper linked below has some ...
cvlad's user avatar
  • 181
8 votes
Accepted

Should dimensionality reduction for visualization be considered a "closed" problem, solved by t-SNE?

I would still love to hear other comments but I'll post my own answer for now, as I see it. While I was looking for a more "practical" answer, there are two theoretical "dis-advantages" to t-sne which ...
galoosh33's user avatar
  • 2,302
8 votes

Why is t-SNE not used as a dimensionality reduction technique for clustering or classification?

t-SNE does not preserve distances, but it basically estimates probability distributions. In theory, the t-SNE algorithms maps the input to a map space of 2 or 3 dimensions. The input space is assumed ...
prashanth's user avatar
  • 4,127

Only top scored, non community-wiki answers of a minimum length are eligible