3
$\begingroup$

I am using t-sne on 252 dimensional data to embed to lower-dimensions. I am curious to know if it is academically justifiable to embed it into medium dimensions such as 100 dimensions, or 80 dimensions, instead of 2 dimensions.

I have read many examples of t-sne that embeds high-dimensional data to 2 dimensions for plotting. However, I haven't seen much examples of embedding data from 252 dimensions to 100 dimensions.

Can we academically justify embedding to medium-dimensions(e.g. 100 dimensions) using t-sne?

$\endgroup$

2 Answers 2

1
$\begingroup$

Here's an idea I am currently pursuing. I have large biological data sets (in about 20,000 dimensions, one for each gene). The mental model is that the actual data lives on (or at least near) a manifold of dimension smaller than 20,000. There are some existing mathematical/statistical tools to try to compute the "local dimension" of that manifold near any data point. (See Ellis and McDermott, Computational Statistics & Data Analysis, Volume 17, Issue 3, March 1994, Pages 317-326.) These frequently give local estimates in the range of 20-30 or more dimensions. In that case, we know that shrinking things down to 2 dimensions will destroy some (possibly all) of the topological structure of the manifold.

By the Whitney Embedding Theorem, we can embed any N-dimensional real manifold into 2N-dimensional Euclidean space. So, there must exist a way to put those 20-30 dimensional data manifolds into 40-60 dimensional space.

So, there's your potential mathematical justification. In practice, finding such an embedding may be an open problem. I found your question because I was searching for an implementation of either t-SNE or UMAP that would reduce from large dimensions down to moderate dimensions. (The existing implementations, at least in R, seem to only believe in 2 or 3 for visualization purposes.)

In my case, I can easily reduce from 20,000 dimensions to M-dimensions, where M is the number of samples. (Just use principal components analysis, and that is the maximum number of components you can get.) It's not clear what happoens when you take fewer PCs, since explainng less variance may fail to be an embedding of the data manifold.

$\endgroup$
1
$\begingroup$

WHY?

A troubling point with t-SNE is that it does not give a functional relationship between the high-dimension space and the reduced-dimension space, limiting the usefulness of t-SNE. For instance, if you want to reduce the dimension of a predictive model, when you go to predict on new data, you do not have a way to do so. This is in contrast to, say, PCA (Principal Components Analysis), where you can apply the eigenvectors from some training data to extract features from new data (either for out-of-sample testing or on entirely new data being predicted in production).

Consequently, t-SNE is at its most useful when you just want to visualize high-dimensional data, hence the emphasis on reducing to the two dimensions where our vision operates. You can’t visualize data in $80$ or $100$ dimensions, and you can’t do much beyond visualize with the t-SNE results, so I wonder why one would bother to reduce to a dimension where visualization is not feasible.

An alternative to t-SNE that does allow you to apply the transformation to new data is UMAP: Universal Manifold Approximation and Projection.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.