2
$\begingroup$

The stochastic part of t-SNE is presumably from the randomness of the initial placement of the points in the low-dimensional space.

  1. Does this mean that t-SNE should be re-run on re-seeded randomness to ascertain whether the final embedding of high-dimensional data onto the low-dimensional space is or is not stable? That is at least a rudimentary sensitivity analysis of comparing the results of different runs should be performed?

  2. The minimization of the Kullback–Leibler divergence with respect to the output points is performed using gradient descent. Is stochastic gradient descent usually used here, and if so does this introduce randomness of the output? Would this be of a lesser amount to that introduced by 1., or would the two not be comparable?

Improve this question
$\endgroup$
3
  • $\begingroup$ +1. (1a) If one uses random init, then yes it makes sense to run it a few times with different seeds. (1b) However, using random init is a bad idea, it makes much more sense to use informative non-random init e.g. PCA. See e.g. nature.com/articles/s41467-019-13056-x. It's actually default in modern implementations, e.g. openTSNE. (2) There is no stochasticity in gradient descent, at least in standard implementations. $\endgroup$
    – amoeba
    Commented Nov 4, 2020 at 16:45
  • $\begingroup$ @amoeba the article is readable, will attempt to understand it. Feel free to expand as an answer. For 2. to clarify, for finding a minimum, stochastic gradient descent is not usually used in t-SNE? $\endgroup$
    – Single Malt
    Commented Nov 4, 2020 at 18:44
  • $\begingroup$ That's correct. $\endgroup$
    – amoeba
    Commented Nov 5, 2020 at 9:58

0

Reset to default

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Browse other questions tagged or ask your own question.