7
$\begingroup$

While reading Laurens van der Maaten's paper about t-SNE we can encounter the following statement about perplexity:

The perplexity can be interpreted as a smooth measure of the effective number of neighbors. The performance of SNE is fairly robust to changes in the perplexity, and typical values are between 5 and 50.

What this effective number of neighbors would mean? Should I understand perplexity value as expected number of nearest neighbors to the point $x_i$? But it would mean that having the constant number of groups in our datasets, with increasing number of observations we should increase perplexity value as well, which seems totally counterintuitive and inconsistent with proposed range of values between 5 and 50.

On the other hand, as pointed out here, with values of perplexity approaching the number of points in the dataset we probably end up with a visualisation with no clusters at all.

Therefore, my question is: does perplexity can be associated with expected number of points in a cluster or I just misunderstood the quoted excerpt? Generally, what is an intuition behind perplexity - can it be approximetely anticipated or the only way to set it is to try out different values and visually assess resulting maps?

Improve this question
$\endgroup$

2 Answers 2

Reset to default
2
$\begingroup$

if you write down the equation for perplexity defined by the conditional distribution in the original paper. it does not increase wrt to the entropy simply because the conditional distribution is discrete and is not gaussian. It is not a rigorous term and in the original paper they didn't even talk about it in detail... it is completely a different thing from gaussian entropy....I think that's why tsne gives very confusing results sometimes....

Improve this answer
$\endgroup$
1
  • $\begingroup$ I agree, and never recommend t-SNE as a first unsupervised clustering or distance metric learning method. ANN auto-encoders with e.g. 2 nodes (i.e., 2D) in the hidden layer or NMF often gives better results. $\endgroup$
    – user32398
    Mar 1, 2020 at 2:26
0
$\begingroup$

Yeah, i can't agree with you more, in my view, perplexity doesn't increase monotonically with σ at all. Considering two extrame scenario, when σ is extremely close to zero, and when σ is extremely large, in these two cases, the perplexities are equal.

Improve this answer
$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.