8
$\begingroup$

I generated a t-SNE visualization of a dataset (about 10 standardized (mean = 0, sd = 1) numerical features) and I arrived at a following 2-dimensional plot. I don't have a good intuition why the points are aligning in sort of arc-shaped groups, see for example right bottom part of the enclosed picture (or its leftmost part).

I found a similar effect at the original paper http://www.cs.toronto.edu/~hinton/absps/tsne.pdf - see Figure 4.a. However, I didn't find a good explanation of this kind of a phenomenon.

Any intuitions on why this happens?

enter image description here

$\endgroup$
3
  • 1
    $\begingroup$ Obligatory cool approximate t-SNE animation: link $\endgroup$
    – Firebug
    Jul 13, 2016 at 17:39
  • 1
    $\begingroup$ Firebug's link is broken. Maybe check this tutorial out: distill.pub/2016/misread-tsne $\endgroup$ Nov 29, 2016 at 9:12
  • $\begingroup$ Kind of a random question, which could change with any given dataset. $\endgroup$
    – user32398
    Nov 24, 2017 at 3:53

1 Answer 1

2
$\begingroup$

The a similar phenomenon can be seen if you search for images of "spring graph layout", which show you many examples of such arcs, such as this one from wikipedia. Near the top on the right edge of the image, we see one such arc. Admittedly this isn't the best example. The top right corner of this image from this paper shows the effect a bit

Most of these graph visualizations are generated by simulating a spring force between each pair of connected nodes, and allowing the nodes to move according to this force.

In t-SNE, a similar interpretation of the algorithm is possible -- points in 2D space have a spring, whose resting length depends on the distance of the points in the original high-dimensional space. So points that are closer in 2D space than they are in high-dimensional space are pushed farther apart, and points that are farther in 2D space than in high-dimensional space are pulled together.

So it's likely that the arcs form because they are trying to maintain a constant distance to another group of points in the data.

Unlike in the spring graph layout above, every pair of points in t-SNE has a spring/force attached to it, so it's a valid question to ask why the arcs don't clump together into blobs, as shown in the graph visualization from wikipedia, where some groups of nodes on the edges have formed roundish clusters rather than arcs.

I suspect the reason for this is that each point in t-SNE has a variance attached to it. Points in a sparser region of the high-dimensional space have a higher variance compared to points in a lower dimensional space. The force on the springs of high-variance points is reduced, so if points in an arc were located in a sparse region of the original space, there would only be a weak force trying to pull them into a cluster, which might not overcome other opposing forces.

Furthermore, the method that the authors used to reduce crowding was to use a heavy-tailed distribution in the 2D space, which means points aren't penalized too heavily for being farther away from each other than they should. This also reduces the forces which would try to pull an arc into a cluster.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.