In the original Maaten and Hinton paper, they explicitly say that the class membership is not used by the t-SNE calculations, only for picking colors in the plot.

For all of the data sets, there is information about the class of each datapoint, but the class information is only used to select a color and/or symbol for the map points. The class information is not used to determine the spatial coordinates of the map points.

The StatQuest video on t-SNE, however, makes it seem like the internal workings of the algorithm aims to keep the blue points (red points, orange points) in two dimensions near the blue points (red points, orange points) in one dimension, and I was using that to help me understand what the t-SNE algorithm does.

enter image description here

Is that because the blue points (red points, orange points) are all near one another in the higher dimension? If he used twelve different colors or four colors per cluster, then those clusters would still have to be clustered in the low dimension.

(That's got to be it, right? It sure would be nice to get confirmation after a major mind-blown moment, though.)


Maaten, Laurens van der, and Geoffrey Hinton. "Visualizing data using t-SNE." Journal of machine learning research 9.Nov (2008): 2579-2605.


t-SNE pairwise selects all the distances in the high-dimensional space and tries to preserve these in a lower-dimensional space. For a given point it calculates the distance from all other points in high-dimensional space, and gives each of these a probability (close points have a higher probability). It then does the same for all other points, and translates the closeness in the high-dimensional space onto closeness on a low-dimensional space. In the diagram, the red, orange and blue groups will have high probability within groups and lower between groups. I believe the “color” explanation given in the video is twofold:

  1. To show the points before and after t-SNE.
  2. To “point” to individual points more easily in the explanation. It is an iterative process to fix the points in the low-dimensional space so this may be the reason for needing to use the colors to point to individual points.

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.