I am interested in tSNE for data analysis but am very fuzzy on how the embedding process is actually developed and the clusters formed. I know there is a cartesian distance metric and some probabilistic magic going on that I do not understand. I'm seeking an intuitive understanding of what these two key aspects of the algorithm are actually accomplishing.
A good way to get at this might be to compare and contrast the clusters you might get out of a tSNE projection with the results of a trained neural net (NN) model that also classifies the same data. A trained model can classify MNIST images with low error rates. Similarly tSNE can cluster those images in which can be seen as defining unique classes they belong. This is very exciting but it begs the question how are the processes different and how are they the same? What is it that drives this classification / clustering.
My understanding is that a NN trains on labeled data and determines weights and thresholds for how much each dimension contributes to a given labeling/classification. AKA each dimension certainly does not contribute equally to the classification of any given data point. It seems that tSNE accomplishes something similar (clustering not far from classification) in an unsupervised way (ignoring parameter tuning) but its not clear weather or not each dimension contributes equally in tSNE or if it effectively sets up something like a weighting/thresholding of dimensions (not unlike a trained NN classifier) as it evolves the projection over many iterations.
Another real world example would be customer purchase data. Customer data can have hundreds of dimensions, traditionally recency, frequency and monetary (RFM) dimensions are monitored closely as they are most indicative of customer behavior and you find that in trained models these are indeed always heavily influential.
If we then take the same data and use tSNE to understand clusters within the data I would expect to see differences in RFM dominating those clusters. Is this assumption accurate?