0
$\begingroup$

I'm only starting to understand some of the subtleties that arise when choosing one dimensionality reduction technique over another. Now I'm wondering why it seems that in recent years, one of the more popular methods for nonlinear dimensionality reduction was/is t-SNE, while multilinear Principal Component Analysis seems to be rarely used -- at least in the literature I look at, machine learning, and usually for the purpose of visualization of high-dimensional data.

Could someone comment on what might be the reason for this (unless of course my assumption is wrong to begin with)? In particular, it seems that (linear) PCA has a number of advantages over t-SNE (e.g. learning a function that can be applied to new data, 'preservation' of global structure), so I am wondering whether these advantages are also present in multilinear PCA. If so, I wonder why MPCA doesn't seem to be used that commonly.

Related to the question above: I recently read about Isomap, which by my (very incomplete understanding) looks like it should be another useful nonlinear method for interpreting high-dimensional data. In that case, what would be the distinguishing features between t-SNE, MPCA and Isomap?

Improve this question
$\endgroup$
2
  • 4
    $\begingroup$ Do you mean this en.wikipedia.org/wiki/Multilinear_principal_component_analysis ? What relationship does this have to t-SNE? MPCA is a linear method of dimensionality reduction of "multi-way" tensors. T-SNE is a nonlinear method for dimensionality reduction of usual matrices. They have completely different scope. $\endgroup$
    – amoeba
    Commented Jan 2, 2017 at 18:15
  • 1
    $\begingroup$ @amoeba I see, thank you very much for clearing this up. The patience of contributors on this site with imbeciles like me and the questions they ask is admirable. $\endgroup$
    – Bert Zangle
    Commented Jan 2, 2017 at 18:33

1 Answer 1

Reset to default
1
$\begingroup$

As you can see in this paper: http://jmlr.org/papers/volume9/vandermaaten08a/vandermaaten08a.pdf Isomap is not as good in separation (in that specific type of cases) as t-SNE.

And here is MPCA against t-SNE: http://journals.plos.org/plosone/article/file?id=10.1371/journal.pone.0056883&type=printable

t-SNE seems to produce the best results for visualisation in 2D. Visualisation is often the ultimate purpose of any data-analysis, so t-SNE is the best method of these three (for these particular cases and applications). It is though to be understood that t-SNE may fail in other cases, but in those cases and the purpose of 2D visualisation for clustering it performs well.

Improve this answer
$\endgroup$
2
  • $\begingroup$ Thanks. I skimmed the van der Maaten/Hinton one before but didn't notice they cover Isomap. One question though: I want to visualize/interpret the global structure as much as that is possible. So t-SNE performing better at clustering the different classes (and avoiding the crowding problem) is definitely useful, but I shouldn't draw too many conclusions from the organization of the clusters in relation to each other, right? Put differently: if the goal is less aiming for (optimal) local clustering, and more about interpretable differences on a larger scale, is t-SNE still the right choice? $\endgroup$
    – Bert Zangle
    Commented Jan 2, 2017 at 13:26
  • $\begingroup$ Not necessarily. You should always play with the data. You take a sample of data and apply some methods. Each of the methods have their own advantage, often by design. t-SNE is the right choice only if it works, if not choose an another method. This is why there is so many methods in use. PCA-type of methods have those principal components that can make sense as themselves. It is not only Euclidean distance that can be used. While people live in Euclidean space it can be nonsensical; especially the distance of binary-vectors as Euclidean rarely makes sense. $\endgroup$
    – user3644640
    Commented Jan 2, 2017 at 14:23

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.