1
$\begingroup$

Classic MDS (cMDS or PCoA) preserves global distances, characteristic of linear techniques. However, metric MDS seeks to minimize a cost function (stress), while non-metric MDS (nMDS) preserves only the ranking of dissimilarities between points. It seems to me these techniques produce a kind of embedding, which would be nonlinear, but both cMDS and nMDS are listed as linear techniques in this article. Conversely, Wikipedia describes MDS in general as a form of nonlinear dimensionality reduction.

It is possible to use a nonlinear kernel in MDS to preserve smaller distances, as in the case of a Sammon mapping. This is definitely a nonlinear technique.

So: are multidimensional scaling and its variants considered linear or nonlinear dimensionality reduction techniques, and why?

$\endgroup$
  • $\begingroup$ "but both cMDS and nMDS are listed as linear techniques in this article" -- where exactly? $\endgroup$ – amoeba says Reinstate Monica Jul 30 at 11:05
  • $\begingroup$ It clearly lists nMDS as nonlinear in "Tip 1". $\endgroup$ – amoeba says Reinstate Monica Jul 30 at 11:06
  • $\begingroup$ The article itself seems to be contradictory. NMDS is not classified as nonlinear in table 1 in the same paper. $\endgroup$ – sara-es Jul 31 at 13:22
  • $\begingroup$ Oh I see. So it's just an unfortunate typo in the Table 1. $\endgroup$ – amoeba says Reinstate Monica Jul 31 at 13:33
  • $\begingroup$ I might write-up a proper answer later, but I wouldn't call t-SNE/nMDS/etc "nonlinear", I think it's sloppy. They should rather be called "nonparametric". See here stats.stackexchange.com/questions/142960 about this distinction. cMDS is called "linear" because it's equivalent to PCA but it's actually an abuse of terminology. $\endgroup$ – amoeba says Reinstate Monica Jul 31 at 13:39

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.