6
$\begingroup$

I am having trouble reconciling between several terms in MDS. According to [1], Section 14.8, Classical MDS takes similarities as inputs. In [2], also cited in Wikipedia, Classical MDS takes dissimilarities as inputs.

What is the agreed upon terminology?

[1] Hastie, T, R Tibshirani, and JH Friedman. The Elements of Statistical Learning. Springer, 2003.

[2] Borg, I., and P. J. F. Groenen. Modern Multidimensional Scaling: Theory and Applications. 2nd edition. New York: Springer, 2005.

Improve this question
$\endgroup$
2
  • 2
    $\begingroup$ I suppose you may be mistaken. Term "Classic MDS" or "simplest MDS" is usually understood as unweighted euclidean MDS, that is, not INDSCAL model. Classic MDS works with one matrix. It can be similarities or dissimilarities. Usually a program first converts the first into the second and proceeds. Because the algorithm itself is typically written for dissimilarities. $\endgroup$
    – ttnphns
    Commented Apr 24, 2015 at 14:30
  • $\begingroup$ Plugging similarities or dissimilarities in the squared stress function leads to different solutions (see discussion in [1] between Eq(14.100) and Eq.(14.101)). It is thus of importance to define which goes into the stress function. $\endgroup$
    – JohnRos
    Commented Apr 24, 2015 at 14:33

1 Answer 1

Reset to default
5
$\begingroup$

These two books are in full agreement.

Classical multidimensional scaling (where by "classical MDS" I understand Torgerson's MDS, following both Hastie et al. and Borg & Groenen) finds points $z_i$ such that their scalar products $\langle z_i, z_j \rangle$ approximate a given similarity matrix as well as possible. However, any dissimilarity matrix can be converted into a similarity matrix: dissimilarities are assumed to be Euclidean distances, from which centered scalar products can be computed and taken as similarities.

So the algorithm of classical/Torgerson MDS is as follows: $$\text{Euclidean distances}\to\text{Centered scalar products}\to\text{Optimal mapping},$$ i.e. $$\text{Dissimilarities}\to\text{Similarities}\to\text{Optimal mapping}.$$ What you consider an "input" here, does not really matter.

This is exactly what is written in Hastie et al.:

In classical scaling, we instead [as opposed to metric scaling in general] start with similarities [...]. This is attractive because there is an explicit solution in terms of eigenvectors [...]. If we have distances rather than inner-products, we can convert them to centered inner-products if the distances are Euclidean [...]. If the similarities are in fact centered inner-products, classical scaling is exactly equivalent to principal components [...]. Classical scaling is not equivalent to least squares scaling [that minimizes reconstruction of dissimilarities].

See my answer in What's the difference between principal components analysis and multidimensional scaling? for mathematical details.

Improve this answer
$\endgroup$
12
  • $\begingroup$ Thank you. What you are saying is that Borg & Groenen definition is wrong: the stress is not a function of dissimilarities, but rather of similarities. Put differently, Borg & Groenen "classical MDS" is Hastie's Least Squares Scaling. Would you agree? $\endgroup$
    – JohnRos
    Commented Apr 24, 2015 at 21:53
  • $\begingroup$ No, definitely not. Borg & Groenen seem to be in full agreement with Hastie et al., and use the term "classical scaling" (Chapter 12) to refer to Torgerson scaling, which is what Hastie et al. also do. $\endgroup$
    – amoeba
    Commented Apr 24, 2015 at 22:04
  • $\begingroup$ @amoeba, I've just surfed over some texts on MDS and I found a confusion with the definition of term "Classical MDS". Some sources define it as you did, as =Torgerson's MDS aka PCoA. But more sources define it as "single matrix Euclidean model MDS" (I myself thought this way and continue to agree with that custom). If to admit to it then Classical MDS (aka Identity MDS) is what is not Replicated MDS or Weighted/Generalized MDS (INDSCAL's versions). $\endgroup$
    – ttnphns
    Commented Apr 24, 2015 at 23:04
  • $\begingroup$ (cont.) It - classical MDS - can be metric or nonmetric, be based on different stress measures, be algorithmically based on eigen or ALSCAL or PREFSCAL etc. Torgerson's PCoa is just one (and simplest) kind of it. Note that in my answer I implied that definition of classical mds. $\endgroup$
    – ttnphns
    Commented Apr 24, 2015 at 23:06
  • $\begingroup$ See also stats.stackexchange.com/a/31291/3277 where I also implied that definition of classical mds. $\endgroup$
    – ttnphns
    Commented Apr 24, 2015 at 23:12

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.