The publication introducing Isomap compares PCA and Isomap by means of

$$\text{residual variance} = 1 - R^2(\hat D_M, D_y)$$

where $R$ is the standard linear correlation coefficient over all entries of $\hat D_M$ and $D_Y$. $\hat D_M$ is the euclidean distance matrix for PCA and the geodesic distance matrix for Isomap. $D_Y$ is the euclidean distance matrix of the low dimensional embedding, this matrix changes with the number of dimensions you use for the embedding.

Doing some testing reveals that "residual variance" is numerically different from $$ 1 - \text{explained variance}$$ derived from the eigenvalue spectrum.

Is there a link between "residual variance" and "explained variance" for PCA and possibly Isomap?

  • $\begingroup$ That's presumably correlation coefficient across all entries of $D$ matrices, i.e. these $n\times n$ matrices are "vectorized" as $n^2$ datapoints, right? $\endgroup$
    – amoeba
    May 25 '18 at 9:22
  • $\begingroup$ PCA is not even trying to match $D_Y$ with $D_M$, that's the task of multi-dimensional scaling. There is no relation between PCA explained variance and this correlation coefficient. $\endgroup$
    – amoeba
    May 25 '18 at 9:23
  • $\begingroup$ Regarding your first comment: I think so, this is how I read the footnote in the original paper. $\endgroup$
    – gdkrmr
    May 25 '18 at 13:00
  • $\begingroup$ Regarding your second comment: MDS gives the same result as PCA when used with Euclidean distances, so there should be some connection. $\endgroup$
    – gdkrmr
    May 25 '18 at 13:06
  • $\begingroup$ It's so called "Torgerson" (aka "classical") MDS that is equivalent to PCA, but it does not match distances. The version of MDS that actually tries to match distances is not equivalent to PCA. $\endgroup$
    – amoeba
    May 25 '18 at 13:09

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.