7
$\begingroup$

I am using R princomp function (from stats package) to run a PCA on a data set and I want to compare its output to that of the nonlinear dimensionality reduction method ISOMAP, which I am using under matlab through this toolbox: http://isomap.stanford.edu on the same dataset.

What I am interested in is the intrinsic dimensionality of the dataset as determined according to PCA and to ISOMAP separately, the ultimate goal being to check whether nonlinear dimensionality reduction works better on this dataset than PCA.

With princomp I get the standard deviations associated to each component, while the ISOMAP package returns residual variances as a function of the manifold dimensionality. How do I compare these two quantities? In other words, how is the residual variance defined in ISOMAP?

This should be irrelevant, but the dataset is 54 points in 5 dimensions.

$\endgroup$
4
  • $\begingroup$ Add up squared standard deviations of PCA components, subtract from the total variance of the dataset, and you get residual variance that can be directly compared to ISOMAP values. $\endgroup$
    – amoeba
    Dec 29 '14 at 15:32
  • $\begingroup$ @amoeba I know this is an old question, but you should post that as an answer. $\endgroup$ Apr 9 '15 at 2:08
  • $\begingroup$ @ssdecontrol: My comment only answers one part of the question. The other part is about how the residual variance is defined in ISOMAP (and why). Seeing your interest, I might write an answer, but it will require some research. I think I know approximately how it is supposed to work, but I need to check the mathematical details. $\endgroup$
    – amoeba
    Apr 9 '15 at 13:25
  • $\begingroup$ @amoeba I just stumbled on this question by following related links. OP seems long gone and you probably have other things to do, and more important questions to spend your time on. $\endgroup$ Apr 9 '15 at 15:03
3
$\begingroup$

The original Isomap paper defined "residual variance" as follows (reference 42):

$$\text{residual variance} = 1 - R^2(\hat D_M, D_y)$$

where $R$ is the Pearson correlation coefficient over all entries of $\hat D_M$ and $D_Y$. $\hat D_M$ is the euclidean distance matrix for PCA and the geodesic distance matrix for Isomap. $D_Y$ is the euclidean distance matrix of the low dimensional embedding, this matrix changes with the number of dimensions you use for the embedding.

EDIT: This is numerically different than the explained variance of PCA derived from the eigenvalues and I don't know if there is a direct connection between the two of them.

EDIT: I asked here for the connection between residual and explained variance.

$\endgroup$
2
  • $\begingroup$ Can you explain how R is calculated? I don't know the standard linear correlation coefficient. $\endgroup$ Jun 30 '20 at 15:45
  • 1
    $\begingroup$ is it better now? $\endgroup$
    – gdkrmr
    Jul 22 '20 at 9:28

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.