I have $N$ samples, each described by a vector of $M$ features. I can compute the symmetric distance matrix $D_1(i, j)$ representing the pairwise distances between the samples, using a distance measure. Now let's say that I change the conditions under whch the features are extracted, producing new feature vectors and thus a different matrix $D_2(i, j)$. I am interested in understanding if the differences between samples remained more or less the same (i.e. samples that were similar before are still similar to each other).

  • How can I compare $D_1$ and $D_2$? Would the canonical correlation a suitable measure?
  • Is there a statistical test to assess if the difference between $D_1$ and $D_2$ is significant?

Since your data matrices are symmetric, canonical correlation analysis(CCA) is not the right approach I think. CCA would look for linear combinations of distances that maximize correlations between the two sets. I would drop the correlation option.

The Procrustes distance may be a better option, since it measures the difference in shape of multidimensional ensembles. You could consider a resampling technique (such as the bootstrap) to test for significance, since I am not aware of any theoretical null distributions for the Procrustes distance.

| cite | improve this answer | |
  • $\begingroup$ isn't the shape comparison in Procrustes analysis a simple Euclidean distance? $\endgroup$ – firion Sep 7 '17 at 8:02
  • $\begingroup$ Yes, but after optimal translation, rotation and scaling. If you want to visually inspect differences in distance, consider a dimension reduction as in PCoA. $\endgroup$ – Knarpie Sep 7 '17 at 8:16
  • $\begingroup$ Ok. Also, can you help me understanding your last lines? How can I setup a test using the bootstrap method? I am not very fmailiar with this topic $\endgroup$ – firion Sep 7 '17 at 10:26
  • $\begingroup$ Your resampling procedure would have to occur under the scenario where there is no relationship between the distances between the same two samples in the different datasets. So either resample distances within the column (or row) from one distance matrix and keep the other constant, or resample in both. Calculate Procrustes distances for every instance. Then look at how extraordinary your observed result is compared to the resampling results. $\endgroup$ – Knarpie Sep 7 '17 at 12:03

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.