Is it sensible to do PCA on a distance matrix?

Question

I have 10x10 distance matrix where the distance metrics is (1 - overlap coefficient).

I want to represent the observations in this matrix in a low dimensional space to see how observations relate to each other.

My understanding is that one of the go-to methods for this is multidimensional scaling (would you agree with this?). I have done it and the results make sense - sort of.

I have also tried PCA on the distance matrix and plotted the PC1 scores vs PC2 scores. The results seem better then MDS' in terms of agreement with our background knowledge of the data.

Now, I cannot quite wrap my head around whether:

• Is PCA on a distance matrix a sensible thing to do? I mean, PCA doesn't care where the data comes from but it feels odd to use it in this case. In particular:

1. PCA computes the eigenvectors of the covariance matrix of the input. Since my input is a distance matrix, does it make sense to do that?

2. My distances are bound between 0 and 1 - is this a concern?

• Should I be worried that PCA gives "better" results than MDS? Is it a hint that something may be off?

---

In case anyone wants to have a look, here's the matrix, also as an R object that can be copied & pasted into R

         [,1]   [,2]   [,3]    [,4]    [,5]    [,6]    [,7]  [,8]   [,9] [,10]
 [1,] 0.00000 0.1135 0.6255 0.45565 0.46215 0.34426 0.00847 0.937 0.7055 0.629
 [2,] 0.11348 0.0000 0.3715 0.25532 0.28646 0.21691 0.01176 0.941 0.5857 0.587
 [3,] 0.62550 0.3715 0.0000 0.04264 0.02731 0.07734 0.16880 0.972 0.1226 0.492
 [4,] 0.45565 0.2553 0.0426 0.00000 0.00909 0.13687 0.13333 0.955 0.1350 0.522
 [5,] 0.46215 0.2865 0.0273 0.00909 0.00000 0.09141 0.14948 0.971 0.0535 0.455
 [6,] 0.34426 0.2169 0.0773 0.13687 0.09141 0.00000 0.00763 0.977 0.1164 0.189
 [7,] 0.00847 0.0118 0.1688 0.13333 0.14948 0.00763 0.00000 0.951 0.2835 0.175
 [8,] 0.93694 0.9412 0.9719 0.95455 0.97093 0.97683 0.95122 0.000 0.9762 0.975
 [9,] 0.70547 0.5857 0.1226 0.13498 0.05354 0.11639 0.28350 0.976 0.0000 0.345
[10,] 0.62887 0.5866 0.4924 0.52228 0.45503 0.18887 0.17478 0.975 0.3452 0.000

---

structure(c(0, 0.113475177304965, 0.625498007968127, 0.455645161290323, 
0.462151394422311, 0.344262295081967, 0.00847457627118597, 0.936936936936937, 
0.705467372134039, 0.628865979381443, 0.113475177304965, 0, 0.371490280777538, 
0.25531914893617, 0.286458333333333, 0.216911764705882, 0.011764705882353, 
0.941176470588235, 0.585741811175337, 0.586638830897704, 0.625498007968127, 
0.371490280777538, 0, 0.042643923240938, 0.027310924369748, 0.077338129496403, 
0.168803418803419, 0.971861471861472, 0.122599704579025, 0.49238578680203, 
0.455645161290323, 0.25531914893617, 0.042643923240938, 0, 0.00909090909090904, 
0.136871508379888, 0.133333333333333, 0.954545454545454, 0.134980988593156, 
0.522281639928699, 0.462151394422311, 0.286458333333333, 0.027310924369748, 
0.00909090909090904, 0, 0.091412742382271, 0.149484536082474, 
0.970930232558139, 0.053537284894837, 0.455026455026455, 0.344262295081967, 
0.216911764705882, 0.077338129496403, 0.136871508379888, 0.091412742382271, 
0, 0.00763358778626, 0.976833976833977, 0.116385911179173, 0.188870151770658, 
0.00847457627118597, 0.011764705882353, 0.168803418803419, 0.133333333333333, 
0.149484536082474, 0.00763358778626, 0, 0.951219512195122, 0.283495145631068, 
0.174778761061947, 0.936936936936937, 0.941176470588235, 0.971861471861472, 
0.954545454545454, 0.970930232558139, 0.976833976833977, 0.951219512195122, 
0, 0.976237623762376, 0.975225225225225, 0.705467372134039, 0.585741811175337, 
0.122599704579025, 0.134980988593156, 0.053537284894837, 0.116385911179173, 
0.283495145631068, 0.976237623762376, 0, 0.345195729537367, 0.628865979381443, 
0.586638830897704, 0.49238578680203, 0.522281639928699, 0.455026455026455, 
0.188870151770658, 0.174778761061947, 0.975225225225225, 0.345195729537367, 
0), .Dim = c(10L, 10L), .Dimnames = list(NULL, NULL))

Answer 1

Not a perfect answer, but will push you in the right direction. Are you familiar with the PageRank algorithm? It leverages Eigenvector centrality, which is closely related to application of PCA to network analysis. In PageRank, we start with a set of webpages, S and the transition probability matrix, A. Now, if a bot were to start on a given page then repeatedly move according to probabilities in A, we would slowly converge on expected proportional traffic on each webpage. In fact, this proportional vector is, V, the eigenvector of A. What this vector tells us is, if we randomly stopped the bot and asked, "what webpage are you on?" It would give us the webpage probability distribution. And this is the concept of Eigenvector centrality, it defines which points are most/least likely to be visited. (Note, the specific eigenvector returned has an eigenvalue of 1.) ​​

So, to your question- PCA finds the eigenvectors of a covariance matrix. If you limit yourself to just the first component, then it would function similar to PageRank (encoding entities as scalars.) If you include multiple components, you encode entities as vectors/embeddings. Anyway, a covariance matrix is simply one of many possible encodings of vector similarity. You are using 1- overlap_coefficient, so your matrix encodes dissimilarity of vectors. If you were using PCA on overlap_coefficient, then the results would compress/encode the centrality of entities. However, since you are using 1-overlap_coefficient, then PCA would compress/encode the isolation of entities in your set.

So if you limit yourself to the first principal component, then you have a raw ranking of the isolation of elements in your set. However, if you add multiple components, then it's less interpretable, however, the compression is of higher fidelity.

Edit: It's beyond this Q/A, but eigenvector decomposition behaves in interesting ways when the matrix elements are bounded by [0,1]. This idea and some nuanced requirements are foundational to PCA and Eigenvector centrality.