1
$\begingroup$

I have 10x10 distance matrix where the distance metrics is (1 - overlap coefficient).

I want to represent the observations in this matrix in a low dimensional space to see how observations relate to each other.

My understanding is that one of the go-to methods for this is multidimensional scaling (would you agree with this?). I have done it and the results make sense - sort of.

I have also tried PCA on the distance matrix and plotted the PC1 scores vs PC2 scores. The results seem better then MDS' in terms of agreement with our background knowledge of the data.

Now, I cannot quite wrap my head around whether:

  • Is PCA on a distance matrix a sensible thing to do? I mean, PCA doesn't care where the data comes from but it feels odd to use it in this case. In particular:
  1. PCA computes the eigenvectors of the covariance matrix of the input. Since my input is a distance matrix, does it make sense to do that?

  2. My distances are bound between 0 and 1 - is this a concern?

  • Should I be worried that PCA gives "better" results than MDS? Is it a hint that something may be off?

In case anyone wants to have a look, here's the matrix, also as an R object that can be copied & pasted into R

         [,1]   [,2]   [,3]    [,4]    [,5]    [,6]    [,7]  [,8]   [,9] [,10]
 [1,] 0.00000 0.1135 0.6255 0.45565 0.46215 0.34426 0.00847 0.937 0.7055 0.629
 [2,] 0.11348 0.0000 0.3715 0.25532 0.28646 0.21691 0.01176 0.941 0.5857 0.587
 [3,] 0.62550 0.3715 0.0000 0.04264 0.02731 0.07734 0.16880 0.972 0.1226 0.492
 [4,] 0.45565 0.2553 0.0426 0.00000 0.00909 0.13687 0.13333 0.955 0.1350 0.522
 [5,] 0.46215 0.2865 0.0273 0.00909 0.00000 0.09141 0.14948 0.971 0.0535 0.455
 [6,] 0.34426 0.2169 0.0773 0.13687 0.09141 0.00000 0.00763 0.977 0.1164 0.189
 [7,] 0.00847 0.0118 0.1688 0.13333 0.14948 0.00763 0.00000 0.951 0.2835 0.175
 [8,] 0.93694 0.9412 0.9719 0.95455 0.97093 0.97683 0.95122 0.000 0.9762 0.975
 [9,] 0.70547 0.5857 0.1226 0.13498 0.05354 0.11639 0.28350 0.976 0.0000 0.345
[10,] 0.62887 0.5866 0.4924 0.52228 0.45503 0.18887 0.17478 0.975 0.3452 0.000

structure(c(0, 0.113475177304965, 0.625498007968127, 0.455645161290323, 
0.462151394422311, 0.344262295081967, 0.00847457627118597, 0.936936936936937, 
0.705467372134039, 0.628865979381443, 0.113475177304965, 0, 0.371490280777538, 
0.25531914893617, 0.286458333333333, 0.216911764705882, 0.011764705882353, 
0.941176470588235, 0.585741811175337, 0.586638830897704, 0.625498007968127, 
0.371490280777538, 0, 0.042643923240938, 0.027310924369748, 0.077338129496403, 
0.168803418803419, 0.971861471861472, 0.122599704579025, 0.49238578680203, 
0.455645161290323, 0.25531914893617, 0.042643923240938, 0, 0.00909090909090904, 
0.136871508379888, 0.133333333333333, 0.954545454545454, 0.134980988593156, 
0.522281639928699, 0.462151394422311, 0.286458333333333, 0.027310924369748, 
0.00909090909090904, 0, 0.091412742382271, 0.149484536082474, 
0.970930232558139, 0.053537284894837, 0.455026455026455, 0.344262295081967, 
0.216911764705882, 0.077338129496403, 0.136871508379888, 0.091412742382271, 
0, 0.00763358778626, 0.976833976833977, 0.116385911179173, 0.188870151770658, 
0.00847457627118597, 0.011764705882353, 0.168803418803419, 0.133333333333333, 
0.149484536082474, 0.00763358778626, 0, 0.951219512195122, 0.283495145631068, 
0.174778761061947, 0.936936936936937, 0.941176470588235, 0.971861471861472, 
0.954545454545454, 0.970930232558139, 0.976833976833977, 0.951219512195122, 
0, 0.976237623762376, 0.975225225225225, 0.705467372134039, 0.585741811175337, 
0.122599704579025, 0.134980988593156, 0.053537284894837, 0.116385911179173, 
0.283495145631068, 0.976237623762376, 0, 0.345195729537367, 0.628865979381443, 
0.586638830897704, 0.49238578680203, 0.522281639928699, 0.455026455026455, 
0.188870151770658, 0.174778761061947, 0.975225225225225, 0.345195729537367, 
0), .Dim = c(10L, 10L), .Dimnames = list(NULL, NULL))
$\endgroup$

1 Answer 1

2
$\begingroup$

Not a perfect answer, but will push you in the right direction. Are you familiar with the PageRank algorithm? It leverages Eigenvector centrality, which is closely related to application of PCA to network analysis. In PageRank, we start with a set of webpages, S and the transition probability matrix, A. Now, if a bot were to start on a given page then repeatedly move according to probabilities in A, we would slowly converge on expected proportional traffic on each webpage. In fact, this proportional vector is, V, the eigenvector of A. What this vector tells us is, if we randomly stopped the bot and asked, "what webpage are you on?" It would give us the webpage probability distribution. And this is the concept of Eigenvector centrality, it defines which points are most/least likely to be visited. (Note, the specific eigenvector returned has an eigenvalue of 1.) ​​

So, to your question- PCA finds the eigenvectors of a covariance matrix. If you limit yourself to just the first component, then it would function similar to PageRank (encoding entities as scalars.) If you include multiple components, you encode entities as vectors/embeddings. Anyway, a covariance matrix is simply one of many possible encodings of vector similarity. You are using 1- overlap_coefficient, so your matrix encodes dissimilarity of vectors. If you were using PCA on overlap_coefficient, then the results would compress/encode the centrality of entities. However, since you are using 1-overlap_coefficient, then PCA would compress/encode the isolation of entities in your set.

So if you limit yourself to the first principal component, then you have a raw ranking of the isolation of elements in your set. However, if you add multiple components, then it's less interpretable, however, the compression is of higher fidelity.

Edit: It's beyond this Q/A, but eigenvector decomposition behaves in interesting ways when the matrix elements are bounded by [0,1]. This idea and some nuanced requirements are foundational to PCA and Eigenvector centrality.

$\endgroup$
3
  • $\begingroup$ Thanks - I'll have to digest your answer. My aim is to plot the 9 observations on a 2D plot and I'm not sure if your answer goes in that direction. Nevertheless, you make explicit some concerns I have - I'm going to edit my question to include them. $\endgroup$
    – dariober
    Mar 23, 2021 at 17:39
  • $\begingroup$ Semi-related, I've used PCA on word_embeddings before (~300 dimensions) then plotted the first two principal components (in 2D, of course.) This helps visualize the similarity of words. Since you're using the conjugate of a similarity function, things will function...differently. Some entities may be mapped close to one another, however, you may be stuck wondering "are they similar intrinsically or are they similar in that they're both outliers?" Definitely interested in what your plot looks like! $\endgroup$
    – jbuddy_13
    Mar 23, 2021 at 17:49
  • $\begingroup$ I put the data in my question - if you want to have a go please do! As I said, there is nothing weird with the data or from the PCA results. I'm only worried that I'm getting good-looking results by accident from an otherwise meaningless procedure. $\endgroup$
    – dariober
    Mar 23, 2021 at 18:18

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.

Not the answer you're looking for? Browse other questions tagged or ask your own question.