Does it make sense to use a matrix that is made up of jaccard indexes instead of a covariance matrix and perform principal component analysis on that?
Add a comment
|
3 Answers
$\begingroup$
$\endgroup$
0
the Jaccard index is a positive definite kernel as can be checked in A Short Tour of Kernel Methods for Graphs, by Gärtner, Le, and Smola; see definition 1.4 and references below.
Doing a PCA on a matrix of Jaccard similarities is akin to doing kernel PCA, that is doing PCA in the reproducing kernel Hilbert space of functions (on sets) induced by the Jaccard similarity (or better said, kernel). There's a relatively good understanding of such a method for data analysis.
$\begingroup$
$\endgroup$
Linear Principal Component or Factor analyses are based on linear regression model and this implies that the input similarities must be covariances, correlations, cosines, or sum-of-cross-products (all these similarities are known as scalar products). You may input any other sort of similarity, such as Jaccard measure or Kendall correlation, but only keeping in mind that the analysis will "think" it is scalar product, i.e. usual Pearson correlation or cosine, in this case.
When applied to true Pearson correlations or other type of scalar product, PCA reduces dimensionality with minimal distortion of data cloud's shape in terms of sum of squared euclidean distances between the data points. With Jaccard measure or such, you can't say that PCA reduces dimensionality with the above mentioned objective function.
$\begingroup$
$\endgroup$
In PCA, we attempt to "concisely" explain the variation in POSSIBLY CORRELATED data using principal components which are pairwise orthogonal to each other. The variation in data is represented by the variance-covariance matrix.
On the other hand, a jaccard index is a similarity coefficient. Similarity and correlation are pretty different concepts. According to your description, if we take a matrix of Jaccard indices, the eigenvectors will be orthogonal to each other-that is fine: but will we be in a position to say what fraction of variation is explained by a given $"Jaccard PC"$, so to speak? In case of regular PC, we can surely say what fraction of the variation in data is represented by a given PCA, which is nothing but the ratio of corresponding eigenvalue and sum of all eigenvalues.
To summarize, the fundamentally different concepts in jaccard index makes it not sensible for use in PCA.
