Principal Component Analysis among matrices

Question

I have the following setting. I have n Hermitian Positive Semidefinite (HPSD) matrices, and a metric induced by a matrix norm. I am primarily interested in the Frobenius norm and the operator norm. I want to extract the principal "principal component" for this set of observations, i.e., a 1-dimensional subspace in HPSD such that that the sum of squares of the minimum distances between the matrices with this subspace is minimized. I am not interested in second, third etc components, since they are not even well defined in this space, since I have not defined a scalar product for simplicity.

In the case of the Frobenius norm, the problem can be reduced to traditional PCA, by using as input vectors the stacked versions of the input matrices. But in the case of the operator norms, I can't find a strategy to attack the problem.

Questions:

1. Has anyone seen this specific problem before? Recommendations and references are highly appreciated.
2. Has anyone dealt with computation of PCA in the case of non-euclidean distances?

Answer 1

I don't know if this is exactly what you are looking for (esp. I don't know how large is $n$ and what you intend to do with these results), however I have successfully used coinertia analysis when I was working with two data sets (same observations in rows), and for more than two data sets there are K-table methods, as implemented in the ade4 R package. An introduction to K-table analyses outlines the main principles. When the objective is to link two or more Tables, Generalized Canonical Correlation Analysis is also an option.

It seems to me that you can choose non-euclidean metric, provided it has some meaning for the data at hand and the interpretation of the factorial space. You can see an example with the use of kdist() in ade4 for applying an PCA on different distance matrices. Jollife's book on Principal component analysis should provide additional hints about this (but I didn't check). There's also all the work made in the spirit of Gifi on non-linear methods (in R, a lot of packages have been developed by Jan de Leeuw, see the PsychoR project).