1
$\begingroup$

As we know, we can get the same eigenvector if we apply PCA to the same data. But, is it possible that we get the same eigenvectors after we apply PCA to two totally different data sets (still same dimension)?

$\endgroup$
6
$\begingroup$

Any pure rotation of the dataset would also give the same set of principal components, in which case the answer would be trivially "yes", depending on the definition of "totally different" (arguably if two datasets have the same principal components then they can't be "totally" different).

$\endgroup$
  • $\begingroup$ Than you for comment. $\endgroup$ – Peter Nov 19 '10 at 13:17
  • $\begingroup$ Then, do two datasets having the same eigenvectors means they are same or similar data sets? $\endgroup$ – Peter Nov 19 '10 at 13:23
  • 4
    $\begingroup$ @Peter if the eigenvectors are identical and so are their corresponding eigenvalues, then you can recreate the covariance (or correlation) matrix you used for PCA. However, such a matrix obviously does not determine the data. For instance, subtracting a constant from all values won't change the covariances. $\endgroup$ – whuber Nov 19 '10 at 16:18
  • 2
    $\begingroup$ re: your last comment. Take any k datasets you like, each with n values, and combine them into one k-variate dataset of n records after first randomly permuting the order of each dataset. The random permutations assure the correlation matrix will be close to the identity. Do the same for another collection of k completely different datasets. The two PCAs will be essentially the same, but by construction there is no relationship between the two collections of data. What is common is the mutual independence of the coordinates in each k-variate combination, nothing more. $\endgroup$ – whuber Nov 19 '10 at 16:23
3
$\begingroup$

My answer to this question provides one example.

Two data sets, both containing two variables, with the same sign correlation (positive or negative or zero), will get the same PCs if you base your PCA on the correlation matrix (i.e. if you standardise your variables). The two PCs are always $\pm$45 degree rotations, with the first PC being the rotation with same sign as correlation, and second PC being the rotation with opposite sign. The size of the correlation is informative about the strength of the decomposition (via % variance explained), but totally uninformative about the direction.

Another example, in any dimension is if the correlations are all equal. The correlation matrix then has two distinct eigenvalues $\lambda_1=1+(d-1)\rho$ (multiplicity $1$) and $\lambda_2=(1-\rho)$ (multiplicity $d-1$), with normalised eigenvector $e_1= \frac{1}{\sqrt{d}}(1,\dots,1)^T$. The remaining $e_2,\dots,e_d$ are not unique, but span the subspace orthogonal to $e_1$ (they are still pairwise orthogonal because the matrix is symmetric and real). One way to do this is for $j=2,\dots,d$

$$e_{j1}=\frac{1}{\sqrt{2}},e_{jj}=-\frac{1}{\sqrt{2}},e_{jk}=0\forall k\notin\{1,j\}$$

You can then create an infinite number of solutions by arbitrarily rotating this one (in $d-1$ dimesnions). $(\tilde{e}_2,\dots,\tilde{e}_d)=(e_2,\dots,e_d)R$.

$\endgroup$
2
$\begingroup$

An eigenvector is just a statistic. You can get copies of almost any useful statistic from a wide variety of different data, as long as they are similar in the relevant manner.

The eigenvector represents a particular linear relationship between the variables. As long as you use variables with the same units and what appears to be the same linear relationship, after normalization, you will get the same eigenvector.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy