0
$\begingroup$

I will create an example to explain my question. Say I have two independent data clusters in a 3D space (x,y,z). The two data clusters, Cluster A and B, are very distant to each other, such that running PCA will results in the first component explaining this distance. These two clusters also have different magnitude of "clusterness".

I ran PCA to find whether Cluster A or B is more clustered, and plotted PC1 vs. PC2. As stated before, PC1 explained the physical distance between Cluster A and B. Along PC2, I found that Cluster A data exhibited a linear variation, while Cluster B data are clustered together.

By using this fact alone, can I conclude that since the data of Cluster A are differentiated in PC2, then Cluster B is relatively more tightly clustered than Cluster A?

I understand that PC1, PC2 and PC3 are orthogonal. However, try to look at a 3D problem. If two different clusters have the same length in PC1, it means PC1 is not useful to distinguish which of the two is more clustered. But in PC2, the less clustered group will always be explained by PC2 (because by definition PC2 explains more variance than PC3). Hence, in this case PC3 is not needed because PC1 and PC2 are sufficient to infer this observation.

What if we extend it beyond 3D? If I have hundreds of variables, will PC1 vs. PC2 still be able to distinguish the magnitude of "clusterness" of two indepedent data sets?

For example, this is a PC1 vs. PC2 plot of data with hundreds and thousands of variables. The eigenvalue for PC2 is 3.5, while for PC3 is 2.2.

PC1 vs. PC2

Just from the above picture, am I allowed to conclude that the right group is more clustered than the left group?

$\endgroup$
  • $\begingroup$ While not always necessary, it's also rarely harmful to include a visual aid, and a plot of PC1 vs. PC2 shouldn't be too hard to produce. $\endgroup$ – AkselA Jul 28 '18 at 10:10
  • $\begingroup$ Obviously, the answer to your question is no, generally. PC2 is only one direction, and the direction driven by PC1 (because it's perpendicular). But clusters could be oblong, differently rotated, etc., and you may not conclude about their density without assumptions looking only at one PC. $\endgroup$ – ttnphns Jul 28 '18 at 10:18
  • $\begingroup$ I have added an example of PC1 vs. PC2 I created. The data has hundreds of thousands of variables, but what I'm actually interested is which of the two is more clustered together relative to the other. $\endgroup$ – Rudy Winono Jul 28 '18 at 10:19
  • 1
    $\begingroup$ To assess cluster density search "internal clustering criterions", "internal cluster validation indices" on this site, and especially Silhouette width index. $\endgroup$ – ttnphns Jul 28 '18 at 10:19
  • $\begingroup$ @ttnphns: I understand that PC1, PC2 and PC3 are orthogonal. However, try to look at a 3D problem. If two different clusters have the same length in PC1, it means PC1 is not useful to distinguish which of the two is more clustered. But in PC2, the less clustered group will always be explained by PC2 (because by definition PC2 explains more variance than PC3). Hence, in this case PC3 is not needed because PC1 and PC2 are sufficient to infer this observation. $\endgroup$ – Rudy Winono Jul 28 '18 at 10:25

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.