0
$\begingroup$

I have got the following PCA plot. I am not able to interpret the two seemingly orthogonal axes in this PCA plot.

  1. Do they imply something about the data?
  2. Why did PCA not take these two axes as the first two PCA dimensions?

enter image description here

$\endgroup$
  • $\begingroup$ 1) They do imply things about your data. You can do clustering and explain variations with respect to dimension 1 and 2. 2) From your plot labels it seems PCA took those 2 principal dimensions. Provided you share sample data and code you can get more help. $\endgroup$ – cbo Jun 28 at 16:41
  • $\begingroup$ Thanks, @cbo. The data that I can not share has around 8591 features. That is, the data is originally of 8591 dimensions. Here I have plotted the first two PCA dimensions. So the two apparent axes are not the PCA dimensions. $\endgroup$ – Soumitra Jun 28 at 16:55
  • $\begingroup$ The simplified code looks like: cols = df.columns; from sklearn.decomposition import PCA; pca = PCA(n_components=2); projected = pca.fit_transform(df.values); cols['dim.one'] = projected[:,0]; cols['dim.two'] = projected[:,1]; ggplot(cols, aes(x=dim.one, y=dim.two)) + geom_point() $\endgroup$ – Soumitra Jun 28 at 16:57
0
$\begingroup$

PCA seems to have separated clusters in your data. It is working as intended. However, PCA has limitations.

Principal components are eigenvectors of the covariance matrix of all of your data. They capture direction of the greatest variance.

PCA looks at the aggregate data, so finds the direction along which all data seem to vary the greatest, not just one cluster. Basically, this would average out the directions of variances of individual clusters.

I'll use an example with clean data here:

import numpy as np
import matplotlib.pyplot as plt

a = np.stack((np.arange(20), np.zeros(20)))  # horizontal cluster
b = np.stack((np.zeros(20), np.arange(20)))  # vertical cluster
c = np.hstack((a, b))                        # combined
plt.plot(a[0], a[1], b[0], b[1])

cov = np.cov(c)
_, eig = np.linalg.eig(cov)
plt.arrow(0, 0, *(5 * eig[0]))
plt.arrow(0, 0, *(5 * eig[1]))
plt.xlim(left=-5)
plt.title('Original')


plt.figure()
a_ = np.linalg.inv(eig) @ a
b_ = np.linalg.inv(eig) @ b
plt.plot(a_[0], a_[1], b_[0], b_[1])
plt.title('Transformed')

Original data

Above, you can see that there are two "clusters" varying in perpendicular directions (orange, blue). The calculated eigenvectors (black) find the average direction of greatest variance since they are looking at all of the data together. If half the data are going up-down, and the other half are going right-left, then the average direction will be a diagonal.

Transformed

$\endgroup$
  • 1
    $\begingroup$ Thanks, @hazrmard. With your example, my plot makes more sense. I could not upvote because I have less than 15 reputations. I have accepted the answer though. Regards. $\endgroup$ – Soumitra Jul 1 at 14:42

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.