2
$\begingroup$

I have 10000 samples, each of which has 100 features. To visualize this high dimensional sample, I use 2 component PCA. Here is a the result: enter image description here Here, results are colored based on a target value for each sample. Then, I tried to draw a new sample, and plot it on top of the previous visualization using

from sklearn.decomposition import PCA

pca = PCA(n_components=2)
feature_space_2d = pca.fit_transform(np.vstack((sample, new_draw)))

When I tried to compare these two, I realized that the visualization of original 10000 samples have essentially been rotated . You can see newly drawn sample at top left (please discard other three black points).

enter image description here

  • I understand that adding a point would change the PCA output, but it seems so drastically to me. Would some one be able to shed some light on it?
  • Is there a way to apply the same PCA transform as we did on the original samples to any drawn samples?

Thanks!

Improve this question
$\endgroup$
2
  • $\begingroup$ I'm not sure I understand your plots correctly. What does the color represent? A continuous value, and you are trying to solve a regression problem? $\endgroup$
    – FirefoxMetzger
    Jul 18, 2019 at 18:19
  • $\begingroup$ no, Its a training set, and I want to visualize it. I have 10000 training data, each has 100 dimension, and we have a target value. Eventually I will use it for something but my question at this point only deals with the training set itself. So, colors just represent the target value. $\endgroup$
    – Blade
    Jul 18, 2019 at 18:23

1 Answer 1

Reset to default
3
$\begingroup$

Is it rotated or flipped (like in a mirror)? The latter happens when you use eigen analysis to do PCA. You're essentially plotting the eigenvectors, and they are invariant to the sign. The sign can flip arbitrarily when you run eigenvalue routines, and it doesn't matter but can produce "flipped" plots

Here's the horizontally flipped second image, compare it to the first picture: enter image description here

Improve this answer
$\endgroup$
2
  • $\begingroup$ Teşekkürler arkadaşım! Do you happen to know how to prevent this in python? $\endgroup$
    – Blade
    Jul 18, 2019 at 18:43
  • 1
    $\begingroup$ You mean statsmodels PCA? I think it uses SVD inside, and does happen to have this "issue." I don't know if it's possible to do through API. I simply standardize the sign. For your plotting you could simply use the sign of the first element of the PCA component to normalize the signs in two plots. Look at your plots, they seem to be flipped over the horizontal axis. $\endgroup$
    – Aksakal
    Jul 18, 2019 at 18:45

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.

Not the answer you're looking for? Browse other questions tagged or ask your own question.