9
$\begingroup$

Assume that you have made a PCA analysis and you got your eigenvectors inside the projection matrix $W$. If you project your data $X$ with $W$, then you get the desired projected dimension.

But PCA and ICA are opposite. If we look at this picture. PCA is projecting the most principal data, while ICA is projecting the most non commonly data.

My question is simple: Is it possible to turn PCA into ICA by rotating the eigenvectors in some angles? If yes, how?

enter image description here

$\endgroup$
3
  • 1
    $\begingroup$ Are you asking how to obtain ICA solution out of PCA solution without having the ICA solution at hand? $\endgroup$
    – ttnphns
    Commented Apr 10, 2023 at 21:04
  • 1
    $\begingroup$ @ttnphns Om asking if it's possible to rotate the eigenvectors of SVD so it will become ICA. My goal is to separate classes from data. $\endgroup$
    – euraad
    Commented Apr 10, 2023 at 21:34
  • 1
    $\begingroup$ @ttnphns yes I know that LDA exist, but it's not the same $\endgroup$
    – euraad
    Commented Apr 10, 2023 at 21:34

1 Answer 1

11
$\begingroup$

No, in general, you can't rotate the principal components to obtain ICA. One of the defining traits of PCA is that the component directions are orthogonal. If you rotate the principal components, they'll still be orthogonal after the rotation. (This is because a rotation matrix is an orthogonal transformation.) Almost always, ICA components are not orthogonal, so rotation of principal components will not recover ICA components.

The only caveat is trivial -- if the ICA directions are orthogonal to begin with, then they will still be orthogonal after rotation, for the same reasons.

$\endgroup$
7
  • 2
    $\begingroup$ +1 "if the ICA directions are orthogonal to begin with, then they will still be orthogonal after rotation, for the same reasons." And you would therefore not need PCA in that circumstance. :) $\endgroup$
    – Alexis
    Commented Apr 10, 2023 at 17:01
  • 1
    $\begingroup$ Thank you for the answer. Do you know what to use if I want to project and only see the most unique data ? $\endgroup$
    – euraad
    Commented Apr 10, 2023 at 17:30
  • 1
    $\begingroup$ Comment boxes are limited, so it would be best to ask that in its own Question, with some elaboration of what “most unique data” is, in particular. $\endgroup$
    – Sycorax
    Commented Apr 10, 2023 at 17:36
  • 1
    $\begingroup$ @Sycorax ICA return a projection matrix that project the data so only unique data can be view. As I understand the answer, I cannot take the eigenvectors from e.g SVD and change those so they project the data matrix in a different way. $\endgroup$
    – euraad
    Commented Apr 10, 2023 at 19:10
  • 1
    $\begingroup$ @euraad the plot of the data appears to arise from one of two lines, plus some noise. So the “typical” data are close to one of the ICA vectors. The unusual data are far from both vectors. $\endgroup$
    – Sycorax
    Commented Apr 10, 2023 at 20:22

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.