4
$\begingroup$

$X = AS$ where $A$ is my mixing matrix and each column of $S$ represents my sources. $X$ is the data I observe.

If the columns of $S$ are independent and Gaussian, will the components of PCA be extremely similar to that of ICA? Is this the only requirement for the two methods to coincide?

Can someone provide an example of this being true when the $cov(X)$ isn't diagonal?

$\endgroup$
  • $\begingroup$ I'd love to see a mathematical answer here -- something that starts from a derivation and demonstrates concretely where the two diverge and in what general case they are equivalent. $\endgroup$ – jvriesem Nov 30 '16 at 18:15
3
$\begingroup$

My answer here may help you.

I believe that PCA will be equivalent to ICA, only when the independent components are orthogonal to begin with. PCA gives the independent components of your data, only in so far as an orthogonal transformation might.

$\endgroup$
3
$\begingroup$

PCA will be equivalent to ICA if all the correlations in the data are limited to second-order correlations and no higher-order correlations are found. Said another way, when the covariance matrix of the data can explain all the redundancies present in the data, ICA and PCA should return same components.

$\endgroup$
-1
$\begingroup$

Putting aside the issue whether 'extremely similar' (please define what you mean by this; do you mean within the sampling error?) is the same as 'coincide', I believe the answer is no. To be equivalent, they must imply each other in both directions. If outputs are independent as in ICA (but not Gaussian, because ICA does not work for (circular) Gaussian data), then they are uncorrelated as in PCA. The converse is not always true. So one can only compare both methods for non-Gaussian data. For these, since both methods operate on different moments, they are not equivalent. ICA gives generally oblique components, while PCA components are always orthogonal. Therefore, orthongality (i.e., E(X'.X)=0) is a necessary condition for equivalence. But E(Xi'.Xj)=0 does not imply E(Xi'^2.Xj)=0, E(Xi'. Xj^2)=0, etc. for all higher-order moments and n-tuples to vanish as in ICA, so it is not a sufficient condition.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.