Cross Validated is a question and answer site for people interested in statistics, machine learning, data analysis, data mining, and data visualization. It only takes a minute to sign up.
Sign up to join this community
In the book by Aapo Hyvärinen, it is shown that:
Where z is the white vector of a data matrix x, s are the IC's and à is the mixing matrix of the whitened data matrix z. My question is: If the matrix à is orthonormal as show above, does it implies that s is orthogonal as $\\E(ss^T)$ has to be $\\I$? If so should the IC's in this case be orthogonal?
$E[ss^T]$ is $I$ via the assumptions of ICA, i.e. $s_i$ are statistically independent basis signals. $E[zz^T]$ is also $I$ since we know that it is white. Knowing the two, we can say that $\tilde A$ is orthonormal. Conversely, if we know that $\tilde A$ is orthonormal, we can multiply the equation with $\tilde A^T$ and $\tilde A$ from left and right respectively and conclude that $E[ss^T]=I$: $$\tilde AE[ss^T]\tilde A^T=I\rightarrow \underbrace{\tilde A^T\tilde A}_IE[ss^T]\underbrace{\tilde A^T\tilde A}_I=\underbrace{\tilde A^T\tilde A}_I\rightarrow E[ss^T]=I$$
If you run ICA without pre-decorrelating the data, as may be done with the Infomax/Maximum Likelihood algorithm (by me in 1995) then the resulting independent component transform is non-orthogonal. The point about ICA is that it is a non-orthogonal decorrelating transform who's solution is constrained by higher-order statistics. You mustn't confuse orthogonality (which is a geometric property of the matrix transform) with decorrelation (which is a statistical property of the transformed data). In PCA, which is the orthonormal decorrelating transform, the two coincide.
