I'm reading this interesting paper on application of ICA to gene expression data.
The authors write:
[T]here is no requirement for PCA components to be statistically independent.
That is true, but the PCs are orthogonal, are they not?
I am a bit fuzzy as to what is the relationship between statistical inedpendence and orthogonality or linear independence.
It is worth noting that while ICA also provides a linear decomposition of the data matrix, the requirement of statistical independence implies that the data covariance matrix is decorrelated in a non-linear fashion, in contrast to PCA where the decorrelation is performed linearly.
I don't understand that. How does lack of linearity follow from statistical independence?
Question: how does statistical independence of components in ICA relate to linear independence of components in PCA?