I could use a little bit of help in understanding a concept with regards to ICA:
ICA decomposes a multivariate signal into 'independent' components through 1. orthogonal rotation and 2. maximizing statistical independence between components in some way - one method used is to maximize non-gaussianity (kurtosis). That being said, ICA assumes that the multivariate signal is a mixture of independent, non-gaussian components, so I understand that independence is assumed in the model.
From CLT we know that a linear combination of independent RV's is more Gaussian than the original RV's, but why and how does this relate 'independence' with kurtosis? Does it hold that if two vectors/RV's that are orthogonal and have non-normal distributions (or perhaps one at most has a normal distribution) are statistically independent? Maybe I'm missing something here.
Thanks in advance. Feel free to correct me if I've made an error above.
This is a great passage (user777's post) and I've come across it many times. Though after lots of searching, I have not found an explicitly clear answer to my question - this passage does not satisfy my question either.
I'm convinced this is so because the independence is assumed in order to make ICA valid. We find linear combinations of sources that are as non-gaussian as possible (via some measure like kurtosis) yet still combine to form our original, more-gaussian signal - then these uncorrelated sources, by CLT (and the inital assumptions of ICA), are simply assumed to be independent; in reality we will never know with absolute certainty if the uncorrelated sources are truly independent, we can only assume it via the theory, similarly to how we assume normality or independence of variables quite broadly with other statistical methods.
I suppose that if we use an objective criteria like minimizing the Mutual Information this could be more convincing of independence, but then we aren't putting as much emphasis on the non-gaussianity. I'd be very interested in seeing how the independent components differ when computing ICA via kurtosis maximization versus Mutual Information minimization.